 |
Medication Safety Officers Society 3759 Members Strong A society of healthcare professionals dedicated to improving medication safety in healthcare organizations |
Audit and feedback is a strategy used to improve healthcare delivery and promote the implementation of evidence-based practice.1 A recipient, at individual, team or unit level, is provided with summary data of their performance over a specified period of time to act as a stimulus for behaviour change and quality improvement. Audit and feedback is used across clinical conditions and settings around the world. When used correctly, it is effective, scalable and relatively inexpensive.1 However, every healthcare improvement strategy has the potential to have a ‘dark side’2; the potential for adverse effects to arise from providing audit and feedback warrants careful consideration. How can we ensure that audit and feedback initiatives will lead to more benefits than harms?
Catlow et al3 in this edition of the journal investigated the unintended consequences of audit and feedback using an innovative approach. They undertook...
When a hospitalised patient begins to deteriorate clinically, prompt detection and early intervention by trained clinicians is critical for preventing in-hospital cardiac arrest. This is a common event that affects nearly 300 000 patients annually in the USA1 and is associated with a high risk of death and neurological disability among survivors. A broad range of strategies have been proposed for the early detection and management of the deteriorating patient. Early warning systems (EWS) are an example of one strategy. EWS use clinical prediction models to identify patients who are likely to be deteriorating, with triggers and protocols for detecting and escalating care for such patients.2 3
The use of EWS to monitor patients is now widespread, and two papers in this month’s issue of BMJ Quality & Safety address this topic in different ways. First, Blythe and colleagues conducted a scoping review of...
Optimising patient safety in the hospital setting remains a significant challenge for modern healthcare. Substantial efforts have been made to eradicate patient harm events since the 1999 publication of To Err is Human: Building a Safer Health System.1 Nevertheless, a recent meta-analysis of 94 adult inpatient studies concluded that 8.6 hospital harm events occur for every 100 patient admissions, with over half (52.6%) judged to be preventable.2 Estimates in high-risk paediatric settings suggest a rate as high as 40 patient harm events per 100 admissions.3–5 Although patient harms within the subset known as hospital-acquired conditions in the USA have declined in the adult and paediatric populations,6 7 multicentred, longitudinal studies of adult2 8 and paediatric inpatients9 have shown no significant improvement in overall harm rates over the past 20...
Audit and feedback (A&F) interventions improve patient care but may result in unintended consequences. To evaluate plausible harms and maximise benefits, theorisation using logic models can be useful. We aimed to explore the adverse effects of colonoscopy A&F using a feedback intervention theory (FIT) dark logic model before the National Endoscopy Database Automated Performance Reports to Improve Quality Outcomes Trial study.
MethodsWe undertook a qualitative study exploring A&F practices in colonoscopy. Interviews were undertaken with endoscopists from six English National Health Service endoscopy centres, purposively sampled for professional background and experience. A thematic framework analysis was performed, mapping paradoxical effects and harms using FIT and the theory of planned behaviour.
ResultsData saturation was achieved on the 19th participant, with participants from nursing, surgical and medical backgrounds and a median of 7 years’ experience.
When performance was below aspirational targets participants were falsely reassured by social comparisons. Participants described confidence as a requirement for colonoscopy. Negative feedback without a plan to improve risked reducing confidence and impeding performance (cognitive interference). Unmet targets increased anxiety and prompted participants to question messages’ motives and consider gaming.
Participants described inaccurate documentation of subjective measures, including patient comfort, to achieve targets perceived as important. Participants described causing harm from persevering to complete procedures despite patient discomfort and removing insignificant polyps to improve detection rates without benefiting the patient.
ConclusionOur dark logic model highlighted that A&F interventions may create both desired and adverse effects. Without a priori theorisation evaluations may disregard potential harms. In colonoscopy, improved patient experience measures may reduce harm. To address cognitive interference the motivation of feedback to support improvement should always be clear, with plans targeting specific behaviours and offering face-to-face support for confidence.
Trial registration numberUnrecognised changes in a hospitalised patient’s clinical course may lead to a preventable adverse event. Early warning systems (EWS) use patient data, such as vital signs, nursing assessments and laboratory values, to aid in the detection of early clinical deterioration. In 2018, an EWS programme was deployed at an academic hospital that consisted of a commercially available EWS algorithm and a centralised virtual nurse team to monitor alerts. Our objective was to understand the nursing perspective on the use of an EWS programme with centralised monitoring.
MethodsWe conducted and audio-recorded semistructured focus groups during nurse staff meetings on six inpatient units, stratified by alert frequency (high: >100 alerts/month; medium: 50–100 alerts/month; low: <50 alerts/month). Discussion topics included EWS programme experiences, perception of EWS programme utility and EWS programme implementation. Investigators analysed the focus group transcripts using a grounded theory approach.
ResultsWe conducted 28 focus groups with 227 bedside nurses across all shifts. We identified six principal themes: (1) Alert timeliness, nurses reported being aware of the patient’s deterioration before the EWS alert, (2) Lack of accuracy, nurses perceived most alerts as false positives, (3) Workflow interruptions caused by EWS alerts, (4) Questions of actionability of alerts, nurses were often uncertain about next steps, (5) Concerns around an underappreciation of core nursing skills via reliance on the EWS programme and (6) The opportunity cost of deploying the EWS programme.
ConclusionThis qualitative study of nurses demonstrates the importance of earning user trust, ensuring timeliness and outlining actionable next steps when implementing an EWS. Careful attention to user workflow is required to maximise EWS impact on improving hospital quality and patient safety.
Hospital patients experiencing clinical deterioration are at greater risk of adverse events. Monitoring patients through early warning systems is widespread, despite limited published evidence that they improve patient outcomes. Current limitations including infrequent or incorrect risk calculations may be mitigated by integration into electronic medical records. Our objective was to examine the impact on patient outcomes of systems for detecting and responding to real-time, automated alerts for clinical deterioration.
MethodsThis review was conducted according to the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews checklist. We searched Medline, CINAHL and Embase for articles implementing real-time, automated deterioration alerts in hospitalised adults evaluating one or more patient outcomes including intensive care unit admission, length of stay, in-hospital cardiopulmonary arrest and in-hospital death.
ResultsOf 639 studies identified, 18 were included in this review. Most studies did not report statistically significant associations between alert implementation and better patient outcomes. Four studies reported statistically significant improvements in two or more patient outcomes, and were the only studies to directly involve the patient’s clinician. However, only one of these four studies was robust to existing trends in patient outcomes. Of the six studies using robust study designs, one reported a statistically significant improvement in patient outcomes; the rest did not detect differences.
ConclusionsMost studies in this review did not detect improvements in patient outcomes following the implementation of real-time deterioration alerts. Future implementation studies should consider: directly involving the patient’s physician or a dedicated surveillance nurse in structured response protocols for deteriorating patients; the workflow of alert recipients; and incorporating model features into the decision process to improve clinical utility.
Diagnostic errors (DxEs) are an understudied source of patient harm in children rarely captured in current adverse event reporting systems. Applying electronic triggers (e-triggers) to electronic health records shows promise in identifying DxEs but has not been used in the emergency department (ED) setting.
ObjectivesTo assess the performance of an e-trigger and subsequent manual screening for identifying probable DxEs among children with unplanned admission following a prior ED visit and to compare performance to existing incident reporting systems.
Design/methodsRetrospective single-centre cohort study of children ages 0–22 admitted within 14 days of a previous ED visit between 1 January 2018 and 31 December 2019. Subjects were identified by e-trigger, screened to identify cases where index visit and hospital discharge diagnoses were potentially related but pathophysiologically distinct, and then these screened-in cases were reviewed for DxE using the SaferDx Instrument. Cases of DxE identified by e-trigger were cross-referenced against existing institutional incident reporting systems.
ResultsAn e-trigger identified 1915 unplanned admissions (7.7% of 24 849 total admissions) with a preceding index visit. 453 (23.7%) were screened in and underwent review using SaferDx. 92 cases were classified as likely DxEs, representing 0.4% of all hospital admissions, 4.8% among those selected by e-trigger and 20.3% among those screened in for review. Half of cases were reviewed by two reviewers using SaferDx with substantial inter-rater reliability (Cohen’s =0.65 (95% CI 0.54 to 0.75)). Six (6.5%) cases had been reported elsewhere: two to the hospital’s incident reporting system and five to the ED case review team (one reported to both).
ConclusionAn e-trigger coupled with manual screening enriched a cohort of patients at risk for DxEs. Fewer than 10% of DxEs were identified through existing surveillance systems, suggesting that they miss a large proportion of DxEs. Further study is required to identify specific clinical presentations at risk of DxEs.
Direct observation is valuable for identifying latent threats and elucidating system complexity in clinical environments. This approach facilitates prospective risk assessment and reveals workarounds, near-misses and recurrent safety problems difficult to diagnose retrospectively or via outcome data alone. As observers are an instrument of data collection, developing effective and comprehensive observer training is critical to ensuring the reliability of the data collection and reproducibility of the research. However, methodological rigour for ensuring these data collection properties remains a key challenge in direct observation research in healthcare. Although prior literature has offered key considerations for observational research in healthcare, operationalising these recommendations may pose a challenge and unless guidance is also provided on observer training. In this article, we offer guidelines for training non-clinical observers to conduct direct observations including conducting a training needs analysis, incorporating practice observations and evaluating observers and inter-rater reliability. The operationalisation of these guidelines is described in the context of a 5-year multisite observational study investigating technology integration in the operating room. We also discuss novel tools developed during the course our project to support data collection and examine inter-rater reliability among observers in direct observation studies.
The effectiveness of audit and feedback (A&F) interventions to improve compliance to healthcare guidelines is supported by randomised controlled trials (RCTs) and meta-analyses of RCTs. However, there is currently a knowledge gap on their cost-effectiveness.
ObjectiveWe aimed to assess whether A&F interventions targeting improvements in compliance to recommended care are economically favourable.
MethodsWe conducted a systematic review including experimental, observational and simulation-based economic evaluation studies of A&F interventions targeting healthcare providers. Comparators were a ‘do nothing’ strategy, or any other intervention not involving A&F or involving a subset of A&F intervention components. We searched MEDLINE, CINAHL, CENTRAL, Econlit, EMBASE, Health Technology Assessment Database, MEDLINE, NHS Economic Evaluation Database, ABI/INFORM, Web of Science, ProQuest and websites of healthcare quality associations to December 2021. Outcomes were incremental cost-effectiveness ratios, incremental cost-utility ratios, incremental net benefit and incremental cost-benefit ratios. Pairs of reviewers independently selected eligible studies and extracted relevant data. Reporting quality was evaluated using CHEERS (Consolidated Health Economic Evaluation Reporting Standards). Results were synthesised using permutation matrices for all studies and predefined subgroups.
ResultsOf 13 221 unique citations, 35 studies met our inclusion criteria. The A&F intervention was dominant (ie, at least as effective with lower cost) in 7 studies, potentially cost-effective in 26 and was dominated (ie, the same or less effectiveness and higher costs) in 2 studies. A&F interventions were more likely to be economically favourable in studies based on health outcomes rather than compliance to recommended practice, considering medical costs in addition to intervention costs, published since 2010, and with high reporting quality.
DiscussionResults suggest that A&F interventions may have a high potential to be cost-effective. However, as is common in systematic reviews of economic evaluations, publication bias could have led to an overestimation of their economic value.
Healthcare quality and safety span multiple topics across the spectrum of academic and clinical disciplines. Keeping abreast of the rapidly growing body of work can be challenging. In this series, we provide succinct summaries of selected relevant studies published in the last several months. Some articles will focus on a particular theme, whereas others will highlight unique publications from high-impact medical journals.
