In this issue of BMJ Quality and Safety, two articles consider how patients’ opinions of care can be collected, analysed and used to inform healthcare delivery. In the first of the two studies, Lee and colleagues examine how written patient experience comments feedback is used in the National Health Service (NHS).1 Uniquely, the authors focus their investigation on the way in which Boards of Directors use patient experience information to monitor and improve care.
The second study, conducted by Griffiths and Leaver, illustrates how computational tools could automate the collection and analysis of patient experience data. The authors’ system scrapes comments from social media websites and machine learning algorithms convert this unstructured information (ie, free text comments) into a zero-to-five ‘star’ rating, which they suggest could help prioritise hospital inspections.2
Lee and colleagues focused their investigation on two NHS Foundation Trusts with experience in collecting patient feedback...
Patients and family members are the closest observers of care, with their focus on one patient in one hospital bed or clinic room. While patient-centred care is a well-accepted domain of quality, our ability to gather and use the patient and family member perspective to improve care is still relatively limited.
One potential source of data for this perspective is online reviews of care from social media sources such as Yelp, the online review site, and Facebook, the popular social networking site. These online sources of consumer-generated content likely exert greater influence than the scientifically validated measures of quality published on government-sponsored public reporting websites, such as the Consumer Assessment of Healthcare Providers and Systems (CAHPS) and Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) measures. The influence of online sources of patient-generated assessments of care likely reflects a number of factors: Yelp and Facebook receive far greater...
Surgical performance is a function of technical and non-technical skills, the latter of which encompasses both cognitive and interpersonal skills.1 The important role of intraoperative non-technical skills in determining surgical safety and outcomes has been increasingly recognised in recent years. To that end, in this issue, Sexton et al2 evaluate the non-technical skills of teamwork and situation awareness. Specifically, they examine the effect of team anticipation on operative time and cognitive load in the environment of robotic surgery. Through analysis of video and audio from 12 robot-assisted radical prostatectomies, they were able to calculate anticipation ratios (per cent of requests that were non-verbal) and investigate the impact on surgical performance. While increased anticipation led to decreased operating room time, it also increased the assistant surgeon’s cognitive workload.2 Prior work has shown that increased cognitive load is associated with errors and poorer performance, which demonstrates...
Although previous research suggests that different kinds of patient feedback are used in different ways to help improve the quality of hospital care, there have been no studies of the ways in which hospital boards of directors use feedback for this purpose.Objectives
To examine whether and how boards of directors of hospitals use feedback from patients to formulate strategy and to assure and improve the quality of care.Methods
We undertook an in-depth qualitative study in two acute hospital National Health Service foundation trusts in England, purposively selected as contrasting examples of the collection of different kinds of patient feedback. We collected and analysed data from interviews with directors and other managers, from observation of board meetings, and from board papers and other documents.Results
The two boards used in-depth qualitative feedback and quantitative feedback from surveys in different ways to help develop strategies, set targets for quality improvement and design specific quality improvement initiatives; but both boards made less subsequent use of any kinds of feedback to monitor their strategies or explicitly to assure the quality of services.Discussion and conclusions
We have identified limitations in the uses of patient feedback by hospital boards that suggest that boards should review their current practice to ensure that they use the different kinds of patient feedback that are available to them more effectively to improve, monitor and assure the quality of care.
The Care Quality Commission (CQC) is responsible for ensuring the quality of healthcare in England. To that end, CQC has developed statistical surveillance tools that periodically aggregate large numbers of quantitative performance measures to identify risks to the quality of care and prioritise its limited inspection resource. These tools have, however, failed to successfully identify poor-quality providers. Facing continued budget cuts, CQC is now further reliant on an ‘intelligence-driven’, risk-based approach to prioritising inspections and a new effective tool is required.Objective
To determine whether the near real-time, automated collection and aggregation of multiple sources of patient feedback can provide a collective judgement that effectively identifies risks to the quality of care, and hence can be used to help prioritise inspections.Methods
Our Patient Voice Tracking System combines patient feedback from NHS Choices, Patient Opinion, Facebook and Twitter to form a near real-time collective judgement score for acute hospitals and trusts on any given date. The predictive ability of the collective judgement score is evaluated through a logistic regression analysis of the relationship between the collective judgement score on the start date of 456 hospital and trust-level inspections, and the subsequent inspection outcomes.Results
Aggregating patient feedback increases the volume and diversity of patient-centred insights into the quality of care. There is a positive association between the resulting collective judgement score and subsequent inspection outcomes (OR for being rated ‘Inadequate’ compared with ‘Requires improvement’ 0.35 (95% CI 0.16 to 0.76), Requires improvement/Good OR 0.23 (95% CI 0.12 to 0.44), and Good/Outstanding OR 0.13 (95% CI 0.02 to 0.84), with p<0.05 for all).Conclusions
The collective judgement score can successfully identify a high-risk group of organisations for inspection, is available in near real time and is available at a more granular level than the majority of existing data sets. The collective judgement score could therefore be used to help prioritise inspections.
Hospital care costs are high while quality varies across hospitals. Patient satisfaction may be associated with better clinical quality, and social media ratings may offer another opportunity to measure patient satisfaction with care.Objectives
To test if Facebook user ratings of hospitals are associated with existing measures of patient satisfaction, cost and quality.Research design
Data were obtained from Centers for Medicare and Medicaid Services Hospital Compare, the Hospital Inpatient Prospective Payment System impact files and the Area Health Resource File for 2015. Information from hospitals’ Facebook pages was collected in July 2016. Multivariate linear regression was used to test if there is an association between Facebook user ratings (star rating and adjusted number of ‘likes’) and Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) patient satisfaction measures, the 30-day all-cause readmission rate, and the Medicare spending per beneficiary (MSPB) ratio.Subjects
One hundred and thirty-six acute care hospitals in New York State in 2015.Results
An increase in the Facebook star rating is associated with significant increases in 21/23 HCAHPS measures (p≤0.003). An increase in the adjusted number of ‘likes’ is associated with very small increases in 3/23 HCAHPS measures (p<0.05). Facebook user ratings are not associated with the 30-day all-cause readmission rate or the Medicare spending per beneficiary ratio.Conclusions
Results demonstrate an association between HCAHPS patient satisfaction measures and Facebook star ratings. Adjusted number of ‘likes’ may not be a useful measure of patient satisfaction.
The growing use of social media creates opportunities for patients and families to provide feedback and rate individual healthcare providers. Whereas previous studies have examined this emerging trend in hospital and physician settings, little is known about user ratings of nursing homes (NHs) and how these ratings relate to other measures of quality.Objective
To examine the relationship between Facebook user-generated NH ratings and other measures of NH satisfaction/experience and quality.Methods
This study compared Facebook user ratings of NHs in Maryland (n=225) and Minnesota (n=335) to resident/family satisfaction/experience survey ratings and the Centers for Medicare and Medicaid (CMS) 5-star NH report card ratings.Results
Overall, 55 NHs in Maryland had an official Facebook page, of which 35 provided the opportunity for users to rate care in the facility. In Minnesota, 126 NHs had a Facebook page, of which 78 allowed for user ratings. NHs with higher aid staffing levels, not affiliated with a chain and located in higher income counties were more likely to have a Facebook page. Facebook ratings were not significantly correlated with the CMS 5-star rating or survey-based resident/family satisfaction ratings.Conclusions
Given the disconnect between Facebook ratings and other, more scientifically grounded measures of quality, concerns about the validity and use of social media ratings are warranted. However, it is likely consumers will increasingly turn to social media ratings of NHs, given the lack of consumer perspective on most state and federal report card sites. Thus, social media ratings may present a unique opportunity for healthcare report cards to capture real-time consumer voice.
Despite evidence against the use of antimicrobials for asymptomatic bacteriuria (ASB), they are frequently prescribed leading to unnecessary adverse events. Prior studies have shown that reducing unnecessary urine cultures (UCs) results in decreased antimicrobial utilisation for ASB. Emergency departments (EDs) submit the largest volume of UCs, yet efforts to limit overordering in this patient setting have had limited success.Methods
A new two-step model of care for urine collection, using a novel UC collection container, was implemented in the ED of a large community hospital. The collection system contains a preservative allowing UCs to be held at room temperature for up to 48 hours before processing. UCs were collected by front-line staff, but only processed in the microbiology lab if requested by ED physicians after clinical assessment.Results
Following implementation there was a decrease in the percentage of weekly ED visits associated with a processed UC (5.97% vs 4.68%, p<0.001), a decrease in the percentage of monthly ED visits requiring a callback for positive urine culture (1.84% to 1.12%, p<0.001) and a decrease in antimicrobial prescriptions for urinary indication among admitted patients (20.6% to 10.9%, p<0.01). There was a false omission rate of 1.35% (95% CI 0.7% to 2.2%), yet no identified cases of untreated urinary tract infection (UTI), or significant change in repeat ED visits or ED length of stay.Conclusions
Changing to two-step urine culture ordering in the ED resulted in a decrease in UCs processed, callbacks for positive results and antimicrobial use without evidence of untreated UTIs. This model of care has strong potential to improve the use of hospital resources while minimising detection and inappropriate treatment of ASB.
Robot-assisted surgery (RAS) has changed the traditional operating room (OR), occupying more space with equipment and isolating console surgeons away from the patients and their team. We aimed to evaluate how anticipation of surgical steps and familiarity between team members impacted efficiency.Methods
We analysed recordings (video and audio) of 12 robot-assisted radical prostatectomies. Any requests between surgeon and the team members were documented and classified by personnel, equipment type, mode of communication, level of inconvenience in fulfilling the request and anticipation. Surgical team members completed questionnaires assessing team familiarity and cognitive load (National Aeronautics and Space Administration – Task Load Index). Predictors of team efficiency were assessed using Pearson correlation and stepwise linear regression.Results
1330 requests were documented, of which 413 (31%) were anticipated. Anticipation correlated negatively with operative time, resulting in overall 8% reduction of OR time. Team familiarity negatively correlated with inconveniences. Anticipation ratio, per cent of requests that were non-verbal and total request duration were significantly correlated with the console surgeons’ cognitive load (r=0.77, p=0.006; r=0.63, p=0.04; and r=0.70, p=0.02, respectively).Conclusions
Anticipation and active engagement by the surgical team resulted in shorter operative time, and higher familiarity scores were associated with fewer inconveniences. Less anticipation and non-verbal requests were also associated with lower cognitive load for the console surgeon. Training efforts to increase anticipation and team familiarity can improve team efficiency during RAS.
With great interest we read the article of Flott et al1 describing the challenges of using patient-reported feedback. We recognise the challenges described and performed a bachelor project in the intensive care unit (ICU) in the University Medical Center Groningen (UMCG). We think the results from our project provide a potential promising practical solution to make feedback more useful.
In 2013 the UMCG participated in an independent multicentre study conducted among relatives of ICU patients.2 In the open questions of the questionnaire, more dissatisfaction than expected was found, which fuelled the quest for an alternative, simple and continuous feedback system. In this study we compared the quality and amount of feedback gathered by an oral survey during the first 2 weeks and an app during the consecutive 2 weeks.
Between 20 February and 18 March 2017, patients above 16 years old, listed for discharge from the ICU that day, and...
Resident work hour restrictions have led to the creation of the ‘night float’ to care for the patients of multiple primary teams after hours. These residents are often inundated with acute issues in the numerous patients they cover and are less able to address non-urgent issues that arise at night. Further, non-urgent pages may contribute to physician alarm fatigue and negatively impact patient outcomes.Objective
To delineate the burden of non-urgent paging at night and propose solutions.Methods
We performed a resident review and categorisation of 1820 pages to night floats between September 2014 and December 2014. Both attending and nursing review of 10% of pages was done and compared.Results
Of reviewed pages, 62.1% were urgent and 27.7% were non-urgent. Attending review of random page samples correlated well with resident review. Common reasons for non-urgent pages were non-urgent patient status updates, low-priority order requests and non-critical lab values.Conclusions
A significant number of non-urgent pages are sent at night. These pages likely distract from acute issues that arise at night and place an unnecessary burden on night floats. Both behavioural and systemic adjustments are needed to address this issue. Possible interventions include integrating low-priority messaging into the electronic health record system and use of charge nurses to help determine urgency of issues and batch non-urgent pages.
Since their inception, hospital infection prevention (IP) and Antimicrobial Stewardship Programs (ASP) have worked to deploy interventions to mitigate risk of infection and antimicrobial resistance arising from our usual systems of care.1 They generated advances in quality improvement and patient safety, even before these were recognised fields. In an early evidence report commissioned by the US Agency for Healthcare Research and Quality in 2002, 4 of 11 safety practices with strongest supporting evidence were directly related to IP.2 An updated report in 2013 included six IP/ASP interventions in the top 10 safety strategies ready for widespread adoption.3
Despite the high-quality evidence supporting these IP/ASP interventions, our approach to adding these to our current practice sometimes feels like adding scaffolding to a rickety building. It supports the underlying structure but remove the scaffolding without fixing the building, and it may just come tumbling down. Consider efforts...
The National Health Service (NHS) outcomes frameworks for public health, health and adult social care in England were launched in 2010–2012 with ambitions that they would transform health outcomes in England. However, unprecedented financial pressures in the NHS are compelling changes in government policy designed to make the NHS financially sustainable while ensuring high-quality care for a population with a growing burden of chronic disease and multimorbidity. Radical changes in the way health and care services are commissioned and delivered locally are under way or planned. The models of care emerging are almost without exception predicated on greater integration of public health, health and care services, making the three discrete outcomes frameworks increasingly anomalous. As a centrally funded service working to policies defined by government, the NHS needs a performance framework that clearly articulates national priorities and the systemic changes needed to deliver them. The authors argue for a...