Authors' objectives: Acute inpatient mental health services report high levels of safety incidents. The application of patient safety theory has been sparse, particularly concerning interventions that proactively seek patient perspectives. Develop and evaluate a theoretically based, digital monitoring tool to collect real-time information from patients on acute adult mental health wards about their perceptions of ward safety.

Authors' results and conclusions: Coronavirus disease 2019 restrictions greatly impacted the study. Stakeholder engagement permeated the project. Phase 1 delivered a theory-based, collaboratively designed digital tool for proactive patient safety monitoring. Phase 2 showed that the tool was user friendly and broadly acceptable to patients and staff. The aggregated safety data were infrequently used by staff. Feasibility depended on engaged staff and embedding use of the tool in ward routines. There is strong evidence that an incident leads to increased probability of further incidents within the next 4 hours. This puts a measure on the extent to which social/behavioural contagion persists. There is weak evidence to suggest that an incident leads to a greater use of the WardSonar tool in the following hour, but none to suggest that ward atmosphere predicts future incidents. Therefore, how often patients use the tool seems to send a stronger signal about potential incidents than patients’ real-time reports about ward atmosphere. WardSonar has the potential to provide a valuable route for patients to communicate safety concerns. The WardSonar monitoring tool has a strong patient perspective and uses proactive real-time safety monitoring rather than traditional retrospective data review. Statistical analysis showed substantial and significant variation in the use of the device across wards, both in terms of the likelihood of any submission at a given time and the number of submissions. There were no statistically significant differences in staff perceptions of ward atmosphere or safety culture pre and post WardSonar implementation. Owing to the sample size, analyses were not conducted at the individual ward level. The volume of patient submissions via the monitoring tool corresponded more closely to incidents than ward atmosphere averages. It was lower over time, at night, at weekends, and when there were peaks in incidents. Submission volume decreased over time, but the probability of a submission reporting a better atmosphere increased slightly. Submission volume was higher during the daytime or before an incident and there was weak evidence of increased volume in the hour after an incident. The type of response for both direction and current atmosphere was not sensitive to whether or not there had been an incident. In terms of direction of atmosphere, there were significant differences across wards, but this was the only significant variable. Evenings led to greater likelihood of a worse atmosphere being reported, given that a submission was made. An incident in the previous four hours was strongly predictive of a further incident. An individual ‘stormy’ response or increased volume of submissions within the previous hour had some predictive value regarding a further incident. Qualitative analysis highlighted some data quality issues; for example, staff were more likely to collect submissions during quiet times and very unlikely to collect submissions during busy times; the tool was not used if the device was lost or otherwise out of action. There were some periods of days or weeks on some wards when no submissions were collected. The technology was pragmatically adapted for use within an NHS context. Some connectivity issues were identified, although data inputting did not seem to be affected. Further understanding of relationships between ward atmosphere and staff stress or absence due to sickness, contagion between incidents in the seclusion room and impact on staff and patients on the main ward and implementation processes would inform future implementation of WardSonar and interpretation of WardSonar data.

Authors' methods: Theory-informed mixed-methods study. A prototype digital monitoring tool was developed from a co-design approach, implemented in hospital settings, and subjected to qualitative and quantitative evaluation. Phase 1: scoping review of the literature on patient involvement in safety interventions in acute mental health care; evidence scan of digital technology in mental health contexts; qualitative interviews with mental health patients and staff about perspectives on ward safety. This, alongside stakeholder engagement with advisory groups, service users and health professionals, informed the development processes. Most data collection was virtual. Phase 1 resulted in the technical development of a theoretically based digital monitoring tool that collected patient feedback for proactive safety monitoring. Phase 2: implementation of the tool in six adult acute mental health wards across two UK NHS trusts; evaluation via focused ethnography and qualitative interviews. Statistical analysis of WardSonar data and routine ward data involving construction of an hour-by-hour data set per ward, permitting detailed analysis of the use of the WardSonar tool. A total of 8 patients and 13 mental health professionals participated in Phase 1 interviews; 33 staff and 34 patients participated in Phase 2 interviews. Patients could use a web application (the WardSonar tool) to record real-time perceptions of ward safety. Staff could access aggregated, anonymous data to inform timely interventions. Implementation was limited to two NHS trusts. Coronavirus disease 2019 impacted design processes including stakeholder engagement; implementation; and evaluation of the monitoring tool in routine clinical practice. Higher uptake could enhance validity of the results. Two-phase, mixed-methods design. Focused ethnographic observations on the six wards explored implementation context. Patient and staff perspectives were explored through individual interviews. Data were synthesised using a pen portrait analytical process. Staff perceptions of safety culture and ward atmosphere were examined using a pre- and post-implementation design. Baseline measurements including the EssenCES© (Climate Evaluation Schema; Institute of Forensic Psychiatry and Sex Research, Essen, Germany) scale and the Agency for Healthcare Research and Quality Hospital Survey on Patient Safety Culture were taken at baseline and after a 10-week implementation period. Various statistical models were used to explore relationships between wards, WardSonar use and occurrence of incidents over time, including zero-inflated negative binomial models and ordinal logistic regression. The design matrix was consistent across all models. It consisted of fixed effects to capture time-invariant ward-specific effects, time of day variables and lagged values of incidents of WardSonar responses, depending on whether the model was explaining current incidents or current response. A simple linear trend determined any systematic deviation in use or response over time.

Project Status: Completed

URL for project: https://www.journalslibrary.nihr.ac.uk/programmes/hsdr/NIHR128070

Year Published: 2024

URL for published report: https://www.journalslibrary.nihr.ac.uk/hsdr/UDBQ8402

URL for additional information: English

English language abstract: An English language summary is available

Publication Type: Full HTA

Country: England, United Kingdom

DOI: 10.3310/UDBQ8402

Organisation Name: NIHR Health Services and Delivery Research programme

Contact Address: NIHR Journals Library, National Institute for Health and Care Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK

Contact Name: journals.library@nihr.ac.uk

Contact Email: journals.library@nihr.ac.uk