Evaluating the national rollout of the NHS App in England using qualitative and quantitative methods

Greaves F, Papoutsi C, Kc S, Tewolde S, Reidy C, Gudgin B, Laverty AA, Majeed A, Costelloe C, Powell J
Record ID 32018015616
English
Authors' objectives: In 2019, the NHS App was launched as a ‘digital front door’ to England’s National Health Service, aiming to improve access to primary care, enhance patient experience, save time in general practitioner practices and promote self-care. The NHS App was developed as a ‘digital front door’ to England’s NHS. The initial goals for the app were to improve access to primary care services, improve patient experience, save time in general practitioner (GP) practices and promote self-care. There was no planned national evaluation of this app, which was introduced as a policy priority and constituted a novel complex intervention with potentially wide-ranging impacts on the use of healthcare services. This project provides the only national evaluation of a major component of the central plan to digitally transform the NHS. The roll-out of the NHS App was in line with a general political discourse and drive towards harnessing digital tools for health care and was tied in with the roll-out of the NHS login. The NHS Long Term Plan set out the ambition for all patients in England to have the right to access digital consultations from their GP, while NHS England’s ‘digital first’ strategy, aimed to direct patients away from in-person engagements, through telephone, online or video consulting before face-to-face consultations. Throughout roll-out, the app has been dubbed as serving to ‘ease pressures on GPs’, ‘provide more effective, personalised care’ and ‘freeing up valuable clinician time’. There have been ongoing developments of the NHS App, with additional features and integrations added throughout roll-out, and it continues to be at the forefront of the digital health agenda in government policy. In the spring budget of March 2024, the Chancellor of the Exchequer announced £3.4B of funding for the NHS to invest in digital technology, including the specific aim to make the NHS App a single front door to the NHS for patients. The plans for the NHS App include greater use of messaging with healthcare providers, more preventative ‘keep me healthy’ content and supporting integration with third-party apps and devices. This project aimed to identify and understand the use and acceptability of the NHS App, measure the extent to which it improves patient experience and influences health service access, and assess the impact of the app on reducing demand on NHS services. Much of the evaluation covered the period during and immediately following the COVID-19 pandemic.
Authors' results and conclusions: The qualitative analysis guided by the Non-adoption, Abandonment, Scale-up, Spread and Sustainability framework illustrated the multiple layers of complexity when introducing a constantly shifting technology into a challenging environment such as English general practice, during and after a period of considerable societal turbulence caused by the COVID-19 pandemic. Quantitative work showed there was strong adoption of the NHS App even before the onset of the pandemic, although the introduction of the COVID-19 Pass feature was linked to a fourfold increase in downloads. Analyses by sociodemographic data found higher usage in less-deprived and less ethnically diverse practices, with a generally younger population. There were 25% lower registrations in the most deprived practices (p 
Authors' methods: Qualitative work explored experiences and views on the acceptability of the app through 60 hours of observation in general practices, document analysis (approximately 100 documents), and 62 interviews and four focus groups with patients, carers, members of the public and staff across five general practices, as well as commissioners and policy-makers. Our theoretical approach used the Non-adoption, Abandonment, Scale-up, Spread and Sustainability framework. Quantitative work examined the impact of the NHS App on the usage of primary and secondary care, using routinely collected data. Firstly, using monthly NHS App user data at general practice level in England, descriptive statistics and time series analysis explored monthly NHS App use from January 2019 to May 2021. Secondly, data on the sociodemographic characteristics of the general practitioner-registered population and their healthcare needs at the general practitioner level were used as covariates to explore inequalities in app usage. Finally, NHS App usage data were also compared with measures of patient experience of care and care access extracted from the General Practitioner Patient Survey database. There was no opportunity to evaluate the app or the app functionality in an experimental design. The technology itself, and the context, was changing during the study, which added challenges and complexity. The quantitative analyses used aggregated data rather than individual-level linked data. To evaluate the app, this study had two workstreams. A qualitative approach explored experiences and views on the acceptability of the NHS App through 62 semistructured and think-aloud interviews, and four focus groups with 88 participants. These participants included patients, carers, members of the public (who used the app to different degrees or not at all) and clinical/non-clinical staff in five general practices (where we also conducted over 60 hours of observations), as well as other industry, policy, commissioning and civil rights stakeholders. Document analysis of approximately 100 documents (blogs, government reports, newspaper articles, digital access documents in GP practices) also contributed to participant recruitment and data interpretation. Data were collected between June 2021 and April 2023. Our theoretical approach was based on the Non-adoption, Abandonment, Scale-up, Spread and Sustainability (NASSS) framework. A quantitative element examined the impact of the NHS App on the usage of primary and secondary care, using routinely collected statistics. Firstly, using monthly NHS App user data at general practice level in England, descriptive statistics and time series analysis explored monthly NHS App use from January 2019 to May 2021. Secondly, data on the sociodemographic characteristics of the GP-registered population and their healthcare needs at the GP level were used as covariates to explore inequalities in app usage. Finally, NHS App usage data were also compared with measures of patient experience of care and care access extracted from the GP Patient Survey (GPPS) database. This was a multimethod study using separate qualitative and quantitative approaches to study the app, with the results presented separately, rather than a mixed-methods study where such approaches would be integrated in the analysis.
Details
Project Status: Completed
Year Published: 2026
URL for additional information: English
English language abstract: An English language summary is available
Publication Type: Full HTA
Country: England, United Kingdom
MeSH Terms
  • Mobile Applications
  • State Medicine
  • General Practice
  • COVID-19
  • SARS-CoV-2
  • Program Evaluation
  • Primary Health Care
Contact
Organisation Name: NIHR Health Services and Delivery Research programme
Contact Address: NIHR Journals Library, National Institute for Health and Care Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK
Contact Name: journals.library@nihr.ac.uk
Contact Email: journals.library@nihr.ac.uk
This is a bibliographic record of a published health technology assessment from a member of INAHTA or other HTA producer. No evaluation of the quality of this assessment has been made for the HTA database.