Authors' objectives: Lung cancer is one of the most common types of cancer in the United Kingdom. It is often diagnosed late. The 5-year survival rate for lung cancer is below 10%. Early diagnosis may improve survival. Software that has an artificial intelligence-developed algorithm might be useful in assisting with the identification of suspected lung cancer. This review sought to identify evidence on adjunct artificial intelligence software for analysing chest X-rays for suspected lung cancer, and to develop a conceptual cost-effectiveness model to inform discussion of what would be required to develop a fully executable cost-effectiveness model for future economic evaluation. Lung cancer occurs when abnormal cells multiply in an uncontrolled way to form a tumour in the lung. It is one of the most common types of cancer in the UK, and each year over 43,000 new cases are diagnosed. In the early stages of the disease, people usually do not have symptoms, which means that lung cancer is often diagnosed late. The 5-year survival rate for lung cancer is low, at below 10%. Early diagnosis may improve survival. The National Institute for Health and Care Excellence (NICE) has identified software that has an artificial intelligence (AI)-developed algorithm (referred to hereafter as AI software) as potentially useful in assisting with the identification of suspected lung cancer. AI combines computer science and data sets to enable problem-solving. Machine learning and deep learning are subfields of AI. They comprise AI algorithms that seek to create expert systems to make predictions or classifications based on data input. This assessment covers the use of AI software as an adjunct to an appropriate radiology specialist to assist in the identification of suspected lung cancer on chest X-rays (CXRs). AI technologies subject to this assessment are standalone software platforms developed with deep-learning algorithms to interpret CXRs. The algorithms are fixed but updated periodically. The AI software automatically interprets radiology images from the CXR to identify abnormalities or suspected abnormalities. The abnormalities detected and the methods of flagging the location and type of abnormalities differ between different AI technologies. For example, a CXR may be flagged as suspected lung cancer when a lung nodule, lung mass or hilar enlargement, or a combination of these, is identified. A technology may classify CXRs into those with and without a nodule, or it may identify several different abnormalities or lung diseases. The overall aim of this early value assessment (EVA) is to identify evidence on adjunct AI software for analysing CXRs for suspected lung cancer and identify evidence gaps to help direct data collection and further research. A conceptual modelling process was undertaken to inform discussion of what would be required to develop a fully executable cost-effectiveness model for future economic evaluation. The assessment is not intended to replace the need for a full assessment (Diagnostic Assessment Report) or to provide sufficient detail or synthesis to enable a recommendation to be made about whether AI software can be implemented in clinical practice at the present time. There are two populations of interest in this EVA: (1) people referred from primary care for a CXR because they have symptoms suggestive of lung cancer (symptomatic population) and (2) people referred from primary care for a CXR for reasons unrelated to lung cancer (incidental population). Based on the scope produced by NICE, we defined the following questions to inform future assessment on the benefits, harms and costs of adjunct AI for analysing on CXRs for suspected lung cancer compared with human reader alone in these populations: What is the test accuracy and test failure rate of adjunct AI software to detect lung cancer on CXRs? What are the practical implications of adjunct AI to detect lung cancer on CXRs? What is the clinical effectiveness of adjunct AI software applied to CXRs? What are the cost and resource use considerations relating to use of adjunct AI to detect lung cancer? What would a health economic model to estimate the cost-effectiveness of adjunct AI to detect lung cancer look like?

Authors' results and conclusions: None of the studies identified in the searches or submitted by the companies met the inclusion criteria of the review. Contextual information from six studies that did not meet the inclusion criteria provided some evidence that sensitivity for lung cancer detection (but not nodule detection) might be higher when chest X-rays are interpreted by radiology specialists in combination with artificial intelligence software than when they are interpreted by radiology specialists alone. No significant differences were observed for specificity, positive predictive value or number of cancers detected. None of the six studies provided evidence on the clinical effectiveness of adjunct artificial intelligence software. The conceptual model highlighted a paucity of input data along the course of the diagnostic pathway and identified key assumptions required for evidence linkage. There is currently no evidence applicable to this review on the use of adjunct artificial intelligence software for the detection of suspected lung cancer on chest X-ray in either people referred from primary care with symptoms of lung cancer or people referred from primary care for other reasons. Test accuracy, practical implications and clinical effectiveness No studies met the inclusion criteria of the review. Two ongoing studies with unclear eligibility were identified. In the absence of available evidence, we summarised data from six studies that had unclear populations but included a comparison of CXRs read by readers with and without the use of commercial AI software. Statistical comparisons were not undertaken in most of the studies, but there was some evidence that sensitivity might be higher among specialist radiologist with AI than among specialist radiologist without AI. This finding was not consistent between studies, however. No significant differences were observed for specificity, positive predictive value or number of cancers detected. None of the studies provided evidence on the clinical effectiveness of adjunct AI software. The summarised studies were small retrospective studies with important methodological limitations, and their generalisability to the UK population is unclear. There is currently no evidence applicable to this review on the use of adjunct AI software for the detection of suspected lung cancer on CXRs in either people referred from primary care with symptoms of lung cancer or people referred from primary care for other reasons.

Authors' methods: The data sources were MEDLINE All, EMBASE, Cochrane Database of Systematic Reviews, Cochrane CENTRAL, Epistemonikos, ACM Digital Library, World Health Organization International Clinical Trials Registry Platform, clinical experts, Tufts Cost-Effectiveness Analysis Registry, company submissions and clinical experts. Searches were conducted from 25 November 2022 to 18 January 2023. Rapid evidence synthesis methods were employed. Data from companies were scrutinised. The eligibility criteria were (1) primary care populations referred for chest X-ray due to symptoms suggestive of lung cancer or reasons unrelated to lung cancer; (2) study designs that compared radiology specialist assessing chest X-ray with adjunct artificial intelligence software versus radiology specialists alone and (3) outcomes relating to test accuracy, practical implications of using artificial intelligence software and patient-related outcomes. A conceptual decision-analytic model was developed to inform a potential full cost-effectiveness evaluation of adjunct artificial intelligence software for analysing chest X-ray images to identify suspected lung cancer. This review employed rapid evidence synthesis methods. This included only one reviewer conducting all elements of the review, and targeted searches that were conducted in English only. No eligible studies were identified. Data sources MEDLINE All (via Ovid), EMBASE (via Ovid), Cochrane Database of Systematic Reviews (via Wiley), Cochrane CENTRAL (via Wiley), Epistemonikos, ACM Digital Library, World Health Organization International Clinical Trials Registry Platform, clinical experts, and company submissions.

Project Status: Completed

URL for project: https://www.journalslibrary.nihr.ac.uk/programmes/hta/NIHR135755

Year Published: 2024

URL for published report: https://www.journalslibrary.nihr.ac.uk/hta/LKRT4721

URL for additional information: English

English language abstract: An English language summary is available

Publication Type: Full HTA

Country: United Kingdom

DOI: 10.3310/LKRT4721

Organisation Name: NIHR Health and Social Care Delivery Program

Contact Name: Rhiannon Miller

Contact Email: rhiannon.m@prepress-projects.co.uk