Original Title: État des connaissances: télédépistage de la rétinopathie diabétique à l’aide d’une solution d’intelligence artificielle

Authors' objectives: Diabetic retinopathy (DR) is an alteration of retinal blood vessels that affects nearly 80% of patients with type 2 diabetes in Quebec. It can remain asymptomatic up to an advanced stage and lead to blindness if not treated in time. Opportunistic DR screening services are available across the province, following different care pathways. However, the clinical needs of patients are significant and yet remain unmet. The unequal distribution of medical resources and the high prevalence of diabetes do not allow for equitable access to ocular health care and services. According to the opinion of the experts consulted, most diabetic patients referred to ophthalmology have DR. The challenge is to distinguish between those requiring treatment by a specialist and those in need of a follow-up that can be provided by specially trained nurses. The implementation of a program of systematic DR tele-screening (DRTS) in Quebec would allow for early detection and management of the disease. Screening for DR using artificial intelligence (AI) could be an option to consider in order to compensate for the lack of specialized medical resources. As part of the creation and the implementation of the provincial DRTS program, the Ministère de la Santé et des Services sociaux (MSSS) wishes to explore the possibility of integrating an AI dedicated to DR screening (AIDR). The Institut national d'excellence en santé et en services sociaux (INESSS) has been asked to assist the MSSS in this process.

Authors' results and conclusions: RESULTS: (MARKET REVIEW) Several AIDRs are marketed worldwide. Only two of them are available in Canada (CARA / Neoretina and EyeArt). Most commercially available AIDR can detect the presence of referable DR (triage) but they require human intervention to grade the severe levels of the disease. However, some do identify sight-threatening DR (IDx-DR V2.0; EyeArt, OphthalAI, SELANA+) as well as degenerative macular edema (DME, a serious complication of DR) or other eye conditions such as age-related macular degeneration (AMD) and glaucoma (EyeArt, OphthAI, SELANA+). The technical specifications to be considered when choosing an algorithm are: its maturity (development process, evolution, etc.); its adaptability to different types of data (images obtained by digital camera, optical coherence tomography - OCT) and its interoperability; evaluation and clinical performance parameters (real-world data on sensitivity and specificity, explainability of results; percentage of ungradable images that can vary from 10% to 38.5%); data security and confidentiality. AI is extremely suitable for performing repetitive tasks rapidly. The potential for optimizing clinical-administrative information flows and reserving clinical expertise for high value-added activities is a valuable asset. However, the anticipated benefits must be weighed against the challenges related to the way AIDR operates and integrates into the healthcare system. (CLINICAL EFFICACY) Several studies document the clinical efficacy of AIDRs measured from selected retinal images (experimental setting). Approximately one third of the 22 studies considered were conducted in a real-world setting. The data must therefore be interpreted with caution. In addition, sole one study compares AIDRs to each other. AIDR can identify patients in need of treatment by a specialist (mild DR or higher grade, DME). Sensitivity of the first generations of AIDRs ranges from 73% to 96.8%; specificity is as low as 71.5%. Newer systems have similar or lower clinical efficacy (sensitivity and specificity up to 91%) as compared to human readers. However, a greater variability is observed for real-world data on specificity. Not all AIDRs can detect DME. Some can identify sight-threatening DR (EyeArt, IDx-DR, OphtAI, SELENA+) or other ocular pathologies (SELENA+). According to the experts consulted, the main risk related to AIDR is that the disease is undetected, severe or not. A number of clinical issues must be considered, including the clinical performance of AIDR in a real-world setting, the choice of method that defines the ground truth, the detection of other ocular pathologies requiring medical attention, and the evolution of AIDR in relation to clinical practice. (ORGANIZATIONAL CONSIDERATIONS) According to the experts consulted, it would be important to base the integration of AIDR on a yet well-structured, established and robust organization for tele-screening and care. AI has been integrated for triage purposes (presence/absence of DR) in a few DRTS programmes as research projects. To our knowledge, solely Portugal and Scotland have effectively adopted AIDR. Based on the experience of the DRTS projects implemented in Montreal and the James Bay Territory (RUIS-McGill and RUIS-UdeM), the MSSS plans to set up a provincial DRTS program and to integrate AI. There are 3 likely scenarios: AIDR would replace the 1st level of reading (triage by trained nurses) or the 2nd one (detection of mild DR and DME by trained nurses); it could also be used as a decision-making tool, in parallel with human readers. The 3rd reading level would be provided by retinologists (severe grades of DR). AIDR could help address the shortage of retinologists, increase the number of cases detected and the number of patients that can be treated, and harmonize clinical practices. However, it involves an overhaul of care and service delivery models and may require other innovations (technological or process) that may be slow to implement. There are multiple issues that should be carefully analyzed for successful integration. The available studies from very specific contexts (highly specialized centers) do not shed sufficient light on the nature and extent of the changes needed to realize the value of AIDR in real-life care settings. (EFFICIENCY) Two identified studies evaluate AIDR in comparison to tele-screening without AI (UK and Singapore). AIDR (EyeArt, Retmarker, SELENA+) seems to be an efficient technology since the cost reduction it generates would compensate for the decrease in efficacy (specificity). However, these results cannot be fully transposed to Quebec due to different contexts. At the clinical level, the sensitivity and specificity of the AIDRs offered in Quebec may differ from those reported in the literature; at the organizational level, not having a well-established DRTS program has a significant impact on efficiency; and finally, the cost of the technology offered in Quebec may differ. In an innovation context, and more particularly for AI technologies where the pace of development and replacement is very rapid, the evidence supporting the benefits of AIDR is limited and leads to great uncertainty as to the real benefits for Quebec. Unrecoverable costs are also to be expected, given the significant investments made in the acquisition of these technologies and their short life cycle. (ETHICAL CONSIDERATIONS AND INFORMATION SECURITY) The responsible development and management of AI calls for the definition of necessary safeguards at all stages of the process and at all levels of care. In the AI development process, equity issues arise, depending on the source of the images used. Technical (ungradable images) and clinical (especially specificity) performance could be significantly lower when AI is applied to the Quebec real-world context, with specific populations (older people, several eye pathologies, ethno-diversity). However, an effective DRTS program at the organizational level should allow greater access to care in remote areas, among others, where unmet needs are significant. From a governance perspective, particular attention to the security of data throughout its lifecycle is paramount, given its potential identifying nature and the risk of monetization of personal data. This includes informed consent, protection processes according to current regulations (anonymization), secure data management at the organizational level (staff training) and data storage. Finally, the stake for the patient, in terms of confidence in the accuracy of his health outcome, leads to the need for a legal framework for liability in case of error. CONCLUSION:   AIDR allows for triaging patients with type 2 diabetes according to whether they have clinical signs of DR or not. The most advanced solutions can identify the severe levels of the disease (from mild grade, suspected DME) or sight-threatening grades. However, the global value of AIDR cannot be realized without a DRTS program with a strong organizational structure. Well-defined and operationally sound clinical processes and service corridors are essential. Anticipating the healthcare system capacity to provide care and follow-up to screened individuals is also paramount. Since the ultimate goal of any DRTS program is to reduce vision loss due to DR, it seems essential to set up a continuous performance evaluation process. Finally, when choosing an AIDR, it would be wise to consider the possibilities of technological advances, adaptation to the Quebec context (clinical performance based on Quebec data, interoperability, alignment with the organization of care) and flexibility with regard to the cost of the technology (minimization of risk and irrecoverable implementation costs).

Authors' methods: A market review of commercially available AIDR systems and those under development was conducted. A rapid review of their clinical performance and efficiency was also conducted as well as a reflection on the various issues related to their use.

Project Status: Completed

URL for project: https://www.inesss.qc.ca/publications/repertoire-des-publications/publication/teledepistage-de-la-retinopathie-diabetique-a-laide-dune-solution-dintelligence-artificielle.html

Year Published: 2021

URL for published report: https://www.inesss.qc.ca/publications/repertoire-des-publications/publication/teledepistage-de-la-retinopathie-diabetique-a-laide-dune-solution-dintelligence-artificielle.html

English language abstract: An English language summary is available

Publication Type: Full HTA

Country: Canada

Province: Quebec

Organisation Name: Institut national d'excellence en sante et en services sociaux

Contact Address: L'Institut national d'excellence en sante et en services sociaux (INESSS) , 2021, avenue Union, bureau 10.083, Montreal, Quebec, Canada, H3A 2S9;Tel: 1+514-873-2563, Fax: 1+514-873-1369

Contact Name: demande@inesss.qc.ca

Contact Email: demande@inesss.qc.ca

Copyright: Gouvernement du Québec