Diagnostic Accuracy of Artificial Intelligence-Based Systems for Detecting Diabetic Retinopathy: A Systematic Review
Abstract
Diabetic retinopathy (DR) represents a leading cause of blindness worldwide, early detection is critical to prevent vision loss. However, traditional screening methods, which rely on human experts, prove to be costly and time-consuming. The systematic review aims to assess the validity of artificial intelligence (AI) as a screening tool for detecting DR among diabetic patients. A systematic literature search was performed of the following databases: PubMed, Scopus, CINAHL, and Web of Science. The last date of our search was January 31, 2024. We included all observational studies, including cohort, case–control and cross-sectional studies and evaluated their quality using the Joanna Briggs Institute tool. We included diagnostic test accuracy studies evaluating the use of AI algorithms for DR screening in patients with diabetes. Studies were excluded if they exclusively assessed diagnostic accuracy for DR that did not use AI algorithms as a diagnostic tool and studies with incomplete or inaccessible data. Thirteen studies with sample sizes ranging from 69 to 1378 participants, reported good sensitivity of AI for detecting visually threatening DR (VTDR). The lowest sensitivity was 89.2%, and the highest was 100%. In terms of specificity, Any DR exhibited higher specificity compared to RDR and VTDR, ranging from 80.2% to 100%. The sensitivity and specificity of the Artificial Intelligence (AI)-based tools available for DR screening was considered acceptable, especially in detecting VTDR and Any DR, was regarded as good. These results implied the potential usefulness of these tools for DR screening in settings with limited resources. However, further high-quality comparative studies were deemed necessary to evaluate their effectiveness in real-world clinical settings.
References
Ahsan, S., Basit, A., Ahmed, K. R., Ali, L., Shaheen, F., Ulhaque, M. S., & Fawwad, A. (2014). Diagnostic accuracy of direct ophthalmoscopy for detection of diabetic retinopathy using fundus photographs as a reference standard. Diabetes and Metabolic Syndrome: Clinical Research and Reviews, 8(2), 96–101. https://doi.org/10.1016/j.dsx.2014.04.015
Aromataris E, M. Z. (Editors). (2020). CHECKLIST FOR COHORT STUDIES. Critical Appraisal tools for use in JBI Systematic Reviews. JBI Manual for Evidence Synthesis.
Balasopoulou, A., Κokkinos, P., Pagoulatos, D., Plotas, P., Makri, O. E., Georgakopoulos, C. D., Vantarakis, A., Li, Y., Liu, J. J., Qi, P., Rapoport, Y., Wayman, L. L., Chomsky, A. S., Joshi, R. S., Press, D., Rung, L., Ademola-popoola, D., Africa, S., Article, O., … Loukovaara, S. (2017). Symposium Recent advances and challenges in the management of retinoblastoma Globe ‑ saving Treatments. BMC Ophthalmology, 17(1), 1. https://doi.org/10.4103/ijo.IJO
Bourne, R. R. A., Stevens, G. A., White, R. A., Smith, J. L., Flaxman, S. R., Price, H., Jonas, J. B., Keeffe, J., Leasher, J., Naidoo, K., Pesudovs, K., Resnikoff, S., & Taylor, H. R. (2013). Causes of vision loss worldwide, 1990-2010: A systematic analysis. The Lancet Global Health, 1(6), 339–349. https://doi.org/10.1016/S2214-109X(13)70113-X
Burgess, P. I., MacCormick, I. J. C., Harding, S. P., Bastawrous, A., Beare, N. A. V., & Garner, P. (2013). Epidemiology of diabetic retinopathy and maculopathy in Africa: A systematic review. Diabetic Medicine, 30(4), 399–412. https://doi.org/10.1111/j.1464-5491.2012.03756.x
Care, D., & Suppl, S. S. (2021). Microvascular complications and foot care: Standards of medical care in diabetes—2021. Diabetes Care, 44(January), S151–S167. https://doi.org/10.2337/dc21-S011
Cheung, C. Y., Tang, F., Wei Ting, D. S., Wei Tan, G. S., & Wong, T. Y. (2019). Artificial intelligence in diabetic eye disease screening. Asia-Pacific Journal of Ophthalmology, 8(2), 158–164. https://doi.org/10.22608/APO.201976
Esteva, A., Robicquet, A., Ramsundar, B., Kuleshov, V., DePristo, M., Chou, K., Cui, C., Corrado, G., Thrun, S., & Dean, J. (2019). A guide to deep learning in healthcare. Nature Medicine, 25(1), 24–29. https://doi.org/10.1038/s41591-018-0316-z
Fenner, B. J., Wong, R. L. M. M., Lam, W.-C. W. C., Tan, G. S. W. W., & Cheung, G. C. M. M. (2018). Advances in Retinal Imaging and Applications in Diabetic Retinopathy Screening: A Review. Ophthalmology and Therapy, 7(2), 333–346. https://doi.org/10.1007/s40123-018-0153-7
Gulshan, V., Rajan, R. P., Widner, K., Wu, D., Wubbels, P., Rhodes, T., Whitehouse, K., Coram, M., Corrado, G., Ramasamy, K., Raman, R., Peng, L., & Webster, D. R. (2019). Performance of a Deep-Learning Algorithm vs Manual Grading for Detecting Diabetic Retinopathy in India. JAMA OPHTHALMOLOGY, 137(9), 987–993. https://doi.org/10.1001/jamaophthalmol.2019.2004 WE - Science Citation Index Expanded (SCI-EXPANDED)
Huemer, J., Wagner, S. K., & Sim, D. A. (2020). The evolution of diabetic retinopathy screening programmes: A chronology of retinal photography from 35 mm slides to artificial intelligence. Clinical Ophthalmology, 14, 2021–2035. https://doi.org/10.2147/OPTH.S261629
Jamaaluddin, & Indah, S. (2021). Buku Ajar Kecerdasan Buatan. Umsida Press, 121.
Jayanegara, A. P., & Lestari, Y. D. (2023). The Validity Of Artificial Intelligence ( Ai ) For Diabetic Retinopathy Screening In Asia : 006(2), 118–129.
JBI. (2020). Checklist for Analytical Cross Sectional Studies - Critical Appraisal tools for use in JBI Systematic Reviews. The Joanna Briggs Institute Critical Appraisal Tools for Use in JBI Systematic Reviews, 1–6.
Kemp, O., Bascaran, C., Cartwright, E., McQuillan, L., Matthew, N., Shillingford-Ricketts, H., Zondervan, M., Foster, A., & Burton, M. (2023). Real-world evaluation of smartphone-based artificial intelligence to screen for diabetic retinopathy in Dominica: a clinical validation study. BMJ OPEN OPHTHALMOLOGY, 8(1). https://doi.org/10.1136/bmjophth-2023-001491 WE - Emerging Sources Citation Index (ESCI)
Kim, T. N., Aaberg, M. T., Li, P. R., Davila, J. R., Bhaskaranand, M., Bhat, S., Ramachandra, C., Solanki, K., Myers, F., Reber, C., Jalalizadeh, R., Margolis, T. P., Fletcher, D., & Paulus, Y. M. (2021). Comparison of automated and expert human grading of diabetic retinopathy using smartphone-based retinal photography. EYE, 35(1), 334–342. https://doi.org/10.1038/s41433-020-0849-5
Krishnan, R., Jain, A., Rogye, A., & Natarajan, S. (2021). Use of offline artificial intelligence in a smartphone-based fundus camera for community screening of diabetic retinopathy. INDIAN JOURNAL OF OPHTHALMOLOGY, 69(11), 3150-+. https://doi.org/10.4103/ijo.IJO_3808_20 WE - Science Citation Index Expanded (SCI-EXPANDED)
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. PLoS Medicine, 6(7). https://doi.org/10.1371/journal.pmed.1000100
Lupidi, M., Danieli, L., Fruttini, D., Nicolai, M., Lassandro, N., Chhablani, J., & Mariotti, C. (2023a). Artificial intelligence in diabetic retinopathy screening: clinical assessment using handheld fundus camera in a real-life setting. ACTA DIABETOLOGICA, 60(8), 1083–1088. https://doi.org/10.1007/s00592-023-02104-0
Lupidi, M., Danieli, L., Fruttini, D., Nicolai, M., Lassandro, N., Chhablani, J., & Mariotti, C. (2023b). Artificial intelligence in diabetic retinopathy screening: clinical assessment using handheld fundus camera in a real-life setting. Acta Diabetologica, 60(8), 1083–1088. https://doi.org/10.1007/s00592-023-02104-0
Malerbi, F. K., Andrade, R. E., Morales, P. H., Stuchi, J. A., Lencione, D., de Paulo, J. V, Carvalho, M. P., Nunes, F. S., Rocha, R. M., Ferraz, D. A., & Belfort, R. (2022). Diabetic Retinopathy Screening Using Artificial Intelligence and Handheld Smartphone-Based Retinal Camera. JOURNAL OF DIABETES SCIENCE AND TECHNOLOGY, 16(3), 716–723. https://doi.org/10.1177/1932296820985567 WE - Emerging Sources Citation Index (ESCI)
Midena, E., Zennaro, L., Lapo, C., Torresin, T., Midena, G., Pilotto, E., & Frizziero, L. (2022). Handheld Fundus Camera for Diabetic Retinopathy Screening: A Comparison Study with Table-Top Fundus Camera in Real-Life Setting. JOURNAL OF CLINICAL MEDICINE, 11(9). https://doi.org/10.3390/jcm11092352 WE - Science Citation Index Expanded (SCI-EXPANDED)
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., Antes, G., Atkins, D., Barbour, V., Barrowman, N., Berlin, J. A., Clark, J., Clarke, M., Cook, D., D’Amico, R., Deeks, J. J., Devereaux, P. J., Dickersin, K., Egger, M., Ernst, E., Gøtzsche, P. C., … Tugwell, P. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine, 6(7). https://doi.org/10.1371/journal.pmed.1000097
Murchison, A. P., Hark, L., Pizzi, L. T., Dai, Y., Mayro, E. L., Storey, P. P., Leiby, B. E., & Haller, J. A. (2017). Non-adherence to eye care in people with diabetes. BMJ Open Diabetes Research and Care, 5(1), 1–10. https://doi.org/10.1136/bmjdrc-2016-000333
Nagendran, M., Chen, Y., Lovejoy, C. A., Gordon, A. C., Komorowski, M., Harvey, H., Topol, E. J., Ioannidis, J. P. A., Collins, G. S., & Maruthappu, M. (2020). Artificial intelligence versus clinicians: Systematic review of design, reporting standards, and claims of deep learning studies in medical imaging. The BMJ, 368, 1–12. https://doi.org/10.1136/bmj.m689
Natarajan, S., Jain, A., Krishnan, R., Rogye, A., & Sivaprasad, S. (2019a). Diagnostic Accuracy of Community-Based Diabetic Retinopathy Screening with an Offline Artificial Intelligence System on a Smartphone. JAMA Ophthalmology, 137(10), 1182–1188. https://doi.org/10.1001/jamaophthalmol.2019.2923
Natarajan, S., Jain, A., Krishnan, R., Rogye, A., & Sivaprasad, S. (2019b). Diagnostic Accuracy of Community-Based Diabetic Retinopathy Screening with an Offline Artificial Intelligence System on a Smartphone. JAMA Ophthalmology, 137(10), 1182–1188. https://doi.org/10.1001/jamaophthalmol.2019.2923
Natarajan, S., Jain, A., Krishnan, R., Rogye, A., & Sivaprasad, S. (2019c). Diagnostic Accuracy of Community-Based Diabetic Retinopathy Screening With an Offline Artificial Intelligence System on a Smartphone. JAMA OPHTHALMOLOGY, 137(10), 1182–1188. https://doi.org/10.1001/jamaophthalmol.2019.2923 WE - Science Citation Index Expanded (SCI-EXPANDED)
National Institute for Clinical Excellence. (2002). Management of Type 2 Diabetes: Retinopathy Screening and Early Management. Inherited Clinical Guideline.
Özbay, E. (2023). An active deep learning method for diabetic retinopathy detection in segmented fundus images using artificial bee colony algorithm. Artif Intell Rev, 56, 3291–3318. https://doi.org/https://doi.org/10.1007/s10462-022-10231-3
Prayogo, M. E., Zaharo, A. F., Damayanti, N. N. R., Widyaputri, F., Thobari, J. A., Susanti, V. Y., & Sasongko, M. B. (2023). Accuracy of Low-Cost, Smartphone-Based Retinal Photography for Diabetic Retinopathy Screening: A Systematic Review. Clinical Ophthalmology, 17(July), 2459–2470. https://doi.org/10.2147/OPTH.S416422
Rajalakshmi, R., Subashini, R., Anjana, R. M., & Mohan, V. (2018a). Automated diabetic retinopathy detection in smartphone-based fundus photography using artificial intelligence. EYE, 32(6), 1138–1144. https://doi.org/10.1038/s41433-018-0064-9 WE - Science Citation Index Expanded (SCI-EXPANDED)
Rajalakshmi, R., Subashini, R., Anjana, R. M., & Mohan, V. (2018b). Automated diabetic retinopathy detection in smartphone-based fundus photography using artificial intelligence. Eye (Basingstoke), 32(6), 1138–1144. https://doi.org/10.1038/s41433-018-0064-9
Rao, D. P., Sindal, M., Sengupta, S., Baskaran, P., Venkatesh, R., Sivaraman, A., & Savoy, F. M. (2022). Towards a Device Agnostic AI for Diabetic Retinopathy Screening: An External Validation Study. CLINICAL OPHTHALMOLOGY, 16, 2659–2667. https://doi.org/10.2147/OPTH.S369675 WE - Emerging Sources Citation Index (ESCI)
Royle, P., Mistry, H., Auguste, P., Shyangdan, D., Freeman, K., Lois, N., & Waugh, N. (2015). Pan-retinal photocoagulation and other forms of laser treatment and drug therapies for non-proliferative diabetic retinopathy: Systematic review and economic evaluation. Health Technology Assessment, 19(51), 1–248. https://doi.org/10.3310/hta19510
Ruan, S., Liu, Y., Hu, W. T., Jia, H. X., Wang, S. S., Song, M. L., Shen, M. X., Luo, D. W., Ye, T., & Wang, F. H. (2022). A new handheld fundus camera combined with visual artificial intelligence facilitates diabetic retinopathy screening. INTERNATIONAL JOURNAL OF OPHTHALMOLOGY, 15(4), 620–627. https://doi.org/10.18240/ijo.2022.04.16 WE - Science Citation Index Expanded (SCI-EXPANDED)
Sasongko, M. B., Widyaputri, F., Agni, A. N., Wardhana, F. S., Kotha, S., Gupta, P., Widayanti, T. W., Haryanto, S., Widyaningrum, R., Wong, T. Y., Kawasaki, R., & Wang, J. J. (2017). Prevalence of Diabetic Retinopathy and Blindness in Indonesian Adults With Type 2 Diabetes. American Journal of Ophthalmology, 181, 79–87. https://doi.org/10.1016/j.ajo.2017.06.019
Sheikh, A., Bhatti, A., Adeyemi, O., Raja, M., & Sheikh, I. (2021). The Utility of Smartphone-Based Artificial Intelligence Approaches for Diabetic Retinopathy: A Literature Review and Meta-Analysis. Journal of Current Ophthalmology, 33(3), 219–226. https://doi.org/10.4103/2452-2325.329064
Sosale, B., Aravind, S. R., Murthy, H., Narayana, S., Sharma, U., Gowda, S. G. V, & Naveenam, M. (2020). Simple, Mobile-based Artificial Intelligence Algorithm in the detection of Diabetic Retinopathy (SMART) study. BMJ OPEN DIABETES RESEARCH & CARE, 8(1). https://doi.org/10.1136/bmjdrc-2019-000892 WE - Science Citation Index Expanded (SCI-EXPANDED)
Sosale, B., Sosale, A. R., Murthy, H., Sengupta, S., & Naveenam, M. (2020). Medios- An offline, smartphone-based artificial intelligence algorithm for the diagnosis of diabetic retinopathy. INDIAN JOURNAL OF OPHTHALMOLOGY, 68(2), 391–395. https://doi.org/10.4103/ijo.IJO_1203_19 WE - Science Citation Index Expanded (SCI-EXPANDED)
Srisubat, A., Kittrongsiri, K., Sangroongruangsri, S., Khemvaranan, C., Shreibati, J. B., Ching, J., Hernandez, J., Tiwari, R., Hersch, F., Liu, Y., Hanutsaha, P., Ruamviboonsuk, V., Turongkaravee, S., Raman, R., & Ruamviboonsuk, P. (2023). Cost-Utility Analysis of Deep Learning and Trained Human Graders for Diabetic Retinopathy Screening in a Nationwide Program. OPHTHALMOLOGY AND THERAPY, 12(2), 1339–1357. https://doi.org/10.1007/s40123-023-00688-y
Tan, T. E., & Wong, T. Y. (2023). Diabetic retinopathy: Looking forward to 2030. Frontiers in Endocrinology, 13(January), 1–8. https://doi.org/10.3389/fendo.2022.1077669
Tao, X., Li, J., Zhu, X., Zhao, B., Sun, J., Ji, L., Hu, D., Pan, C., Huang, Y., Jiang, S., Feng, Q., & Jiang, C. (2016). Association between socioeconomic status and metabolic control and diabetes complications: A cross-sectional nationwide study in Chinese adults with type 2 diabetes mellitus. Cardiovascular Diabetology, 15(1), 1–10. https://doi.org/10.1186/s12933-016-0376-7
Taye, M. M. (2023). Understanding of Machine Learning with Deep Learning: Architectures, Workflow, Applications and Future Directions. Computers, 12(5). https://doi.org/10.3390/computers12050091
Thomas, R. L., Halim, S., Gurudas, S., Sivaprasad, S., & Owens, D. R. (2019). IDF Diabetes Atlas: A review of studies utilising retinal photography on the global prevalence of diabetes related retinopathy between 2015 and 2018. Diabetes Research and Clinical Practice, 157, 107840. https://doi.org/10.1016/j.diabres.2019.107840
Ting, D. S. W., Cheung, C. Y. L., Lim, G., Tan, G. S. W., Quang, N. D., Gan, A., Hamzah, H., Garcia-Franco, R., Yeo, I. Y. S., Lee, S. Y., Wong, E. Y. M., Sabanayagam, C., Baskaran, M., Ibrahim, F., Tan, N. C., Finkelstein, E. A., Lamoureux, E. L., Wong, I. Y., Bressler, N. M., … Wong, T. Y. (2017). Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes. JAMA - Journal of the American Medical Association, 318(22), 2211–2223. https://doi.org/10.1001/jama.2017.18152
Tomić, M., Vrabec, R., Hendelja, Đ., Kolarić, V., Bulum, T., & Rahelić, D. (2023). Diagnostic Accuracy of Hand-Held Fundus Camera and Artificial Intelligence in Diabetic Retinopathy Screening. Biomedicines, 12(1). https://doi.org/10.3390/biomedicines12010034
Tomić, M., Vrabec, R., Hendelja, Đ., Kolarić, V., Bulum, T., & Rahelić, D. (2024). Diagnostic Accuracy of Hand-Held Fundus Camera and Artificial Intelligence in Diabetic Retinopathy Screening. Biomedicines, 12(1). https://doi.org/10.3390/biomedicines12010034
US Department of Health and Human Services. (2020). National Diabetes Statistics Report, 2020. National Diabetes Statistics Report, 2.
Vujosevic, S., Aldington, S. J., Silva, P., Hernández, C., Scanlon, P., Peto, T., & Simó, R. (2020). Screening for diabetic retinopathy: new perspectives and challenges. The Lancet. Diabetes & Endocrinology, 8(4), 337–347. https://doi.org/10.1016/S2213-8587(19)30411-5
Wong, T. Y., Sun, J., Kawasaki, R., Ruamviboonsuk, P., Gupta, N., Lansingh, V. C., Maia, M., Mathenge, W., Moreker, S., Muqit, M. M. K., Resnikoff, S., Verdaguer, J., Zhao, P., Ferris, F., Aiello, L. P., & Taylor, H. R. (2018). Guidelines on Diabetic Eye Care: The International Council of Ophthalmology Recommendations for Screening, Follow-up, Referral, and Treatment Based on Resource Settings. Ophthalmology, 125(10), 1608–1622. https://doi.org/10.1016/j.ophtha.2018.04.007
Zegeye, A. F., Temachu, Y. Z., & Mekonnen, C. K. (2023). Prevalence and factors associated with Diabetes retinopathy among type 2 diabetic patients at Northwest Amhara Comprehensive Specialized Hospitals, Northwest Ethiopia 2021. BMC Ophthalmology, 23(1), 1–8. https://doi.org/10.1186/s12886-022-02746-8.
Copyright (c) 2025 Indonesian Journal of Global Health Research
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
