Automated Diabetic Retinopathy Image Assessment Software Diagnostic Accuracy and Cost-Effectiveness Compared with Human Graders Adnan Tufail, FRCOphth, 1 Caroline Rudisill, PhD, 2 Catherine Egan, FRANZCO, 1 Venediktos V. Kapetanakis, PhD, 3 Sebastian Salas-Vega, MSc, 2 Christopher G. Owen, PhD, 3 Aaron Lee, MD, 1,4 Vern Louw, 1 John Anderson, FRCP, 5 Gerald Liew, FRANZCO, 1 Louis Bolter, 5 Sowmya Srinivas, MBBS, 6 Muneeswar Nittala, MPhil, 6 SriniVas Sadda, MD, 6 Paul Taylor, PhD, 7 Alicja R. Rudnicka, PhD 3 Objective: With the increasing prevalence of diabetes, annual screening for diabetic retinopathy (DR) by expert human grading of retinal images is challenging. Automated DR image assessment systems (ARIAS) may provide clinically effective and cost-effective detection of retinopathy. We aimed to determine whether ARIAS can be safely introduced into DR screening pathways to replace human graders. Design: Observational measurement comparison study of human graders following a national screening program for DR versus ARIAS. Participants: Retinal images from 20 258 consecutive patients attending routine annual diabetic eye screening between June 1, 2012, and November 4, 2013. Methods: Retinal images were manually graded following a standard national protocol for DR screening and were processed by 3 ARIAS: iGradingM, Retmarker, and EyeArt. Discrepancies between manual grades and ARIAS results were sent to a reading center for arbitration. Main Outcome Measures: Screening performance (sensitivity, false-positive rate) and diagnostic accuracy (95% confidence intervals of screening-performance measures) were determined. Economic analysis estimated the cost per appropriate screening outcome. Results: Sensitivity point estimates (95% confidence intervals) of the ARIAS were as follows: EyeArt 94.7% (94.2%e95.2%) for any retinopathy, 93.8% (92.9%e94.6%) for referable retinopathy (human graded as either ungradable, maculopathy, preproliferative, or proliferative), 99.6% (97.0%e99.9%) for proliferative retinopathy; Retmarker 73.0% (72.0 %e74.0%) for any retinopathy, 85.0% (83.6%e86.2%) for referable retinopathy, 97.9% (94.9%e99.1%) for proliferative retinopathy. iGradingM classified all images as either having disease or being ungradable. EyeArt and Retmarker saved costs compared with manual grading both as a replacement for initial human grading and as a filter prior to primary human grading, although the latter approach was less cost-effective. Conclusions: Retmarker and EyeArt systems achieved acceptable sensitivity for referable retinopathy when compared with that of human graders and had sufficient specificity to make them cost-effective alternatives to manual grading alone. ARIAS have the potential to reduce costs in developed-world health care economies and to aid delivery of DR screening in developing or remote health care settings. Ophthalmology 2017;124:343- 351 ª 2016 by the American Academy of Ophthalmology. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Patients with diabetes are at risk of developing retinal microvascular complications that can cause vision loss, and indeed, diabetes is the leading cause of incident blindness among the working-age population. Early detection through regular surveillance by clinical examination or grading of retinal photographs is essential if sight-threatening retinop- athy is to be identified in time to prevent vision loss. 1e4 Annual screening of the retina is recommended but pre- sents a huge challenge, given that the global prevalence of diabetes was estimated to be 9% among adults in 2014. 5 The delivery of diabetic screening will become more problematic as the number of people with diabetic retinopathy (DR) is expected to increase threefold in the United States by 2050 6,7 and to double in the developing world by 2030, particularly in Asia, the Middle East, and Latin America. 8 National screening programs for DR, including that of the UK National Health Service Diabetic Eye Screening Programme (NHS DESP), 9 are effective; however, they are also labor and capital intensive, requiring trained human graders. Similar teleretinal imaging programs have been 343 ª 2016 by the American Academy of Ophthalmology This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Published by Elsevier Inc. http://dx.doi.org/10.1016/j.ophtha.2016.11.014 ISSN 0161-6420/16