TY - JOUR
T1 - Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes
AU - Ting, Daniel Shu Wei
AU - Cheung, Carol Yim Lui
AU - Lim, Gilbert
AU - Tan, Gavin Siew Wei
AU - Quang, Nguyen D.
AU - Gan, Alfred
AU - Hamzah, Haslina
AU - Garcia-Franco, Renata
AU - Yeo, Ian Yew San
AU - Lee, Shu Yen
AU - Wong, Edmund Yick Mun
AU - Sabanayagam, Charumathi
AU - Baskaran, Mani
AU - Ibrahim, Farah
AU - Tan, Ngiap Chuan
AU - Finkelstein, Eric A.
AU - Lamoureux, Ecosse L.
AU - Wong, Ian Y.
AU - Bressler, Neil M.
AU - Sivaprasad, Sobha
AU - Varma, Rohit
AU - Jonas, Jost B.
AU - He, Ming Guang
AU - Cheng, Ching Yu
AU - Cheung, Gemmy Chui Ming
AU - Aung, Tin
AU - Hsu, Wynne
AU - Lee, Mong Li
AU - Wong, Tien Yin
N1 - Publisher Copyright:
© 2017 American Medical Association. All rights reserved.
PY - 2017/12/12
Y1 - 2017/12/12
N2 - IMPORTANCE: A deep learning system (DLS) is a machine learning technology with potential for screening diabetic retinopathy and related eye diseases. OBJECTIVE: To evaluate the performance of a DLS in detecting referable diabetic retinopathy, vision-threatening diabetic retinopathy, possible glaucoma, and age-related macular degeneration (AMD) in community and clinic-based multiethnic populations with diabetes. DESIGN, SETTING, AND PARTICIPANTS: Diagnostic performance of a DLS for diabetic retinopathy and related eye diseases was evaluated using 494 661 retinal images. A DLS was trained for detectingdiabetic retinopathy (using 76 370 images), possible glaucoma (125189 images), and AMD (72 610 images), and performance of DLS was evaluated for detecting diabetic retinopathy (using 112 648 images), possible glaucoma (71896 images), and AMD (35 948 images). Training of the DLS was completed in May 2016, and validation of the DLS was completed in May 2017 for detection of referable diabetic retinopathy (moderate nonproliferative diabetic retinopathy or worse) and vision-threateningdiabetic retinopathy (severe nonproliferative diabetic retinopathy or worse) using a primary validation data set in the Singapore National Diabetic Retinopathy Screening Program and 10 multiethnic cohorts with diabetes. EXPOSURES: Use of a deep learning system. MAIN OUTCOMES AND MEASURES: Area under the receiver operating characteristic curve (AUC) and sensitivity and specificity of the DLS with professional graders (retinal specialists, general ophthalmologists, trained graders, or optometrists) as the reference standard. RESULTS: In the primary validation dataset (n = 14 880 patients; 71 896 images; mean [SD] age, 60.2 [2.2] years; 54.6% men), the prevalence of referable diabetic retinopathy was 3.0%; vision-threatening diabetic retinopathy, 0.6%; possible glaucoma, 0.1%; and AMD 2.5%. The AUC of the DLS for referable diabetic retinopathy was 0.936 (95% CI, 0.925-0.943), sensitivity was 90.5% (95% CI, 873%-93.0%), and specificity was 91.6% (95% CI, 91.0%-92.2%). For vision-threatening diabetic retinopathy, AUC was 0.958 (95% CI, 0.956-0.961), sensitivity was 100% (95% CI, 94.1%-100.0%), and specificity was 91.1% (95% CI, 90.7%-91.4%). For possible glaucoma, AUC was 0.942 (95% CI, 0.929-0.954), sensitivity was 96.4% (95% CI, 81.7%-99.9%), and specificity was 87.2% (95% CI, 86.8%-875%). For AMD AUC was 0.931 (95% CI, 0.928-0.935), sensitivity was 93.2% (95% CI, 91.1%-99.8%), and specificity was 88.7% (95% CI, 88.3%-89.0%). For referable diabetic retinopathy in the 10 additional datasets, AUC range was 0.889 to 0.983 (n = 40 752 images). CONCLUSIONS AND RELEVANCE: In this evaluation of retinal images from multiethnic cohorts of patients with diabetes, the DLS had high sensitivity and specificity for identifying diabetic retinopathy and related eye diseases. Further research is necessary to evaluate the applicability of the DLS in health care settings and the utility of the DLS to improve vision outcomes.
AB - IMPORTANCE: A deep learning system (DLS) is a machine learning technology with potential for screening diabetic retinopathy and related eye diseases. OBJECTIVE: To evaluate the performance of a DLS in detecting referable diabetic retinopathy, vision-threatening diabetic retinopathy, possible glaucoma, and age-related macular degeneration (AMD) in community and clinic-based multiethnic populations with diabetes. DESIGN, SETTING, AND PARTICIPANTS: Diagnostic performance of a DLS for diabetic retinopathy and related eye diseases was evaluated using 494 661 retinal images. A DLS was trained for detectingdiabetic retinopathy (using 76 370 images), possible glaucoma (125189 images), and AMD (72 610 images), and performance of DLS was evaluated for detecting diabetic retinopathy (using 112 648 images), possible glaucoma (71896 images), and AMD (35 948 images). Training of the DLS was completed in May 2016, and validation of the DLS was completed in May 2017 for detection of referable diabetic retinopathy (moderate nonproliferative diabetic retinopathy or worse) and vision-threateningdiabetic retinopathy (severe nonproliferative diabetic retinopathy or worse) using a primary validation data set in the Singapore National Diabetic Retinopathy Screening Program and 10 multiethnic cohorts with diabetes. EXPOSURES: Use of a deep learning system. MAIN OUTCOMES AND MEASURES: Area under the receiver operating characteristic curve (AUC) and sensitivity and specificity of the DLS with professional graders (retinal specialists, general ophthalmologists, trained graders, or optometrists) as the reference standard. RESULTS: In the primary validation dataset (n = 14 880 patients; 71 896 images; mean [SD] age, 60.2 [2.2] years; 54.6% men), the prevalence of referable diabetic retinopathy was 3.0%; vision-threatening diabetic retinopathy, 0.6%; possible glaucoma, 0.1%; and AMD 2.5%. The AUC of the DLS for referable diabetic retinopathy was 0.936 (95% CI, 0.925-0.943), sensitivity was 90.5% (95% CI, 873%-93.0%), and specificity was 91.6% (95% CI, 91.0%-92.2%). For vision-threatening diabetic retinopathy, AUC was 0.958 (95% CI, 0.956-0.961), sensitivity was 100% (95% CI, 94.1%-100.0%), and specificity was 91.1% (95% CI, 90.7%-91.4%). For possible glaucoma, AUC was 0.942 (95% CI, 0.929-0.954), sensitivity was 96.4% (95% CI, 81.7%-99.9%), and specificity was 87.2% (95% CI, 86.8%-875%). For AMD AUC was 0.931 (95% CI, 0.928-0.935), sensitivity was 93.2% (95% CI, 91.1%-99.8%), and specificity was 88.7% (95% CI, 88.3%-89.0%). For referable diabetic retinopathy in the 10 additional datasets, AUC range was 0.889 to 0.983 (n = 40 752 images). CONCLUSIONS AND RELEVANCE: In this evaluation of retinal images from multiethnic cohorts of patients with diabetes, the DLS had high sensitivity and specificity for identifying diabetic retinopathy and related eye diseases. Further research is necessary to evaluate the applicability of the DLS in health care settings and the utility of the DLS to improve vision outcomes.
UR - http://www.scopus.com/inward/record.url?scp=85038438910&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85038438910&partnerID=8YFLogxK
U2 - 10.1001/jama.2017.18152
DO - 10.1001/jama.2017.18152
M3 - Article
C2 - 29234807
AN - SCOPUS:85038438910
SN - 0098-7484
VL - 318
SP - 2211
EP - 2223
JO - JAMA - Journal of the American Medical Association
JF - JAMA - Journal of the American Medical Association
IS - 22
ER -