Table 2 Characteristics of included studies in systematic review.
First author, reference | Factor addressed of training /testing dataset | Data points | Training dataset | Number of Images (training dataset) | Testing dataset | Number of images (testing dataset) | Outcome measures | Results | Implications | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Gulshan9 | Dataset size (% of total training dataset of 103,698) (Training) | 0.2% | EyePACS | 207 | EyePACS | 24,360 | SP (at pre-set 97% SN) | SP 38% | 60,000 Images may be the minimum training dataset size needed for maximum performance | |||||
2% | 2073 | 61% | ||||||||||||
10% | 10,369 | 77% | ||||||||||||
20% | 20,739 | 86% | ||||||||||||
30% | 31,109 | 91% | ||||||||||||
40% | 41,479 | 98% | ||||||||||||
50% | 51,849 | 100% | ||||||||||||
60% | 62,218 | 96% | ||||||||||||
70% | 72,588 | 97% | ||||||||||||
80% | 82,958 | 100% | ||||||||||||
90% | 93,328 | 99% | ||||||||||||
100% | 103,698 | 100% | ||||||||||||
Mydriasis (testing) | Mydriatic | EyePACS | 128,175 | EyePACS-1 | 4236 | SN SP | SN 89.6% | SP 97.9% | Mydriasis may not be required for optimal performance | |||||
Non-Mydriatic | 4534 | 90.9% | 98.5% | |||||||||||
Both | 8770 | 90.1% | 98.2% | |||||||||||
Ting15 | Retinal cameras (testing) | Canon | SiDRP | 76,370 | BES | 1052 | AUC SN SP | AUC 0.929 | SN 94.4% | SP 88.5% | Different types of retinal cameras do not affect the performance | |||
Topcon | CUHK | 1254 | 0.948 | 99.3% | 83.1% | |||||||||
Carl Zeiss | HKU | 7706 | 0.964 | 100% | 81.3% | |||||||||
Fundus Vue | Guangdong | 15,798 | 0.949 | 98.7% | 81.6% | |||||||||
Study type (testing) | Clinic-based | SiDRP | 76,370 | CUHK | 1254 | AUC SN SP | AUC 0.948 | SN 99.3% | SP 83.1% | The study type does not affect the performance in detection of disease | ||||
Community-based | BES | 1052 | 0.929 | 94.4% | 88.5% | |||||||||
Population-based | Guangdong | 15,798 | 0.949 | 98.7% | 81.6% | |||||||||
Reference Standard (testing) | Retinal Specialists | SiDRP | 76,370 | CUHK | 1254 | AUC SN SP | AUC 0.948 | SN 99.3% | SP 83.1% | If minimally professional graders with ≥7 years’ experience grade, performance may not be affected | ||||
Ophthalmologists | BES | 1052 | 0.929 | 94.4% | 88.5% | |||||||||
Optometrists | HKU | 7706 | 0.964 | 100% | 81.3% | |||||||||
Graders | RVEEH | 2302 | 0.983 | 98.9% | 92.2% | |||||||||
Prevalence rate (testing) | 5.5% (BES) | SiDRP | 76,370 | BES | 1052 | AUC SN SP | AUC 0.929 | SN 94.4% | SP 88.5% | Lower prevalence rate does not greatly affect performance | ||||
8.1% (SCES) | SCES | 1936 | 0.919 | 100% | 76.3% | |||||||||
12.9% (AFEDS) | AFEDS | 1968 | 0.980 | 98.8% | 86.5% | |||||||||
Concurrent diseases (testing) | Mixed pathologies | SiDRP | 76,370 | DR | 37,001 | AUC SN SP | AUC 0.936 | SN 90.5% | SP 91.6% | Concurrent ocular pathologies in the same image does not affect the model’s detection of either disease | ||||
AMD | 773 | 0.942 | 96.4% | 87.2% | ||||||||||
Glaucoma | 56 | 0.931 | 93.2% | 88.7% | ||||||||||
Ethnicity (testing) | Malay | SiDRP | 76,370 | SIMES | 3052 | AUC SN SP | AUC 0.889 | SN 97.1% | SP 82.0% | Despite difference in the retina between ethnicities, this does not influence the performance in detection | ||||
Indian | SINDI | 4512 | 0.917 | 99.3% | 73.3% | |||||||||
Chinese | SCES | 1936 | 0.919 | 100% | 76.3% | |||||||||
African American | AFEDS | 1968 | 0.980 | 98.8% | 86.5% | |||||||||
White | RVEEH | 2302 | 0.983 | 98.9% | 92.2% | |||||||||
Hispanic | Mexico | 1172 | 0.950 | 91.8% | 84.8% | |||||||||
Bawankar31 | Mydriasis (testing) | Non-mydriasis (vs ETDRS mydriatic reference standard) | Eye-PACS1, India | 80,000 | India | 1084 | SN SP | SN 91.2% | SP 96.9% | Despite no mydriasis of testing dataset, the DLS was able to perform highly when compared to mydriatic 7-field ETDRS grading reference standard | ||||
Burlina33 | Dataset size (training) | Real | AREDS | 119,090 | AREDS | 13,302 | AUC AC | AUC 0.971 | AC 91.1% | Creating proxy datasets using GANs may provide a solution to those with limited access to large number of images | ||||
Synthetic | Image generated with GANs | 119,090 | 0.924 | 82.9% | ||||||||||
Sahlsten25 | Image pixel size (training) | 256×256 | Digifundus Ltd (Finland) | 24,806 | Digifundus Ltd (Finland) | 7118 | AUC | AUC0.961 | Training with higher resolution images may improve performance | |||||
299×299 | 24,806 | 0.970 | ||||||||||||
512×512 | 24,806 | 0.979 | ||||||||||||
1024×1024 | 24,806 | 0.984 | ||||||||||||
2095×2095 | 24,806 | 0.987 | ||||||||||||
Bellemo32 | Ethnicity (testing) | African | SiDRP | 76,370 | Zambia | 4504 | AUC SN SP | AUC 0.973 | SN 92.3% | SP 89.0% | Differences in ethnicity between training and testing dataset does not affect performance | |||
Ting34 | Prevalence rate (testing) | 4.1% (VTDR) | SiDRP | 76,370 | Pooled dataset (SiDRP, SIMES, SINDI, SCES, BES, AFEDS, CUHK, DMP) | 93,293 | AUC | AUC 0.950 | Prevalence rate of diseases may be estimated accurately by DLS | |||||
6.5% (RDR) | 0.963 | |||||||||||||
15.9% (ADR) | 0.863 | |||||||||||||
