Table 1 Demographics of the NIA Dataset and Top-1 and Top-3 Accuracies of the Algorithm
From: Planet-wide performance of a skin disease AI algorithm validated in Korea
Category | Disease | NIA Dataset | Algo. (Binary) | Algo. (Multiclass) | |||||
|---|---|---|---|---|---|---|---|---|---|
No. of Images | No. of Cases | Average Age | Male (%) | TOP-1 | TOP-3 | TOP-1 | TOP-3 | ||
Malignant Tumors | Basal cell carcinoma | 2010 | 1804 | 69.0 | 48.4 | 62.1% (CI 60.0–64.3%) | 81.6% (CI 79.9–83.3%) | 55.9% (CI 53.7–58.1%) | 73.1% (CI 71.1–75.0%) |
Intraepithelial carcinoma (Bowen disease) | 170 | 111 | 70.6 | 45.9 | 39.4% (CI 32.1–46.8%) | 67.6% (CI 60.6–74.7%) | 28.2% (CI 21.5–35.0%) | 53.5% (CI 46.0–61.0%) | |
Malignant melanoma | 825 | 734 | 63.7 | 51.1 | 49.0% (CI 45.6–52.4%) | 73.9% (CI 70.9–76.9%) | 47.3% (CI 43.9–50.7%) | 66.8% (CI 63.6–70.0%) | |
Squamous cell carcinoma | 1804 | 1601 | 74.2 | 48.5 | 50.2% (CI 47.9–52.5%) | 77.3% (CI 75.3–79.2%) | 33.3% (CI 31.1–35.5%) | 60.0% (CI 57.7–62.2%) | |
Total (Malignant Tumors) | 4250 | 4809 | 70.1 | 48.9 | a50.2% b54.6% | a75.1% b78.2% | a41.2% b45.0% | a63.3% b66.4% | |
Benign Tumors | Actinic keratosis | 1823 | 1405 | 73.7 | 37.4 | 92.9% (CI 91.7–94.1%) | 58.1% (CI 55.8–60.4%) | 40.9% (CI 38.7–43.2%) | 69.9% (CI 67.8–72.0%) |
Becker nevus | 872 | 672 | 14.8 | 63.8 | 100.0% (CI 100.0–100.0%) | 99.9% (CI 99.7–100.1%) | 54.1% (CI 50.8–57.4%) | 72.9% (CI 70.0–75.9%) | |
Blue nevus | 113 | 105 | 33.0 | 45.7 | 92.9% (CI 88.2–97.6%) | 71.7% (CI 63.4–80.0%) | 60.2% (CI 51.2–69.2%) | 75.2% (CI 67.3–83.2%) | |
Cafe au lait macule | 2445 | 1648 | 5.7 | 45.0 | 100.0% (CI 100.0–100.0%) | 99.9% (CI 99.7–100.0%) | 51.7% (CI 49.8–53.7%) | 77.5% (CI 75.9–79.2%) | |
Callus | 1807 | 1472 | 54.7 | 40.0 | 99.5% (CI 99.2–99.8%) | 97.7% (CI 97.0–98.4%) | 28.4% (CI 26.3–30.5%) | 70.3% (CI 68.2–72.4%) | |
Congenital nevus | 1933 | 1546 | 8.1 | 47.2 | 98.8% (CI 98.3–99.3%) | 84.6% (CI 83.0–86.2%) | 51.8% (CI 49.6–54.0%) | 76.0% (CI 74.1–78.0%) | |
Dermatofibroma | 1245 | 1136 | 40.9 | 29.0 | 99.7% (CI 99.4–100.0%) | 95.9% (CI 94.8–97.0%) | 68.8% (CI 66.3–71.4%) | 84.9% (CI 82.9–86.9%) | |
Epidermal cyst | 2732 | 2516 | 36.4 | 53.7 | 99.3% (CI 99.0–99.6%) | 97.4% (CI 96.8–98.0%) | 52.1% (CI 50.2–54.0%) | 74.5% (CI 72.9–76.2%) | |
Epidermal nevus | 1256 | 801 | 10.1 | 49.4 | 99.9% (CI 99.8–100.1%) | 98.9% (CI 98.3–99.5%) | 45.9% (CI 43.1–48.6%) | 68.1% (CI 65.5–70.7%) | |
Hemangioma | 3710 | 3315 | 2.9 | 31.3 | 99.9% (CI 99.8–100.0%) | 95.8% (CI 95.1–96.4%) | 90.3% (CI 89.4–91.3%) | 96.0% (CI 95.3–96.6%) | |
Hypertrophic scar | 489 | 361 | 24.8 | 29.4 | 98.6% (CI 97.5–99.6%) | 96.1% (CI 94.4–97.8%) | 10.8% (CI 8.1–13.6%) | 57.9% (CI 53.5–62.2%) | |
Juvenile xanthogranuloma | 1050 | 753 | 2.8 | 60.6 | 100.0% (CI 100.0–100.0%) | 99.2% (CI 98.7–99.8%) | 51.1% (CI 48.1–54.2%) | 68.5% (CI 65.7–71.3%) | |
Keloid | 2262 | 1494 | 29.5 | 42.8 | 99.2% (CI 98.8–99.5%) | 94.0% (CI 93.0–95.0%) | 45.8% (CI 43.8–47.9%) | 68.7% (CI 66.8–70.6%) | |
Lentigo | 1878 | 890 | 59.2 | 20.2 | 99.8% (CI 99.7–100.0%) | 98.0% (CI 97.3–98.6%) | 48.7% (CI 46.5–51.0%) | 80.2% (CI 78.4–82.0%) | |
Melanocytic nevus | 3803 | 2949 | 25.8 | 41.0 | 98.3% (CI 97.9–98.7%) | 81.2% (CI 80.0–82.5%) | 69.8% (CI 68.3–71.2%) | 91.5% (CI 90.6–92.4%) | |
Melanonychia | 2162 | 1448 | 16.5 | 53.8 | 99.9% (CI 99.7–100.0%) | 83.5% (CI 81.9–85.1%) | 58.3% (CI 56.2–60.4%) | 79.7% (CI 78.0–81.4%) | |
Milia | 1155 | 459 | 44.1 | 22.7 | 100.0% (CI 100.0–100.0%) | 99.3% (CI 98.8–99.8%) | 27.6% (CI 25.0–30.2%) | 58.1% (CI 55.2–60.9%) | |
Nevus depigmentosus | 1911 | 1362 | 3.9 | 51.8 | 100.0% (CI 100.0–100.0%) | 100.0% (CI 100.0–100.0%) | 37.3% (CI 35.1–39.5%) | 82.4% (CI 80.7–84.1%) | |
Orgarnoid nevus | 1334 | 1233 | 10.4 | 51.8 | 100.0% (CI 100.0–100.0%) | 97.7% (CI 96.9–98.5%) | 78.4% (CI 76.2–80.6%) | 90.9% (CI 89.4–92.5%) | |
Ota nevus | 949 | 718 | 8.7 | 38.2 | 100.0% (CI 100.0–100.0%) | 99.1% (CI 98.4–99.7%) | 40.5% (CI 37.3–43.6%) | 61.5% (CI 58.4–64.6%) | |
Portwine stain | 2689 | 1997 | 9.2 | 43.1 | 100.0% (CI 100.0–100.0%) | 99.4% (CI 99.2–99.7%) | 62.7% (CI 60.9–64.5%) | 85.1% (CI 83.7–86.4%) | |
Pyogenic granuloma | 657 | 630 | 17.1 | 52.2 | 99.1% (CI 98.4–99.8%) | 89.6% (CI 87.3–92.0%) | 53.3% (CI 49.5–57.1%) | 81.4% (CI 78.5–84.4%) | |
Sebaceus hyperplasia | 1340 | 609 | 53.2 | 75.9 | 99.8% (CI 99.5–100.0%) | 96.2% (CI 95.2–97.2%) | 32.1% (CI 29.6–34.6%) | 60.2% (CI 57.6–62.8%) | |
Seborrheic keratosis | 2551 | 1982 | 63.7 | 53.5 | 97.9% (CI 97.4–98.5%) | 80.2% (CI 78.7–81.8%) | 58.8% (CI 56.9–60.7%) | 82.5% (CI 81.0–84.0%) | |
Skin tag | 1236 | 798 | 49.3 | 54.0 | 99.7% (CI 99.4–100.0%) | 99.4% (CI 98.9–99.8%) | 34.1% (CI 31.5–36.8%) | 63.6% (CI 60.9–66.3%) | |
Syringoma | 1268 | 552 | 44.2 | 14.9 | 100.0% (CI 100.0–100.0%) | 99.5% (CI 99.1–99.9%) | 25.8% (CI 23.4–28.2%) | 53.5% (CI 50.8–56.3%) | |
Total (Benign Tumors) | 32,851 | 44,670 | 26.8 | 44.1 | a99.0% b99.2% | a92.8% b92.4% | a49.2% b53.5% | a74.3% b77.6% | |
Infectious Disorders | cWart | 3551 | 2435 | 20.7 | 60.0 | 99.2% (CI 98.9–99.5%) | 95.2% (CI 94.5–95.9%) | 64.7% (CI 63.1–66.2%) | 81.7% (CI 80.4–83.0%) |
Acne | 6537 | 3235 | 21.4 | 51.1 | 99.9% (CI 99.9–100.0%) | 99.7% (CI 99.5–99.8%) | 62.2% (CI 61.0–63.3%) | 87.9% (CI 87.1–88.6%) | |
Folliculitis | 1780 | 1005 | 38.0 | 73.7 | 99.9% (CI 99.8–100.1%) | 99.2% (CI 98.8–99.6%) | 39.8% (CI 37.6–42.1%) | 71.3% (CI 69.2–73.4%) | |
Herpes simplex | 1450 | 887 | 34.0 | 45.5 | 97.2% (CI 96.3–98.0%) | 89.9% (CI 88.3–91.4%) | 13.7% (CI 12.0–15.5%) | 32.2% (CI 29.8–34.6%) | |
Herpes zoster | 421 | 387 | 53.3 | 46.8 | 97.6% (CI 96.2–99.1%) | 96.7% (CI 95.0–98.4%) | 49.9% (CI 45.1–54.7%) | 65.6% (CI 61.0–70.1%) | |
Impetigo | 1755 | 745 | 6.0 | 58.4 | 99.4% (CI 99.1–99.8%) | 96.1% (CI 95.2–97.0%) | 28.0% (CI 25.9–30.1%) | 42.6% (CI 40.3–44.9%) | |
Ingrown nail | 765 | 636 | 15.1 | 60.2 | 100.0% (CI 100.0–100.0%) | 99.7% (CI 99.4–100.1%) | 52.5% (CI 49.0–56.1%) | 74.6% (CI 71.6–77.7%) | |
Molluscum contagiosum | 2943 | 1653 | 7.1 | 52.8 | 100.0% (CI 99.9–100.0%) | 99.7% (CI 99.5–99.9%) | 55.8% (CI 54.0–57.6%) | 78.6% (CI 77.1–80.0%) | |
onychomycosis | 2865 | 1212 | 43.9 | 53.5 | 100.0% (CI 100.0–100.0%) | 99.9% (CI 99.8–100.0%) | 73.8% (CI 72.1–75.4%) | 94.8% (CI 93.9–95.6%) | |
Pityriasis rosea | 1355 | 675 | 28.9 | 49.6 | 99.8% (CI 99.5–100.0%) | 99.3% (CI 98.9–99.8%) | 31.7% (CI 29.2–34.1%) | 58.2% (CI 55.5–60.8%) | |
Tinea corporis | 483 | 317 | 45.7 | 55.2 | 98.8% (CI 97.8–99.7%) | 97.1% (CI 95.6–98.6%) | 42.4% (CI 38.0–46.9%) | 62.5% (CI 58.2–66.8%) | |
Tinea pedis | 248 | 155 | 48.3 | 64.5 | 98.0% (CI 96.2–99.7%) | 94.4% (CI 91.5–97.2%) | 29.0% (CI 23.4–34.7%) | 48.4% (CI 42.2–54.6%) | |
Tinea versicolor | 2009 | 1043 | 21.9 | 73.2 | 100.0% (CI 100.0–100.0%) | 99.8% (CI 99.6–100.0%) | 7.8% (CI 6.6–8.9%) | 20.8% (CI 19.0–22.6%) | |
Varicella | 1042 | 375 | 11.3 | 65.1 | 100.0% (CI 100.0–100.0%) | 100.0% (CI 100.0–100.0%) | 25.5% (CI 22.9–28.2%) | 43.8% (CI 40.7–46.8%) | |
Verruca plana | 170 | 73 | 33.2 | 42.5 | 100.0% (CI 100.0–100.0%) | 98.8% (CI 97.2–100.4%) | 0.0% (CI 0.0–0.0%) | 8.8% (CI 4.6–13.1%) | |
Total (Infectious Disorders) | 14,833 | 27,374 | 24.2 | 56.8 | a99.3% b99.6% | a97.7% b98.2% | a38.5% b48.4% | a58.1% b69.8% | |
Allergic Disorders | Atopic dermatitis | 20,449 | 10,440 | 14.1 | 64.9 | 99.9% (CI 99.9–100.0%) | 99.6% (CI 99.5–99.7%) | 47.5% (CI 46.8–48.2%) | 67.7% (CI 67.1–68.4%) |
Contact dermatitis | 2206 | 1022 | 46.8 | 36.3 | 100.0% (CI 100.0–100.0%) | 98.9% (CI 98.4–99.3%) | 16.1% (CI 14.6–17.7%) | 47.9% (CI 45.8–50.0%) | |
Insect bite | 1319 | 730 | 9.6 | 52.5 | 100.0% (CI 100.0–100.0%) | 99.4% (CI 99.0–99.8%) | 15.4% (CI 13.4–17.3%) | 38.7% (CI 36.0–41.3%) | |
Lichen simplex chronicus | 151 | 59 | 51.4 | 47.5 | 99.3% (CI 98.0–100.6%) | 97.4% (CI 94.8–99.9%) | 3.3% (CI 0.5–6.2%) | 29.8% (CI 22.5–37.1%) | |
Nummular eczema | 3725 | 1844 | 25.7 | 53.6 | 99.3% (CI 99.0–99.5%) | 96.2% (CI 95.6–96.8%) | 21.7% (CI 20.4–23.0%) | 40.2% (CI 38.7–41.8%) | |
Pompholyx | 983 | 363 | 17.4 | 46.3 | 99.9% (CI 99.7–100.1%) | 99.4% (CI 98.9–99.9%) | 17.2% (CI 14.8–19.6%) | 49.0% (CI 45.9–52.2%) | |
Prurigo nodularis | 2684 | 924 | 18.3 | 62.7 | 99.8% (CI 99.7–100.0%) | 99.1% (CI 98.7–99.4%) | 36.7% (CI 34.8–38.5%) | 57.5% (CI 55.7–59.4%) | |
Seborrheic dermatitis | 1566 | 843 | 45.2 | 53.9 | 99.7% (CI 99.5–100.0%) | 98.4% (CI 97.8–99.0%) | 10.5% (CI 9.0–12.0%) | 29.3% (CI 27.1–31.6%) | |
Urticaria | 3020 | 1340 | 23.3 | 58.2 | 100.0% (CI 100.0–100.0%) | 99.7% (CI 99.5–99.9%) | 47.0% (CI 45.2–48.8%) | 68.4% (CI 66.8–70.1%) | |
Urticaria pigmentosa | 1718 | 1030 | 1.7 | 63.0 | 99.9% (CI 99.7–100.0%) | 99.4% (CI 99.1–99.8%) | 40.2% (CI 37.9–42.5%) | 58.5% (CI 56.2–60.8%) | |
Total (Allergic Disorders) | 18,595 | 37,821 | 18.6 | 60.1 | a99.8% b99.8% | a98.7% b99.1% | a25.6% b38.4% | a48.7% b59.5% | |
Miscellaneous | Alopecia areata | 4237 | 3014 | 22.7 | 53.7 | 100.0% (CI 100.0–100.0%) | 99.8% (CI 99.7–100.0%) | 69.3% (CI 67.9–70.7%) | 91.9% (CI 91.1–92.7%) |
Androgenic alopecia | 2150 | 1928 | 40.9 | 99.8 | 100.0% (CI 100.0–100.0%) | 99.9% (CI 99.8–100.0%) | 87.5% (CI 86.1–88.9%) | 98.4% (CI 97.9–98.9%) | |
Guttate psoriasis | 354 | 184 | 32.3 | 38.6 | 100.0% (CI 100.0–100.0%) | 100.0% (CI 100.0–100.0%) | 44.1% (CI 38.9–49.2%) | 71.5% (CI 66.8–76.2%) | |
Idiopathic guttate hypomelanosis | 1639 | 762 | 62.1 | 26.9 | 100.0% (CI 100.0–100.0%) | 99.2% (CI 98.8–99.6%) | 39.3% (CI 36.9–41.7%) | 65.8% (CI 63.5–68.1%) | |
Lichen striatus | 1732 | 1211 | 7.1 | 36.3 | 99.9% (CI 99.7–100.0%) | 99.9% (CI 99.7–100.0%) | 62.3% (CI 60.0–64.6%) | 80.6% (CI 78.7–82.5%) | |
Nail dystrophy | 3239 | 1549 | 18.9 | 52.2 | 100.0% (CI 100.0–100.0%) | 99.7% (CI 99.5–99.9%) | 58.6% (CI 56.9–60.3%) | 95.5% (CI 94.7–96.2%) | |
Onycholysis | 182 | 84 | 45.8 | 39.3 | 100.0% (CI 100.0–100.0%) | 100.0% (CI 100.0–100.0%) | 72.5% (CI 66.0–79.0%) | 94.5% (CI 91.2–97.8%) | |
Palmoplantar pustulosis | 159 | 109 | 46.9 | 33.9 | 100.0% (CI 100.0–100.0%) | 100.0% (CI 100.0–100.0%) | 77.4% (CI 70.9–83.9%) | 96.9% (CI 94.1–99.6%) | |
Parapsoriasis | 154 | 116 | 45.2 | 45.7 | 99.4% (CI 98.1–100.6%) | 98.7% (CI 96.9–100.5%) | 40.3% (CI 32.5–48.0%) | 67.5% (CI 60.1–74.9%) | |
Pityriasis lichenoides chronica | 2051 | 818 | 12.0 | 64.4 | 100.0% (CI 100.0–100.0%) | 99.8% (CI 99.6–100.0%) | 7.0% (CI 5.9–8.1%) | 36.4% (CI 34.3–38.5%) | |
Pityriasis lichenoides et varioliformis acuta | 482 | 213 | 11.3 | 71.8 | 100.0% (CI 100.0–100.0%) | 100.0% (CI 100.0–100.0%) | 10.2% (CI 7.5–12.9%) | 36.1% (CI 31.8–40.4%) | |
Psoriasis | 11,073 | 5258 | 33.2 | 65.0 | 99.6% (CI 99.4–99.7%) | 97.6% (CI 97.3–97.9%) | 44.6% (CI 43.7–45.6%) | 65.4% (CI 64.5–66.3%) | |
Rosacea | 2450 | 1321 | 50.9 | 22.5 | 99.9% (CI 99.7–100.0%) | 99.3% (CI 99.0%-99.7%) | 44.9% (CI 42.9%-46.9%) | 77.2% (CI 75.5%-78.8%) | |
Scar | 2887 | 2183 | 21.6 | 35.1 | 99.0% (CI 98.6–99.4%) | 96.2% (CI 95.5–96.9%) | 32.1% (CI 30.4–33.8%) | 56.7% (CI 54.9–58.5%) | |
Vitiligo | 4980 | 2725 | 25.7 | 54.1 | 99.9% (CI 99.9–100.0%) | 99.3% (CI 99.1–99.5%) | 62.3% (CI 61.0–63.7%) | 83.6% (CI 82.6–84.6%) | |
Total (Miscellaneous) | 21,475 | 37,769 | 29.0 | 55.1 | a99.8% b99.8% | a99.3% b98.8% | a50.2% b50.8% | a74.5% b74.4% | |
Total | 152,443 | 92,004 | 27.2 | 52.2 | a96.6% b98.2% | a95.1% b96.2% | a43.3% b47.9% | a66.6% b70.6% | |
