Table 8 Showing performance measure, i.e. (accuracy, sensitivity and specificity) of random forest (RF)/support vector machine (SVM)/artificial neural network (ANN) analysis based on external database validation. The training data set is based on MTBC isolates from Sikkim and testing data set is based on MTBC isolates from a different geographical area (the state of Assam), which was not used for model training.
Model | Type of data set | Performance measure | Beijing* | CAS1-Delhi* | T1* |
|---|---|---|---|---|---|
RF | Training | Sensitivity | 95.89 (range 88.46–99.14) | 95.35 (range 84.19–99.43) | 93.75 (range 69.77–99.84) |
Specificity | 94.92 (range 85.85–98.94) | 100 (range 98.12–100) | 100 (range 96.87–100) | ||
Accuracy | 95 (range 90.37–98.31) | 100 (range 98.46–100) | 99 (range 95.86–99.98) | ||
Testing | Sensitivity | 100 (97.98–100) | 100 (91.96–100) | 100 (75.29–100) | |
Specificity | 100 (93.73–100) | 100 (98.12–100) | 100 (98.37–100) | ||
Accuracy | 100 (98.46–100) | 100 (98.46–100) | 100 (98.46–100) | ||
SVM | Training | Sensitivity | 95.89 (range 88.46–99.14) | 93.02 (range 80.94–98.54) | 75 (range 47–92.73) |
Specificity | 91.53 (range 81.32–97.19) | 94.38 (range 87.37–98.15) | 100 (range 96.87–100) | ||
Accuracy | 94.74 (range 88.41–97.35) | 93.94 (range 88.41–97.35) | 96.97 (range 92.42–99.17) | ||
Testing | Sensitivity | 100 (97.98–100) | 97.73 (87.98–99.94) | 100 (75.29–100) | |
Specificity | 98.25 (90.61–99.96) | 100 (98.12–100) | 100 (98.37–100) | ||
Accuracy | 99.58 (97.68–99.99) | 99.58 (97.68–100) | 100 (98.56–100) | ||
ANN | Training | Sensitivity | 98.63 (range 92.6–99.97) | 83.72 (range 69.3–93.19) | 87.5(range 61.65–98.45) |
Specificity | 88.14 (range 77.07–95.09) | 98.88 (range 93.9–99.97) | 98.28 (range 93.91–99.79) | ||
Accuracy | 93.94 (range 88.41–97.35) | 93.94 (range 88.41–97.35) | 96.97(range 92.42–99.17) | ||
Testing | Sensitivity | 99.43 (96.86–99.99) | 92.5 (76.61–98.43) | 100 (78.2–100) | |
Specificity | 98.18 (90.28–99.95) | 99.47 (97.09–99.99) | 99.07 (96.66–99.89) | ||
Accuracy | 99.13 (96.88–99.89) | 98.25 (95.59–99.52) | 99.13 (96.88–99.89) |
