Table 3 Comparison of our methodology to simpler alternative methods.
McNemar’s Test, χ21 | Macro F1-score | Macro unknown recall/sensitivity | Macro unknown precision | |
|---|---|---|---|---|
Full methods | 86.24 ± 2.48% | 94.09 ± 2.42% | 79.66 ± 3.24% | |
Soft voting of all T2 components | χ21 = 332 a p < 0.00001 | 86.87 ± 3.11% | 88.55 ± 3.58% | 85.36 ± 3.93% |
T2–closed-set Random Forest | χ21 = 920 p < 0.00001 | 82.80 ± 3.84% | 83.79 ± 4.86% | 81.97 ± 4.23% |
T2 – closed-set SVM | χ21 = 1120 p < 0.00001 | 82.68 ± 4.51% | 82.93 ± 5.68% | 82.58 ± 4.60% |
T2–closed-set WDNN | χ21 = 905 p < 0.00001 | 81.87 ± 4.53% | 87.34 ± 5.96% | 77.11 ± 3.93% |
Softmax with a threshold | χ21 = 12,151 p < 0.00001 | 72.38 ± 4.43% | 61.81 ± 4.31% | 87.43 ± 5.47% |
Open-set re-mapped | χ21 = 24,656 p < 0.00001 | 72.72 ± 4.28% | 58.03 ± 5.52% | 98.02 ± 1.98% |
ODIN | χ21 = 6414 p < 0.00001 | 49.58 ± 26.02% | 68.87 ± 42.70% | 82.02 ± 5.77% |
