Table 2 Average performance of various ML classifiers, along with standard deviation, across the 10 testing experiments for the Glider scenario, \(\mathscr{D}^g\) (columns 2 through 4), Satellite scenario \(\mathscr{D}^s\) (columns 5 through 7), and Glider + Satellite scenario, \(\mathscr{D}\) (columns 8 through 10)
 | Glider, \(\mathscr{D}^g\) | Satellite, \(\mathscr{D}^s\) | Glider + Satellite, \(\mathscr{D}\) | ||||||
---|---|---|---|---|---|---|---|---|---|
Accuracy | F1 Score | AUC | Accuracy | F1 Score | AUC | Accuracy | F1 Score | AUC | |
LR | 0.961 ± 0.006 | 0.198 ± 0.068 | 0.625 ± 0.035 | 0.965 ± 0.013 | 0.000 ± 0.000 | 0.560 ± 0.036 | 0.972 ± 0.006 | 0.265 ± 0.093 | 0.624 ± 0.041 |
SVM | 0.946 ± 0.004 | 0.216 ± 0.039 | 0.785 ± 0.025 | 0.885 ± 0.014 | 0.133 ± 0.020 | 0.733 ± 0.030 | 0.923 ± 0.009 | 0.231 ± 0.034 | 0.856 ± 0.027 |
kNN | 0.950 ± 0.006 | 0.298 ± 0.037 | 0.736 ± 0.050 | 0.967 ± 0.005 | 0.431 ± 0.053 | 0.792 ± 0.032 | 0.961 ± 0.005 | 0.373 ± 0.039 | 0.791 ± 0.039 |
RF | 0.975 ± 0.007 | 0.374 ± 0.071 | 0.778 ± 0.030 | 0.965 ± 0.005 | 0.411 ± 0.040 | 0.883 ± 0.017 | 0.975 ± 0.003 | 0.524 ± 0.040 | 0.886 ± 0.032 |
AdaBoost | 0.972 ± 0.004 | 0.377 ± 0.052 | 0.815 ± 0.045 | 0.982 ± 0.003 | 0.615 ± 0.058 | 0.866 ± 0.028 | 0.987 ± 0.002 | 0.675 ± 0.054 | 0.904 ± 0.027 |
± GBoost | 0.971 ± 0.005 | 0.359 ± 0.066 | 0.823 ± 0.030 | 0.983 ± 0.003 | 0.641 ± 0.048 | 0.888 ± 0.029 | 0.986 ± 0.003 | 0.649 ± 0.048 | 0.891 ± 0.028 |
MLP | 0.969 ± 0.011 | 0.334 ± 0.091 | 0.779 ± 0.036 | 0.964 ± 0.006 | 0.358 ± 0.057 | 0.803 ± 0.032 | 0.971 ± 0.008 | 0.421 ± 0.080 | 0.863 ± 0.024 |
CNN | 0.977 ± 0.009 | 0.344 ± 0.091 | 0.764 ± 0.036 | 0.943 ± 0.015 | 0.267 ± 0.064 | 0.801 ± 0.033 | 0.955 ± 0.010 | 0.345 ± 0.072 | 0.842 ± 0.025 |
ResNet | 0.943 ± 0.002 | 0.223 ± 0.108 | 0.769 ± 0.043 | 0.933 ± 0.038 | 0.186 ± 0.065 | 0.735 ± 0.022 | 0.935 ± 0.031 | 0.289 ± 0.096 | 0.862 ± 0.031 |