Table 4 Model’s generalization capability between different regions.
From: Measuring and classifying IP usage scenarios: a continuous neural trees approach
Region | Sichuan \(\rightarrow\) Chongqing | Chongqing \(\rightarrow\) Sichuan | Shandong \(\rightarrow\) Sichuan | Shandong \(\rightarrow\) Chongqing | ||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Metric | Precision | Recall | Auc | Precision | Recall | Auc | Precision | Recall | Auc | Precision | Recall | Auc |
SVM | 0.6714 | 0.6725 | 0.8885 | 0.7993 | 0.7457 | 0.9382 | 0.8671 | 0.7530 | 0.9390 | 0.9114 | 0.7777 | 0.9645 |
NB | 0.5641 | 0.5655 | 0.8650 | 0.6524 | 0.6590 | 0.8861 | 0.6726 | 0.6167 | 0.8928 | 0.7575 | 0.6547 | 0.9270 |
LDA | 0.6809 | 0.6412 | 0.8208 | 0.6621 | 0.6417 | 0.8701 | 0.6606 | 0.5824 | 0.8878 | 0.8036 | 0.6134 | 0.9258 |
RF | 0.6246 | 0.7799 | 0.9377 | 0.7709 | 0.7059 | 0.9106 | 0.8777 | 0.7098 | 0.8928 | 0.8930 | 0.7984 | 0.9632 |
XgBoost | 0.6051 | 0.7543 | 0.9376 | 0.7360 | 0.6628 | 0.9118 | 0.8770 | 0.7065 | 0.9431 | 0.9001 | 0.8125 | 0.9468 |
CatBoost | 0.6137 | 0.7697 | 0.9038 | 0.6136 | 0.4891 | 0.8345 | 0.7400 | 0.6616 | 0.8387 | 0.7589 | 0.7403 | 0.8876 |
TabNet | 0.6336 | 0.7797 | 0.9158 | 0.7326 | 0.6936 | 0.9054 | 0.8063 | 0.7327 | 0.9097 | 0.8261 | 0.7920 | 0.9349 |
NON | 0.6838 | 0.7384 | 0.8825 | 0.7097 | 0.7264 | 0.9041 | 0.8096 | 0.7639 | 0.9191 | 0.8013 | 0.8164 | 0.9267 |
AutoInt | 0.5887 | 0.7402 | 0.8250 | 0.6802 | 0.6232 | 0.7333 | 0.8618 | 0.6741 | 0.7856 | 0.8384 | 0.7623 | 0.8212 |
NODE | 0.6315 | 0.7778 | 0.9145 | 0.7321 | 0.6920 | 0.9034 | 0.8051 | 0.7319 | 0.9025 | 0.8245 | 0.7911 | 0.9335 |
ODTSR | 0.7295 | 0.8042 | 0.9462 | 0.8409 | 0.7714 | 0.9444 | 0.8817 | 0.7679 | 0.9582 | 0.9322 | 0.8654 | 0.9763 |