Table 15 Model performance-breast cancer dataset.

From: Distilling knowledge from graph neural networks trained on cell graphs to non-neural student models

Model	Acc_Train ± std	Acc_Val ± std	Acc_test ± std	F1_train ± std	F1_Val ± std	F1_test ± std
Teacher Model	0.9818 ± 0.0021	0.9466 ± 0.0013	0.9111 ± 0.0138	0.9818 ± 0.0021	0.9460 ± 0.0013	0.9059 ± 0.0159
ExtraTrees trained on hard labels	0.9448 ±0.0020	0.9447 ±0.001	0.90392 ± 0.007	0.9449 ± 0.0019	0.94487 ± 0.0011	0.90752 ±0.006
ExtraTrees trained on logits	0.95027 ± 0.0031	0.9438 ± 0.00533	0.95774064 ± 0.00344	0.95031 ± 0.0031	0.94384 ± 0.0053	0.95788 ± 0.00344
ExtraTrees trained on Calibrated probs using IR	0.9512 ± 0.0025	0.94827 ± 0.0061	0.9584 ± 0.0037	0.9513 ± 0.0026	0.9483 ± 0.0061	0.9587 ± 0.0037
ExtraTrees trained on Calibrated probs using temp scaling-BS	0.95000 ± 0.0030	0.9464 ± 0.004519	0.95623 ± 0.003135	0.9500 ± 0.0030	0.94645 ± 0.0045	0.9565 ± 0.0032
ExtraTrees trained on Calibrated probs using temp scaling-LL	0.95051 ± 0.002	0.94643 ± 0.0041	0.95751 ± 0.0023	0.950 ± 0.0027	0.9464 ± 0.00422	0.95777 ± 0.0024
XGBoost trained on hard labels	0.9683 ± 0	0.9614 ± 1.11e-16	0.90708 ± 0	0.9682 ± 0	0.9613 ± 1.11e-16	0.9105 ± 0
XGBoost trained on logits	0.9575 ± 0	0.9508 ± 0	0.9611 ± 0	0.9574 ±0	0.9507 ± 0	0.9613 ±0
XGBoost trained on calibrated probs using IR	0.96689 ± 0.0000	0.95959 ± 0.0000	0.9085 ± 0.0000	0.96682 ± 0.0000	0.9595 ± 0.0000	0.9119 ± 0.0000
XGBoost trained on calibrated probs using temp scaling-BS	0.9653 ± 0	0.9573 ± 0	0.9073 ± 0	0.9653 ± 0	0.9572 ± 0	0.9109 ±0
XGBoost trained on calibrated probs using temp scaling-LL	0.9653 ± 0	0.9591 ±0	0.9084 ±0	0.9652 ± 0	0.959 ± 1.110e-16	0.9119 ± 1.1105e-16
HistGrad trained on hard labels	0.9843 ± 0.0002	0.9696 ± 0.0016	0.9102 ± 0.0007	0.984 ± 0.00024	0.9696 ± 0.00164	0.91388 ± 0.0007
HistGrad trained on logits	0.9648 ± 0.00115	0.9579 ±0.0008	0.9568 ±0.00064	0.9648 ± 0.00114	0.9579 ± 0.00087	0.9573 ± 0.00062
HistGrad trained on calibrated probs using IR	0.9743 ± 0.0001	0.9666 ± 0.0005	0.9102 ± 0.0004	0.9743 ± 0.0001	0.9666 ± 0.0005	0.9138 ± 0.0004
HistGrad trained on calibrated probs using temp scaling-BS	0.97440 ± 0.0002	0.9667 ± 0.0009	0.91006 ± 0.0007	0.9743 ± 0.0002	0.9667 ± 0.0009	0.9137± 0.0007
HistGrad trained on calibrated probs using temp scaling-LL	0.9738 ± 0.0001	0.966 ± 0.002	0.9107 ± 0.0006	0.9738 ± 0.0001	0.966 ± 0.0025	0.9142 ± 0.00056
Random Forest trained on hard labels	0.9537±0.0015	0.9469±0.0018	0.8999±0.00168	0.9615±0.001267	0.953739±0.0015	0.904±0.0015
Random Forest trained on logits	0.9557±0.0007	0.9474±0.0016	0.9229±0.0063	0.9558±0.0007	0.9474±0.0016	0.9254±0.0059
Random Forest trained on calibrated probs using IR	0.9688 ± 0.0005	0.9597 ± 0.0002	0.9069 ± 0.0013	0.9688 ± 0.0005	0.9597 ± 0.0001	0.9107 ± 0.0012
Random trained on calibrated probs using temp scaling-BS	0.9688±0.0004	0.9597±0.0008	0.9070±0.0012	0.9687±0.0004	0.9597±0.0008	0.9108±0.0011
Random trained on calibrated probs using temp scaling-LL	0.9688±0.0008	0.9593±0.0013	0.9074±0.0006	0.9688±0.0008	0.9593±0.0013	0.9112±0.0005
LightGBM trained on hard labels	0.9793 ± 0.0	0.9683 ± 0.0	0.9103 ± 0.0	0.9793 ± 0.0	0.9683 ± 0.0	0.9139 ± 0.0
LightGBM trained on logits	0.9648 ± 0.0	0.9555 ± 0.0	0.9626 ± 0.0	0.9648 ± 0.0	0.9554 ± 0.0	0.9629 ± 0.0
LightGBM trained on Calibrated probs using IR	0.9746 ± 0.0000	0.9641 ± 0.0000	0.9101 ± 0.0000	0.9746 ± 0.0000	0.9641 ± 0.0000	0.9138 ± 0.0000
LightGBM trained on Calibrated probs using temp scaling-BS	0.9742 ± 0.0	0.9656 ± 0.0	0.9104 ± 0.0	0.9742 ± 0.0	0.9655 ± 0.0	0.9140 ± 0.0
LightGBM trained on Calibrated probs using temp scaling-LL	0.9742 ± 0.0	0.9656 ± 0.0	0.9112 ± 0.0	0.9742 ± 0.0	0.9656 ± 0.0	0.9148 ± 0.0

Note: The logits represent the raw outputs of the teacher model. IR denotes Isotonic Regression, BS denotes Brier score reduction, LL denotes Log Loss reduction. Values in bold denote the performance of student models that learned well from the teacher model and outperformed their counterparts trained on hard labels. Std denotes the standard deviation

Back to article page

Table 15 Model performance-breast cancer dataset.

Search

Quick links