Table 23 Model performance-CoauthorPhysics dataset.

From: Distilling knowledge from graph neural networks trained on cell graphs to non-neural student models

Model

Acc_Train ± std

Acc_Val ± std

Acc_test ± std

F1_train ± std

F1_Val ± std

F1_test ± std

Teacher Model

0.9982±0.0007

0.9707±0.0012

0.9685±0.0012

0.9982±0.0007

0.9707±0.0012

0.9685±0.0012

ExtraTrees trained on hard labels

0.8368 ± 0.0046

0.8337 ± 0.0024

0.8301 ± 0.0023

0.8226 ± 0.0061

0.8202 ± 0.0035

0.8126 ± 0.0028

ExtraTrees trained on logits

0.8873 ± 0.0037

0.8811 ± 0.0032

0.8817 ± 0.0063

0.8823 ± 0.0049

0.8756 ± 0.0044

0.8758 ± 0.0078

ExtraTrees trained on Calibrated

logits using IR

0.9229 ± 0.0011

0.9145 ± 0.0031

0.9072 ± 0.0037

0.9210 ± 0.0012

0.9122 ± 0.0032

0.9047 ± 0.0039

ExtraTrees trained on Calibrated

logits using temp scaling-BS

0.9229 ± 0.0019

0.9127 ± 0.0032

0.9102 ± 0.0029

0.9210 ± 0.0020

0.9104 ± 0.0033

0.9077 ± 0.0030

ExtraTrees trained on Calibrated

logits using temp scaling-LL

0.9233 ± 0.0017

0.9130 ± 0.0037

0.9102 ± 0.0032

0.9215 ± 0.0018

0.9107 ± 0.0038

0.9077 ± 0.0034

XGBoost trained on hard labels

0.9194 ± 0.0000

0.8957 ± 0.0000

0.8959 ± 0.0000

0.9174 ± 0.0000

0.8928 ± 0.0000

0.8928 ± 0.0000

XGBoost trained on logits

0.8675 ± 0.0000

0.8543 ± 0.0000

0.8544 ± 0.0000

0.8580 ± 0.0000

0.8434 ± 0.0000

0.8431 ± 0.0000

XGBoost trained on calibrated

probs using IR

0.9166 ± 0.0000

0.8935 ± 0.0000

0.8924 ± 0.0000

0.9141 ± 0.0000

0.8898 ± 0.0000

0.8885 ± 0.0000

XGBoost trained on calibrated

probs using temp scaling-BS

0.9178 ± 0.0000

0.8946 ± 0.0000

0.8976 ± 0.0000

0.9154 ± 0.0000

0.8908 ± 0.0000

0.8940 ± 0.0000

XGBoost trained on calibrated

probs using temp scaling-LL

0.9167 ± 0.0000

0.8941 ± 0.0000

0.8944 ± 0.0000

0.9142 ± 0.0000

0.8904 ± 0.0000

0.8908 ± 0.0000

HistGrad trained on hard labels

0.9549 ± 0.0002

0.9296 ± 0.0010

0.9273 ± 0.0013

0.9546 ± 0.0002

0.9287 ± 0.0011

0.9263 ± 0.0013

HistGrad trained on logits

0.9085 ± 0.0004

0.8954 ± 0.0025

0.8954 ± 0.0015

0.9053 ± 0.0004

0.8914 ± 0.0025

0.8912 ± 0.0015

HistGrad trained on calibrated

probs using IR

0.9335 ± 0.0016

0.9164 ± 0.0042

0.9126 ± 0.0033

0.9321 ± 0.0017

0.9143 ± 0.0044

0.9103 ± 0.0036

HistGrad trained on calibrated

probs using temp scaling-BS

0.9341 ± 0.0015

0.9171 ± 0.0017

0.9144 ± 0.0029

0.9327 ± 0.0015

0.9151 ± 0.0018

0.9122 ± 0.0032

HistGrad trained on calibrated

probs using temp scaling-LL

0.9333 ± 0.0028

0.9162 ± 0.0045

0.9139 ± 0.0033

0.9319 ± 0.0030

0.9141 ± 0.0048

0.9117 ± 0.0036

Random Forest trained on

hard labels

0.8243 ± 0.0014

0.8124 ± 0.0018

0.8133 ± 0.0006

0.8150 ± 0.0019

0.8027 ± 0.0027

0.8035 ± 0.0011

Random Forest trained on logits

0.8782 ± 0.0015

0.8668 ± 0.0032

0.8684 ± 0.0010

0.8736 ± 0.0017

0.8613 ± 0.0033

0.8631 ± 0.0011

Random Forest trained on calibrated

probs using IR

0.9138 ± 0.0015

0.8942 ± 0.0007

0.8948 ± 0.0009

0.9117 ± 0.0017

0.8912 ± 0.0008

0.8919 ± 0.0010

Random trained on calibrated

probs using temp scaling-BS

0.9133 ± 0.0019

0.8941 ± 0.0004

0.8934 ± 0.0016

0.9111 ± 0.0021

0.8910 ± 0.0006

0.8903 ± 0.0018

Random trained on calibrated

probs using temp scaling-LL

0.9133 ± 0.0020

0.8943 ± 0.0009

0.8936 ± 0.0025

0.9112 ± 0.0022

0.8912 ± 0.0011

0.8905 ± 0.0027

LightGBM trained on hard labels

0.9537 ± 0.0000

0.9269 ± 0.0000

0.9318 ± 0.0000

0.9533 ± 0.0000

0.9258 ± 0.0000

0.9309 ± 0.0000

LightGBM trained on logits

0.9120 ± 0.0000

0.8977 ± 0.0000

0.8979 ± 0.0000

0.9089 ± 0.0000

0.8939 ± 0.0000

0.8940 ± 0.0000

LightGBM trained on Calibrated

probs using IR

0.9377 ± 0.0000

0.9214 ± 0.0000

0.9176 ± 0.0000

0.9365 ± 0.0000

0.9196 ± 0.0000

0.9156 ± 0.0000

LightGBM trained on Calibrated

probs using temp scaling-BS

0.9363 ± 0.0000

0.9186 ± 0.0000

0.9139 ± 0.0000

0.9350 ± 0.0000

0.9167 ± 0.0000

0.9116 ± 0.0000

LightGBM trained on Calibrated

probs using temp scaling-LL

0.9373 ± 0.0000

0.9198 ± 0.0000

0.9165 ± 0.0000

0.9361 ± 0.0000

0.9179 ± 0.0000

0.9144 ± 0.

  1. Note: The logits represent the raw outputs of the teacher model. IR denotes Isotonic Regression, BS denotes Brier score reduction, and LL denotes log loss reduction. Values in bold denote the performance of student models that learned well from the teacher model and outperformed their counterparts trained on hard labels.Std denotes the standard deviation
Back to article page