Table 1 Model performance under temporal split setting. Performance metrics are reported as mean (s.d.)

From: Matching clinicians with clinical trials using AI

Model

CS@GT (↑)

CS@5 (↑)

CS@10 (↑)

CS@20 (↑)

Ground Truth

0.14 (—)

0.30 (—)

0.21 (—)

0.16 (—)

Random

0.05 (0.12)

0.01 (0.18)

0.07 (0.23)

−0.05 (0.30)

Popular

0.18 (—)

0.17 (—)

0.24 (—)

0.12 (—)

LightGCN

0.30 (0.04)

0.27 (0.05)

0.25 (0.03)

0.26 (0.04)

ConvNCF

0.33 (0.03)

0.34 (0.02)

0.35 (0.02)

0.35 (0.04)

NCL

0.37 (0.05)

0.38 (0.05)

0.37 (0.04)

0.37 (0.03)

Doc2Vec

0.30 (0.02)

0.32 (0.02)

0.32 (0.03)

0.33 (0.03)

FRAMM

0.35 (0.03)

0.37 (0.03)

0.37 (0.02)

0.36 (0.04)

DocTr-semantic

0.42 (0.04)

0.43 (0.04)

0.43 (0.03)

0.43 (0.03)

DocTr-structural

0.36 (0.03)

0.37 (0.02)

0.36 (0.04)

0.35 (0.04)

DocTr

0.60 (0.02)*

0.60 (0.02)*

0.60 (0.04)*

0.60 (0.03)*

  1. *The performance difference is statistically significant (P < 0.01). Bold numbers indicate best performance. We evaluate the performance of top K recommended clinicians, shown as CS@K. CS@GT sets K to the actual number of clinicians who enrolled in each trial, thereby varying K across different trials.