Table 2 Phenotype classification performance for severe asthma

From: A weakly supervised transformer for rare disease diagnosis and subphenotyping from EHRs with pulmonary case studies

Metric

Count

KOMAP

XGBoost

Transformer (silver = gold)

Transformer (gold only)

WEST (w/o neg)

WEST (w/ neg)

AUC

0.80 (0.71–0.89)

0.82 (0.74–0.90)

0.83 (0.74–0.90)

0.82 (0.74–0.91)

0.78 (0.69–0.87)

0.85 (0.69–0.89)

0.87 (0.78–0.92)

F1 Score

0.78 (0.69–0.87)

0.81 (0.72–0.88)

0.82 (0.73–0.89)

0.77 (0.68–0.84)

0.76 (0.67–0.84)

0.78 (0.69–0.86)

0.80 (0.70–0.87)

PPV

0.74 (0.62–0.85)

0.74 (0.63–0.84)

0.78 (0.66–0.88)

0.70 (0.56–0.78)

0.69 (0.58–0.81)

0.75 (0.62–0.84)

0.80 (0.65–0.87)

Specificity

0.68 (0.55–0.81)

0.66 (0.52–0.79)

0.72 (0.59–0.84)

0.59 (0.38–0.67)

0.60 (0.45–0.72)

0.69 (0.54–0.81)

0.76 (0.59–0.85)

  1. WEST trained with both positive and negative gold-standard labels, denoted WEST (w/ neg), achieved the highest AUC, PPV, and specificity across all methods. The Transformer (silver = gold) baseline was trained by treating all silver-standard labels as gold-standard (i.e., no iterative updates or augmentation), while Transformer (gold only) used only expert-validated labels. Transformer metrics were averaged across two cross-validation folds, and all metrics are reported with 95% confidence intervals estimated by bootstrapping on patient-level predictions. Bold values denote the best performance per metric.
Back to article page