Fig. 2: Model Performance on a Challenging Subset of the Test Set.
From: Detecting stigmatizing language in clinical notes with large language models for addiction care
This challenging subset contains stigmatizing terms which could make the clinical note stigmatizing or non-stigmatizing depending on the context it’s surrounded with. Bootstrapped (n = 1000) performance on a subset of 6889 clinical notes from the held-out test set, each containing one or more stigmatizing terms. Labels were assigned based on manual review of contextual usage by addiction care expert (ESA), reflecting whether the term was used in a genuinely stigmatizing or non-stigmatizing manner. Results are reported as mean macro F1 score with 95% bootstrapped confidence intervals.
