Fig. 1: Overview of the various existing and proposed workflows.

EHR = Electronic Health Records, CDM = Common Data Model, PPV = Positive Predictive Value. a Current status quo: in EHR systems, chart review is typically only applied to identified cases, allowing estimation of the PPV only. b The KEEPER system replaces charts with summaries from structured data, allowing for more efficient review for a wider variety of data. c Here we propose to use KEEPER with LLMS, further reducing manual effort. d By using KEEPER with LLMS, review can be scaled up to include non-cases, allowing computation of both PPV and sensitivity.