Fig. 5: Size effects in LLM based synoptic reporting.
From: Synoptic reporting by summarizing cancer pathology reports using large language models
a Performance differences in the fine-tuned LLAMA-2 model with variation in the size of the clinical notes. Long reports are clinical notes whose size (in no. of words) exceeds the token limit (4096), whereas short reports are clinical notes whose size (in no. of words) is less than the model’s token limit. Long reports are broken into smaller pieces to fit the model’s token limit, leading to its inability to answer questions accurately and a reduction in performance. The projected accuracy is computed using a threshold of BERT F1 > 0.85 to closely align with manual analysis. b Performance comparison of fine-tuned LLAMA-2 models on the synoptic reporting task as the size of the training data is increased. Two variants of the LLAMA-2 model are tested, the 7 billion parameter version (L-7B) and the 13 billion parameter version (L-13B).
