Table 7 Consistency between extracted histology and IHC/FISH results

From: Iterative refinement and goal articulation to optimize large language models for clinical information extraction

 

Chromophobe RCC

Papillary RCC

CCPRCT

Clear cell RCC

TFE3 Rearranged RCC

TFEB Altered RCC

Total Number of specimensa

84

119

62

1630

6

5

CA-IX

Expected → 

Extracted ↓

Negative

Focal/Patchy Positive or Negative

Positive (Cup-Like)

Positive or Positive (Box-Like)

Negative

Negative

Positive (cup-like)

0

1d

61

3d

0

0

Positive (box-like)

0

0

2c

164

0

0

Focal/patchy positive

0

22

0

6c

0

1

Other positiveb

0

6

3

548

0

0

Negative

24

15

0

2c

5

3

CD117

Expected → 

Extracted ↓

Positive

Negative

Negative

Negative

Negative

Negative

Positive

87

0

0

1d

0

1c

Negative

0

4

6

26

2

3

Racemase

Expected → 

Extracted ↓

Negative

Positive

Negative

Mixed

Mixed

Mixed

Focal/patchy positive

0

2

0

7

0

0

Positive/

diffuse positive

2c

99

0

9

1

2

Negative

3

0

13

4

0

0

TFE3

Expected → 

Extracted ↓

Negative

Negative

Negative

Negative

Rearranged

Negative

Rearranged

0

0

0

0

6

0

Negative

2

7

0

6

0

4

TFEB

Expected → 

Extracted ↓

Negative

Negative

Negative

Negative

Negative

Rearranged/

Amplified

Rearranged/

amplified

0

0

0

0

0

5

Negative

2

6

0

4

6

0

  1. aSingle specimens may have multiple tests, thus column totals may be higher than the number of specimens.
  2. bIncludes “Positive” alone, or with other modifiers not explicitly focal/patchy, cup-like, or box-like.
  3. cReport reviewed and the LLM was correct, thus identifying a typographic mistake in the report free text (Supplementary Table 17 for details).
  4. dReport reviewed and the LLM found to have made a mistake in either histology or IHC/FISH results (Supplementary Table 17 for details).