Table 5 Baseline entity prediction scores (%, Precision/Recall/F1).

From: The Leaf Clinical Trials Corpus: a new resource for query generation from clinical trial eligibility criteria

Category

Entity

Count

biLSTM + CRF

PubMedBERT

SciBERT

Clinical

Condition

7,087

78.6/78.1/78.3

76.1/79.4/77.7

78.4/83.3/80.8

Contraindication

142

93.7/78.9/85.7

77.4/80.0/78.6

100.0/96.6/98.3

Drug

1,404

76.8/81.3/79.0

74.1/80.9/77.4

73.4/80.9/77.0

Encounter

302

64.1/58.1/60.9

51.7/61.7/56.3

58.3/74.4/65.4

Observation

2,558

74.3/66.1/69.9

67.9/73.5/70.6

72.1/77.6/74.7

Procedure

3,016

68.4/75.5/71.9

67.0/75.9/71.2

71.3/79.4/75.1

Demographic

Age

708

91.3/95.4/93.3

82.4/88.5/85.3

99.1/98.3/98.7

Birth

27

100.0/80.0/88.8

100.0/62.5/76.9

100.0/62.5/76.9

Death

35

33.3/33.3/33.3

0.0/0.0/0.0

100.0/20.0/33.3

Family-Member

147

40.0/19.0/25.8

33.3/55.5/41.6

44.9/61.1/51.7

Language

194

92.5/96.1/94.3

73.8/100.0 84.9

96.6/93.5/95.0

Logical

Negation

952

74.3/82.7/78.2

60.9/73.1/66.4

73.5/82.9/77.9

Qualifier

Assertion

1,157

66.6/62.8/64.7

56.1/58.9/57.5

62.1/65.8/63.9

Modifier

3,464

65.0/58.3/61.5

59.2/64.0/61.5

58.5/65.4/61.8

Polarity

360

82.5/88.0/85.1

74.6/67.4/70.8

81.4/79.5/80.4

Risk

117

93.1/96.4/94.7

91.3/91.3/91.3

95.4/91.3/93.3

Severity

569

86.8/90.8/88.7

76.7/79.5/78.1

86.5/94.1/90.2

Stability

397

84.2/67.6/75.0

79.4/75.0/77.1

75.3/84.7/79.7

Temporal and Comparative

Criteria-Count

33

50.0/66.6/57.1

28.5/40.0/33.3

12.5/20.0/15.5

Eq-Comparison

5,298

83.1/83.8/83.4

81.4/85.0/83.2

85.3/89.3/87.3

Eq-Temporal-Period

2,057

88.7/89.2/88.9

70.0/73.9/71.9

82.6/86.3/84.4

Eq-Temporal-Recency

131

68.7/84.6/75.8

43.4/55.5/48.7

50.0/66.6/57.1

Eq-Temporal-Unit

1,808

95.1/97.6/96.4

97.4/98.1/97.8

98.2/99.4/98.8

Eq-Value

3,835

91.8/95.3/93.5

95.5/96.2/95.9

96.4/97.1/96.7

Other

Location

371

68.5/58.7/63.2

65.4/71.6/68.3

73.4/78.3/75.8

—

Total

56,146

80.2/79.6/79.9

75.3/78.7/77.0

79.0/83.7/81.3

  1. Corpus-level micro-averaged scores are shown in the bottom row. For brevity a representative sample of entities is shown. Count refers to the total count of unique spans annotated in the entire corpus. Entities included in the total count and scores but omitted for brevity are Acuteness, Allergy, Condition-Type, Code, Coreference, Ethnicity, Eq-Operator, Eq-Unit, Indication, Immunization, Insurance, Life-Stage-And-Gender, Organism, Other, Specimen, Study and Provider.
Back to article page