Table 9 Inter-annotator agreement (F1-score) for the annotation of relations. Five different settings are presented.
From: PGxCorpus, a manually annotated corpus for pharmacogenomics
Entity matching: | exact | exact | partial | partial | partial |
|---|---|---|---|---|---|
Considering hierarchies: | none | both | none | both | both |
Considering direction: | yes | yes | yes | yes | no |
isAssociatedWith | 12.6 | 14.3 | 13.2 | 33.3 | 33.3 |
↳influences | 12.8 | 12.8 | 17.7 | 29.3 | 29.8 |
↳causes | 35.8 | 35.2 | 37.6 | 37.2 | 39.6 |
↳decreases | 25.8 | 26.8 | 33.6 | 36.7 | 36.7 |
↳increases | 14.5 | 15.6 | 27.4 | 30.2 | 30.2 |
↳metabolizes | 59.0 | 59.0 | 61.5 | 61.5 | 61.5 |
↳transports | 83.1 | 83.1 | 83.1 | 83.1 | 83.1 |
↳treats | 33.2 | 34.7 | 36.3 | 37.3 | 37.3 |
isEquivalentTo | 39.6 | 40.2 | 40.7 | 41.3 | 62.5 |
Macro average | 47.3 | 47.1 | 50.3 | 53.8 | 57.0 |
