Table 2 Accuracy evaluation of literature mined results in Cancer-Alterome.
Named entity recognition and normalization | Evaluation metric | Count of prediction | |||||
|---|---|---|---|---|---|---|---|
Tools | Entity type | Precision | Recall | F1 Score | Mention | Entity | Normalization |
PubTator20 | Gene | 0.79 | 0.81 | 0.80 | 203,939 | 22,021 | Entrez ID |
PubTator20 | Point mutations and SNPs | 0.81 | 0.81 | 0.83 | 197,595 | 164,144 | rsID |
AGAC-NER24 | General genetic alteration | 0.74 | 0.57 | 0.64 | 3,002,082 | 549 | Dictionary |
AGAC-NER24 | Trigger word | 0.78 | 0.70 | 0.74 | 200,670 | — | — |
OGER++22 | GO | 0.72 | 0.17 | 0.27 | 52,140 | 22,874 | GO ID |
PhenoTagger23 | HPO | 0.79 | 0.70 | 0.74 | 24,261 | 8,345 | HPO ID |
PubTator20 | MeSH | 0.83 | 0.82 | 0.81 | 1,131,933 | 45,119 | MeSH ID |
Relation extraction | Evaluation metric | Count of prediction | |||||
Tools | Relation type | Precision | Recall | F1 Score | Relations | ||
AGAC-RE24 | Theme | 0.87 | 0.84 | 0.91 | 37,411,752 | ||
AGAC-RE24 | Cause | 0.88 | 0.85 | 0.82 | 12,420,489 | ||
Regulatory events identification | Evaluation metric | Count of prediction | |||||
Method | Event type | Precision | Recall | F1 Score | Events | ||
Template match | GARE | 0.84 | 0.96 | 0.90 | 16,681,473 | ||
