Table 3 Inter-annotator agreement after annotation round 1, and round 2 for each annotation batch.
From: NLM-Chem, a new resource for chemical entity recognition in PubMed full text literature
| Â | Annotation Round 1 | Annotation Round 2 | ||||
|---|---|---|---|---|---|---|
Batch 1 | Batch 2 | Batch 3 | Batch 1 | Batch 2 | Batch 3 | |
Exact Agreement | 42.20% | 73.70% | 72.90% | 74.1% | 81.8% | 81.2% |
Mismatch ID | 8.60% | 6.50% | 9.80% | 5.6% | 1.0% | 4.3% |
Single - annotation | 49.20% | 19.80% | 17.30% | 20.3% | 17.2% | 14.5% |
