Table 3 Inter-annotator agreement (measured using Cohen’s kappa) between average volunteers (A) and two groups of experienced volunteers (E1 and E2). Region includes both country and state/territories as applicable.
From: AI-assisted tracking of worldwide non-pharmaceutical interventions for COVID-19
| Â | All NPI event types | Top 5 NPI event types | ||||
|---|---|---|---|---|---|---|
A vs E1 | A vs E2 | E1 vs E2 | A vs E1 | A vs E2 | E1 vs E2 | |
Type | 0.63 | 0.69 | 0.80 | 0.81 | 0.77 | 0.85 |
Type + Value | 0.41 | 0.42 | 0.69 | 0.51 | 0.47 | 0.70 |
Date | 0.50 | 0.61 | 0.73 | 0.60 | 0.69 | 0.76 |
Region | 0.99 | 1.00 | 0.99 | 0.98 | 1.00 | 0.98 |
Restriction | 0.36 | 0.43 | 0.74 | 0.74 | 0.58 | 0.69 |
Type + Date | 0.44 | 0.53 | 0.70 | 0.51 | 0.59 | 0.72 |
Type + Value + Date | 0.31 | 0.34 | 0.62 | 0.35 | 0.36 | 0.59 |
Type + Value + Date + Region | 0.30 | 0.33 | 0.62 | 0.35 | 0.36 | 0.59 |
Type + Value + Date + Region + Restriction | 0.26 | 0.29 | 0.61 | 0.34 | 0.35 | 0.59 |
