Table 7 Agreement between majority annotation of the crowd and experts.
From: A national-scale land cover reference dataset from local crowdsourcing initiatives in Indonesia
Land cover | Crowd majority annotation (task was to verify if the image shows the given land cover) | Number of crowd-annotated samples | Number of available coincident experts-annotated samples with expert majority (consensus) annotation | Simple agreement (%) between crowd majority annotation and expert majority annotation |
|---|---|---|---|---|
Undisturbed Forest | All | 24360 | 449 | 55 |
Yes | 24283 | 446 | 54.7 | |
No | 77 | 3 | 100 | |
Maybe | 0 | — | — | |
Logged Over Forest | All | 17871 | 438 | 49.8 |
Yes | 8905 | 214 | 58.4 | |
No | 8828 | 224 | 41.5 | |
Maybe | 138 | — | — | |
Tree Based Not Oil Palm | All | 3405 | 80 | 62.5 |
Yes | 1202 | 21 | 28.6 | |
No | 2161 | 59 | 74.6 | |
Maybe | 42 | — | — | |
Shrub | All | 4730 | 90 | 31.1 |
Yes | 4 715 | 89 | 30.3 | |
No | 15 | 1 | 100 | |
Maybe | 0 | — | — | |
Grass or Savanna | All | 1679 | 40 | 55 |
Yes | 1137 | 22 | 18.2 | |
No | 542 | 18 | 100 | |
Maybe | 0 | — | — | |
Oil Palm Monoculture | All | 5488 | NA | NA |
Yes | 3091 | NA | NA | |
No | 2259 | NA | NA | |
Maybe | 138 | — | — | |
Cropland | All | 12267 | NA | NA |
Yes | 6420 | NA | NA | |
No | 5840 | NA | NA | |
Maybe | 7 | — | — |
