Table 2 MTurk Cohen Kappa Summary.
Cohen Kappa | Normal | Abnormal | SE(Normal) | SE(Abnormal) | P value |
---|---|---|---|---|---|
Top 10 | 0.838 | 0.725 | 0.019 | 0.023 | < 0.001 |
Top 20 | 0.743 | 0.604 | 0.016 | 0.018 | < 0.001 |
Top 30 | 0.643 | 0.507 | 0.015 | 0.016 | < 0.001 |
Top 40 | 0.557 | 0.443 | 0.013 | 0.014 | < 0.001 |
Top 50 | 0.495 | 0.399 | 0.012 | 0.012 | < 0.001 |
Top 100 | 0.339 | 0.291 | 0.009 | 0.009 | < 0.001 |
Top 200 | 0.203 | 0.196 | 0.006 | 0.006 | 0.401 |
Top 300 | 0.129 | 0.139 | 0.005 | 0.005 | 0.101 |