Table 5 Inter-rater agreement

From: Question-based computational language approach outperforms rating scales in quantifying emotional states

Phase

Model

Agreement

Correct

N

2(-P)

RS

90

79

263

2(-P)

Words

82

65

263

1 & 2 (-P)

RS

83

69

721

1 & 2 (-P)

Words

82

66

721

  1. The table shows the percentage of agreement between all pairwise categorizations of the same narratives for different rater models and phases. The columns show the phase, model, agreement (i.e., the percentage of identical categorizations), correct (i.e., the percentage agreement of correct categorizations), and the number of pairwise comparisons. The following abbreviations were used: non-professionals (-P), and rating scales (RS).
Back to article page