Table 2 Inter-human-rater reliability results by categories

From: Evaluating large language models in analysing classroom dialogue

Code

Cohen Kappa’s (κ) value

ELI

0.807

EL

0.739

REI

0.821

RE

0.853

CI

0.724

SC

0.716

RC

0.628

A

0.853

Q

0.745

RB

0.697

RW

0.646

SU

0.782

SA

0.830

OI

0.761

O

0.742

  1. This table presents the inter-human-rater reliability results for various coding categories, measured by Cohen's Kappa (κ) value. The values indicate the level of agreement between human coders for each code, with higher values representing greater reliability. According to the general interpretation standards for Cohen's Kappa, values between 0.61 and 0.80 are considered substantial, and values between 0.81 and 1.00 are considered almost perfect.
Back to article page