Table 1 Variance components for random effects p x r G-studies across subjects and grades.

From: Rater variability and reliability of constructed response questions in New York state high-stakes tests of English language arts and mathematics: implications for educational assessment policy

Task

Source of variability

df

σ²

%

ELA

p

8

0.4167

57.18

grade 4

r

9

0.018

2.47

task 1

pr

72

0.294

40.34

ELA

p

8

0.6235

62.42

grade 4

r

9

0.0821

8.22

task 2

pr

72

0.29.32

29.36

ELA

p

8

0.2753

48.91

grade 6

r

9

0.0907

16.11

task 1

pr

72

0.1969

34.98

ELA

p

8

0.5451

62.14

grade 6

r

9

0.0327

3.73

task 2

pr

72

0.2994

34.13

Math

p

8

0.4580

52.50

grade 4

r

9

0.3946

2.27

task 1

pr

72

0.0198

45.23

Math

p

8

0.3364

60.90

grade 4

r

9

0.0052

0.94

task 2

pr

72

0.2108

38.16

Math

p

8

0.5437

81.65

grade 4

r

9

0.0065

0.98

task 3

pr

72

0.1157

17.38

Math

p

8

0.6167

74.50

grade 4

r

9

0.0083

1.00

task 4

pr

72

0.2028

24.50

Math

p

8

0.5756

81.20

grade 5

r

9

0.0117

1.65

task 1

pr

72

0.1216

17.15

Math

p

8

0.3524

70.58

grade 5

r

9

0.0188

3.77

task 2

pr

72

0.1281

25.66

Math

p

8

0.3762

63.00

grade 5

r

9

0.0139

2.33

task 3

pr

72

0.2071

34.68

Math

p

8

1.1281

91.22

grade 5

r

9

0.0117

1.12

task 4

pr

72

0.0969

7.84

Back to article page