Table 4 Stratified tenfolds cross-validation results. Performance estimates (mean ± sd).

From: Predictive modeling of religiosity, prosociality, and moralizing in 295,000 individuals from European and non-European populations

 

Real test data

Randomly permuted test data

Model comparison

 

F

χ2

RMSEA

F

χ2

RMSEA

Predictive accuracy ratio

 
 

Mean (±sd)

Mean (±sd)

Mean (±sd)

Mean (±sd)

Mean (±sd)

Mean (±sd)

Mean (±sd)

Relative sd

EVS Model 1

0.103 (±0.007)

1038.677 (±66.548)

0.037 (±0.001)

7.820 (±0.057)

78 663.257 (±577.933)

0.273 (±0.001)

76.042 (±4.870)

0.064

EVS Model 2.1

0.143 (±0.007)

1438.145 (±70.351)

0.044 (±0.001)

11.054 (±0.069)

111 190.46 (±577.933)

0.324 (±0.001)

77.498 (±3.783)

0.048

EVS Model 2.2

0.092 (±0.008)

927.762 (±78.875)

0.031 (±0.001)

7.633 (±0.057)

76 780.612 (±571.348)

0.252 (±0.001)

83.449 (±7.021)

0.088

WVS Model 1

0.041 (±0.002)

796.215 (±49.161)

0.040 (±0.001)

2.410 (±0.020)

47 204.293 (±384.187)

0.232 (±0.001)

59.511 (±3.693)

0.062

WVS Model 2.1

0.043 (±0.002)

837.374 (±45.850)

0.036 (±0.001)

3.332 (±0.022)

65 284.53 (±429.89)

0.246 (±0.001)

78.196 (±4.291)

0.055

WVS Model 2.2

0.022 (±0.002)

427.923 (±32.391)

0.026 (±0.001)

2.120 (±0.017)

41 533.45 (±326.62)

0.196 (±0.001)

97.604 (±7.309)

0.075

  1. At each iteration of each model, the model-implied covariance matrix estimated from the training data is compared to the covariance matrix directly observed in the test data (real and randomly permuted). Matrix comparison is achieved by means of a weighted-least square discrepancy function F. The closer to zero the F-value, the greater the predictive power of the model. For facilitating the interpretation of the cross-validation results, the F-value is converted into a χ2 metrics. This conversion is further used to calculate the absolute fit index RMSEA (using the degree of freedom involved in the training model). Finally, a predictive accuracy ratio is calculated to allow for model comparison. It indicates how much time a model is better at predicting real data than random data.
Back to article page