Fig. 5: Replication results based on Bayes factors (secondary replication indicators).

The figure plots the one-sided default Bayes factor (BF₊₀) and the replication Bayes factor (BF_R0) for the 26 replications¹¹³. BF₊₀ > 1 favours the hypothesis of an effect in the direction of the original paper, whereas BF₊₀ < 1 favours the null hypothesis of no effect. BF_R0 quantifies the additional evidence provided by the replication results on top of the original evidence. BF_R0 > 1 indicates additional evidence in favour of the alternative over the null, whereas BF_R0 < 1 indicates additional evidence for the null instead. The evidence categories proposed by Jeffreys¹¹⁵ are also shown (from extreme support for the null hypothesis to extreme support for the original hypothesis). Studies within the three panels (top-12, random, bottom-12) are sorted based on the decision market prices as in Fig. 1. The BF₊₀ is above 1 for all 14 replication studies that successfully replicated according to the statistical significance indicator and below 1 for all 12 replication studies that failed to replicate according to the statistical significance indicator. The BF_R0 is above 1 for 13 of the 14 replication studies that replicated according to the statistical significance indicator and below 1 for Cooney et al.⁵⁶ whose estimated relative effect size of 0.36 is the lowest among these 14 studies; the BF_R0 is below 1 for all of the 12 replication studies that failed to replicate according to the statistical significance indicator. The numbers of observations used to estimate BF₊₀ and BF_R0 are the original and replication sample sizes noted on the right as n_O and n_R.