Table 1 Pairwise comparison of estimates obtained using different study designs.

From: Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences

Design 1

Design 2

No overlap (95% Conf. Ints.)

>100% difference in magnitude (P.E.)

Different significance (95% Conf. Ints.)

Different signs (P.E.)

Significantly different sign (95% Conf. Ints.)

Randomised (R-)

BACI DiD

BACI CA

0.01

0.68

0.27

0.32

0.00

BACI DiD

CI

0.01

0.69

0.27

0.32

0.00

BACI DiD

BA

0.01

0.68

0.29

0.34

0.00

BACI CA

CI

0.00

0.04

0.05

0.01

0.00

BACI CA

BA

0.16

0.82

0.33

0.47

0.06

CI

BA

0.16

0.82

0.30

0.47

0.07

Non-randomised

BACI DiD

BACI CA

0.04

0.58

0.31

0.27

0.00

BACI DiD

CI

0.05

0.61

0.28

0.30

0.01

BACI DiD

BA

0.04

0.61

0.22

0.25

0.01

BACI CA

CI

0.00

0.18

0.08

0.08

0.00

BACI CA

BA

0.14

0.74

0.34

0.36

0.03

CI

BA

0.12

0.71

0.33

0.37

0.02

  1. This shows the proportion of responses in which there were differences in the magnitude (by > 100%) and sign of estimates, and differences in the significance, sign and overlap between associated 95% confidence intervals. For randomised datasets, BACI and CI labels refer to R-BACI and R-CI designs (denoted by ‘R-’). The 100% difference in magnitude criterion is set relative to the smaller estimate.
  2. BA before-after, BACI before-after-control-impact, CI control-impact, DiD difference in differences, CA covariance adjustment, 95% Conf. Ints. refers to 95% confidence intervals, P.E. point estimate.
Back to article page