Table 1 Research design and hypotheses

1. A control analysis—Does limiting report time speed total reaction times?

Perception:

Hypothesis 1.1 Subsetting data to the baseline and time limited contexts in Experiment 1, a main effect of report context on mean reaction time will be observed. Reaction times will be faster in the time limited context than in the baseline context.

Memory:

Hypothesis 1.2 Subsetting data to the baseline and time limited contexts in Experiment 2, a main effect of report context on mean reaction time will be observed. Reaction times will be faster in the time limited context than in the baseline context.

We will stop sampling or recruiting participants when the Bayes factors for Hypothesis 5.1 and Hypothesis 5.2 exceed the strong evidence threshold (10 or 0.1) or when a total of 80 participants have been recruited for each experiment.

Based on sensitivity analyses, expected effect sizes for Hypothesis 1 are greater than for Hypothesis 5 (see Supplementary information for details).

The outcome measure will be the mean reaction time computed for trials in the test phase of all conditions in the baseline and time limited contexts. We will fit a Bayesian hierarchical model with report context included as a fixed effect.

We will use weakly informative priors (see Analysis plan for details).

The results will be consistent with the hypothesis if the Bayes factor (BF₁₀) is ≥ 3 and will be consistent with the null hypothesis if the Bayes factor is ≤ ⅓. Bayes factors larger

than 3 or 10 (or less than ⅓ or

⅒) provide moderate and

strong evidence for the experimental

hypothesis (or null hypothesis). Anecdotal evidence (i.e., BF₁₀ > ⅓ and < 3) will not form valid grounds for interpretation.

2. Does report order affect metacognition independently of report time and motor confounds?

Perception:

Hypothesis 2.1a Subsetting data to the baseline and time limited contexts in Experiment 1, a main effect of report order on metacognitive efficiency will be observed.

Hypothesis 2.1b Subsetting data to the baseline and time limited contexts in Experiment 1, null interaction between report order and report context will be found.

Memory:

Hypothesis 2.2a Subsetting data to the baseline and time limited contexts in Experiment 2, a main effect of report order on metacognitive efficiency will be observed.

Hypothesis 2.2b) Subsetting data to the baseline and time limited contexts in Experiment 2, null interaction between report order and report context will be found.

Same as above.

Based on sensitivity analyses, expected effect sizes for Hypothesis 2 (a and b) are greater than for Hypothesis 5 (see Supplementary information for details).

The outcome measure is metacognitive efficiency (meta-d’/d’) computed for each participant in each condition using the maximum likelihood approach. We will use trials in the test phase of the baseline and time limited contexts. To test for main effects, we will fit a Bayesian hierarchical model with report order, report context, and their interaction included as fixed effects together with individual participant intercepts as random effects. Separate models will be prepared for each context.

We will use weakly informative priors (see Analysis plan for details).

Same as above.

3. Does report time affect metacognitive efficiency?

Perception:

Hypothesis 3.1 Subsetting data to the baseline and time limited contexts in Experiment 1, a main effect of report context will be observed. Metacognitive efficiency will be higher in the baseline than time limited context.

Memory:

Hypothesis 3.2 Subsetting data to the baseline and time limited contexts in Experiment 2, a main effect of report context will be observed. Metacognitive efficiency will be higher in the baseline than time limited context.

Same as above.

Based on sensitivity analyses, expected effect sizes for Hypothesis 3 are greater than for Hypothesis 5 (see Supplementary information for details).

Same as above.

Same as above.

4. Do the effects of report order on metacognitive efficiency differ as a function of report time?

Perception:

Hypothesis 4.1 Subsetting data to the baseline and time limited contexts in Experiment 1, an interaction between report order and report context will be observed.

Memory:

Hypothesis 4.2 Subsetting data to the baseline and time limited contexts in Experiment 2, an interaction between report order and report context will be observed.

Same as above.

Based on sensitivity analyses, expected effect sizes for Hypothesis 4 are identical to Hypothesis 5 (see Simulation and sensitivity analysis for details).

Same as above. In addition, pairwise comparisons will be conducted between each level of report order for each context to test for our a priori interest in these effects.

Same as above.

5. Do the effects of report order on metacognitive efficiency differ as a function of motor preparation?

Perception:

Hypothesis 5.1 Subsetting data to the baseline and motor fixed contexts in Experiment 1, an interaction between report order and report context will be observed.

Memory:

Hypothesis 5.2 Subsetting data to the baseline and motor fixed contexts in Experiment 2, an interaction between report order and report context will be observed.

Same as above.

The sample size for each experiment is determined by simulation and sensitivity analyses using pilot data (see Simulation and sensitivity analysis for details).

The outcome measure is metacognitive efficiency (meta-d’/d’) computed for each participant in each condition using the maximum likelihood approach. We will use trials in the test phase of the baseline and motor fixed contexts. To test for main effects, we will fit a Bayesian hierarchical model with report order, report context, and their interaction included as fixed effects together with individual participant intercepts as random effects. Separate models will be prepared for each context.

Same as above.

6. Does preparing a task decision before confidence rating improve metacognition?

Perception:

Hypothesis 6.1a Subsetting data to the motor fixed context in Experiment 1, a main effect of report order will be observed.

Hypothesis 6.1b In Experiment 1, pairwise contrasts will reveal that metacognitive efficiency is higher in the D → C condition than the C → D condition.

Memory:

Hypothesis 6.2a Subsetting data to the motor fixed context in Experiment 2, a main effect of report order will be observed.

Hypothesis 6.1b In Experiment 2, pairwise contrasts will reveal that metacognitive efficiency is higher in the D → C condition than the C → D condition.

Same as above.

Based on sensitivity analyses, expected effect sizes for Hypothesis 6 are greater than for Hypothesis 5 (see Supplementary information for details).

Same as above but with data from the motor fixed condition only. Pairwise comparisons will be conducted between each level of report order to test for our a priori interest in these effects.

For hypotheses 6.1b and 6.2b, a directional test will be conducted such that metacognitive efficiency will be higher in the D → C condition than the C → D condition.

Same as above.

7. Do the effects of report order and report context on metacognitive efficiency (i.e., hypotheses 2 through 6) differ between perception and memory domains?

Hypothesis 7a Including data from Experiment 1 and Experiment 2, an interaction will be observed between report order, report context, and task domain.

Hypothesis 7b

Including data from Experiment 1 and Experiment 2, an interaction will be observed between report order and task domain.

Hypothesis 7c Including data from Experiment 1 and Experiment 2, an interaction will be observed between report context and task domain.

Same as above.

Based on sensitivity analyses, the expected effect size for Hypothesis 7a is weaker than for Hypothesis 5. The expected effect size for Hypotheses 7b and 7c are equivalent to Hypothesis 5 (see Supplementary information for details).

The outcome measure will be metacognitive efficiency (meta-d’/d’) computed for all trials across Experiment 1 (perception) and Experiment 2 (memory). We will fit a Bayesian hierarchical model with report order, report context, task domain, and their interaction included as fixed effects. Individual participant intercepts will be included as random effects.

Same as above.