Table 2 Proposed terminology for confirmation of results on agricultural research on climate change, and the equivalent term from the National Academies of Sciences, Engineering, and Medicine (NASEM) report⁶

Repeatability

The ability to obtain essentially identical results when an experiment or analysis is repeated under identical conditions using the same instruments, data processing workflows, and modeling or numerical analyses. Confirmation is conducted by the original research group.

Considered only as “measurement repeatability” in relation to when a measurement is repeated by the same operator using the same instrument under constant conditions and close in time. (p. 48)

Replicability

The ability to obtain essentially identical results when an experiment or analysis is repeated under similar conditions as the original study, typically using the same type of instruments, data processing workflows, and modeling or analyses. Confirmation is conducted by the original research group.

Obtaining consistent results across studies aimed at answering the same scientific question, each of which has obtained its own data. (p. 46)

Reproducibility

The ability to obtain results from independent studies that are similar enough that they are considered to confirm the findings of previous experiments or analyses. This usually implies somewhat similar growing conditions, treatments, measurements, data processing workflows, and modeling or analyses. In modeling and numerical analyses, this may involve two approaches: using the original dataset but different software or using both different data and software. Conducted by an independent research group.

Obtaining consistent results using the same input data; computational steps, methods, and code; and conditions of analysis. This definition is synonymous with “computational reproducibility,” and the terms are used interchangeably in this report. (p. 46)