Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Comment
  • Published:

A visual tool for defining reproducibility and replicability

Nature Human Behaviour volume 3, pages 650–652 (2019) Cite this article

Subjects

A Publisher Correction to this article was published on 29 July 2019

This article has been updated

Reproducibility and replicability are fundamental requirements of scientific studies. Disagreements over universal definitions for these terms have affected the interpretation of large-scale replication attempts. We provide a visual tool for representing definitions and use it to re-examine these attempts.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to the full article PDF.

USD 39.95

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: A graphic representation for the scientific process
The alternative text for this image may have been generated using AI.

Data and code availability

The scifigure package and its accompanying vignette can be downloaded from https://CRAN.R-project.org/package=scifigure

Change history

  • 29 July 2019

    An amendment to this paper has been published and can be accessed via a link at the top of the paper.

References

  1. Fanelli, D. Proc. Natl. Acad. Sci. USA 115, 2628–2631 (2018).

    Article  CAS  Google Scholar 

  2. Kenett, R. S. & Shmueli, G. Nat. Methods 12, 699 (2015).

    Article  CAS  Google Scholar 

  3. Open Science Collaboration. Science 349, aac4716 (2015).

  4. Peng, R. D. Science 334, 1226–1227 (2011).

    Article  CAS  Google Scholar 

  5. Gilbert, D. T., King, G., Pettigrew, S. & Wilson, T. D. Science 351, 1037 (2016).

    Article  CAS  Google Scholar 

  6. Heller, R., Bogomolov, M. & Benjamini, Y. Proc. Natl. Acad. Sci. USA 111, 16262–16267 (2014).

    Article  CAS  Google Scholar 

  7. Begley, C. G. & Ellis, L. M. Nature 483, 531–533 (2012).

    Article  CAS  Google Scholar 

  8. Cramer, P. E. et al. Science 335, 1503–1506 (2012).

    Article  CAS  Google Scholar 

  9. Klein, R. A. et al. Adv. Methods Pract. Psychol. Sci. 1, 443–490 (2018).

    Article  Google Scholar 

  10. Baggerly, K. A. & Coombes, K. R. Ann. Appl. Stat. 3, 1309–1334 (2009).

    Article  Google Scholar 

  11. Johnson, R. A. Statistics: Principles and Methods (Wiley, 2009).

Download references

Author information

Authors and Affiliations

  1. Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MA, USA

    Prasad Patil

  2. Department of Data Sciences, Dana-Farber Cancer Institute, Boston, MA, USA

    Prasad Patil

  3. Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA

    Roger D. Peng & Jeffrey T. Leek

Authors
  1. Prasad Patil
    View author publications

    Search author on:PubMed Google Scholar

  2. Roger D. Peng
    View author publications

    Search author on:PubMed Google Scholar

  3. Jeffrey T. Leek
    View author publications

    Search author on:PubMed Google Scholar

Corresponding author

Correspondence to Jeffrey T. Leek.

Ethics declarations

Competing interests

The authors declare no competing interests.

Supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Patil, P., Peng, R.D. & Leek, J.T. A visual tool for defining reproducibility and replicability. Nat Hum Behav 3, 650–652 (2019). https://doi.org/10.1038/s41562-019-0629-z

Download citation

  • Published:

  • Version of record:

  • Issue date:

  • DOI: https://doi.org/10.1038/s41562-019-0629-z

This article is cited by

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics