Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Comment
  • Published:

Harnessing reliability for neuroscience research

Nature Human Behaviour volume 3, pages 768–771 (2019)Cite this article

Subjects

Neuroscientists are amassing the large-scale datasets needed to study individual differences and identify biomarkers. However, measurement reliability within individual samples is often suboptimal, thereby requiring unnecessarily large samples. We focus our comment on reliability in neuroimaging and provide examples of how the reliability can be increased.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to the full article PDF.

USD 39.95

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Interactions between reliability, sample and effect size for a given statistical power.
Fig. 2: Reliability and validity of measurements of individual differences.

Code availability

All code employed in this effort can be found on GitHub at https://github.com/TingsterX/power__reliability_sample_size

References

  1. Laumann, T. O. et al. Neuron 87, 657–670 (2015).

    Article  CAS  Google Scholar 

  2. O’Connor, D. et al. Gigascience 6, 1–14 (2017).

    Article  Google Scholar 

  3. Xu, T., Opitz, A., Craddock, C., Zuo, X. N. & Milham, M. P. Cereb. Cortex 26, 4192–4211 (2016).

    Article  Google Scholar 

  4. Zuo, X. N. & Xing, X. X. Neurosci. Biobehav. Rev. 45, 100–118 (2014).

    Article  Google Scholar 

  5. Kanyongo, G. Y., Brook, G. P., Kyei-Blankson, L. & Gocmen, G. J. Mod. Appl. Stat. Methods 6, Article 9 (2007).

    Article  Google Scholar 

  6. Poldrack, R. A. et al. Nat. Rev. Neurosci. 18, 115–126 (2017).

    Article  CAS  Google Scholar 

  7. Bennett, C. M. & Miller, M. B. Ann. NY Acad. Sci. 1191, 133–155 (2010).

    Article  Google Scholar 

  8. Hedge, C., Powell, G. & Sumner, P. Behav. Res. Methods 50, 1166–1186 (2018).

    Article  Google Scholar 

  9. Button, K. S. et al. Nat. Rev. Neurosci. 14, 365–376 (2013).

    Article  CAS  Google Scholar 

  10. Zuo, X.N., Biswal, B.B. & Poldrack, R.A. Front. Neurosci. 13, 117 (2019).

    Article  Google Scholar 

  11. Tomasi, D. G., Shokri-Kojori, E. & Volkow, N. D. Cereb. Cortex 27, 4153–4165 (2017).

    PubMed  Google Scholar 

  12. Elliott, M. L. et al. Neuroimage 189, 516–532 (2019).

    Article  Google Scholar 

  13. Nichols, T. E. et al. Nat. Neurosci. 20, 299–303 (2017).

    Article  CAS  Google Scholar 

  14. Koo, T. K. & Li, M. Y. J. Chiropr. Med. 15, 155–163 (2016).

    Article  Google Scholar 

  15. Kraemer, H. C. Annu. Rev. Clin. Psychol. 10, 111–130 (2014).

    Article  Google Scholar 

  16. Yan, C. G. et al. Neuroimage 76, 183–201 (2013).

    Article  Google Scholar 

Download references

Acknowledgements

We thank X. Castellanos, A. Franco, H. Kimball, A. Nikolaidis and X.-X. Xing for their helpful comments in the preparation of this commentary, as well as D. Klein for his guidance and encouragement of our focus on issues of reliability over the years, X. Castellanos for his support along the way, and all the contributors from CoRR and R3BRAIN for their enthusiasm on open neuroscience and data sharing. The two consortia are supported in part by the National Basic Research (973) Program (2015CB351702).

Author information

Authors and Affiliations

  1. Department of Psychology, University of Chinese Academy of Sciences (UCAS), Beijing, Beijing, China

    Xi-Nian Zuo

  2. Key Laboratory of Brain and Education, Nanning Normal University, Nanning, Guangxi, China

    Xi-Nian Zuo

  3. Institute for Brain Research and Rehabilitation, South China Normal University, Guangzhou, China

    Xi-Nian Zuo

  4. CAS Key Laboratory of Behavioral Sciences, Research Center for Lifespan Development of Mind and Brain (CLIMB) and Magnetic Resonance Imaging Research Center, CAS Institute of Psychology, Beijing, Beijing, China

    Xi-Nian Zuo & Ting Xu

  5. Center for Developing Brain, Child Mind Institute, New York, NY, USA

    Ting Xu & Michael Peter Milham

  6. Center for Biomedical Imaging and Neuromodulation, Nathan S. Kline Institute for Psychiatric Research, Orangeburg, NY, USA

    Michael Peter Milham

Authors
  1. Xi-Nian Zuo
    View author publications

    Search author on:PubMed Google Scholar

  2. Ting Xu
    View author publications

    Search author on:PubMed Google Scholar

  3. Michael Peter Milham
    View author publications

    Search author on:PubMed Google Scholar

Corresponding author

Correspondence to Xi-Nian Zuo.

Ethics declarations

Competing interests

The authors declare no competing interests.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zuo, XN., Xu, T. & Milham, M.P. Harnessing reliability for neuroscience research. Nat Hum Behav 3, 768–771 (2019). https://doi.org/10.1038/s41562-019-0655-x

Download citation

  • Published:

  • Version of record:

  • Issue date:

  • DOI: https://doi.org/10.1038/s41562-019-0655-x

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing