Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Correspondence
  • Published:

Critiques of network analysis of multivariate data in psychological science

Nature Reviews Methods Primers volume 2, Article number: 90 (2022) Cite this article

Subjects

A recent Primer on the network analysis of multivariate data in psychological science (Borsboom, D. et al. Network analysis of multivariate data in psychological science. Nat. Rev. Methods Primers 1, 58 (2021)) provided an overview of psychometric network analysis, including graphical models, estimation methods for those models and descriptive tools1. These techniques are used for obtaining and examining the statistical associations among psychological variables as a network. However, the Primer omitted or glossed over core critiques of these models and methods, making them appear artificially robust and widely accepted. We highlight four categories of critique: selecting network models when better-suited multivariate methods already exist, adopting study designs that are mismatched to research questions, estimating networks using methods that yield unreliable estimates and interpreting network metrics that are invalid when applied to networks of statistical associations. We point readers to selected critiques that were available when the original Primer was written, and also provide a complete annotated bibliography2 of critiques of psychometric network analysis.

The goal of psychometric network analysis is to understand the relationships among measured psychological variables, including symptoms, beliefs or traits. However, many multivariate methods already exist for this purpose. For example, latent variable models such as factor analysis and structural equation methods already allow researchers to examine the relationships among a set of measured variables, and offer robust ways to evaluate model fit. A series of debates in both clinical and personality psychology have challenged whether psychometric network analysis is the most appropriate method for research questions focused on understanding patterns of variable associations. Some have observed that network models may be identical in many cases to latent variable models; however, because of their structure and number of parameters, network models may overfit and be lacking in parsimony3. Similarly, others note that network models do not provide value beyond a graphical depiction of the correlations or partial correlations among the variables4. Following these observations, we recommend that researchers conduct psychometric network analysis only when it is better matched to their research questions than other existing multivariate analytic methods.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to the full article PDF.

USD 39.95

Prices may be subject to local taxes which are calculated during checkout

References

  1. Borsboom, D. et al. Network analysis of multivariate data in psychological science. Nat. Rev. Methods Primers 1, 58 (2021).

    Article  Google Scholar 

  2. Neal, Z. P. et al. Annotated bibliography of critiques of psychometric network analysis. Preprint at PsyArXiv https://doi.org/10.31234/osf.io/ke6qn (2022).

  3. Krueger, R. F., DeYoung, C. G. & Markon, K. E. Toward scientifically useful quantitative models of psychopathology: the importance of a comparative approach. Behav. Brain Sci. 33, 163–164 (2010).

    Article  Google Scholar 

  4. Asendorpf, J. B. What do the items and their associations refer to in a network approach to personality? Eur. J. Pers. 26, 432–433 (2012).

    Article  Google Scholar 

  5. Bos, E. H. & Wanders, R. B. K. Group-level symptom networks in depression. JAMA Psychiatry 73, 411 (2016).

    Article  Google Scholar 

  6. Bos, F. M. et al. Can we jump from cross-sectional to dynamic interpretations of networks? Implications for the network perspective in psychiatry. Psychother. Psychosom. 86, 175–177 (2017).

    Article  Google Scholar 

  7. Forbes, M. K., Wright, A. G. C., Markon, K. E. & Krueger, R. F. Evidence that psychopathology symptom networks have limited replicability. J. Abnorm. Psychol. 126, 969–988 (2017).

    Article  Google Scholar 

  8. Steinley, D., Hoffman, M., Brusco, M. J. & Sher, K. J. A method for making inferences in network analysis: comment on Forbes, Wright, Markon, and Krueger (2017). J. Abnorm. Psychol. 126, 1000–1010 (2017).

    Article  Google Scholar 

  9. Forbes, M. K., Wright, A. G. C., Markon, K. E. & Krueger, R. F. Quantifying the reliability and replicability of psychopathology network characteristics. Multivariate Behav. Res. 56, 224–242 (2021).

    Article  Google Scholar 

  10. Neal, Z. P. & Neal, J. W. Out of bounds? The boundary specification problem for centrality in psychological networks. Psychol. Methods https://doi.org/10.1037/met0000426 (2021).

    Article  Google Scholar 

  11. Spiller, T. R. et al. On the validity of the centrality hypothesis in cross-sectional between-subject networks of psychopathology. BMC Med. 18, 297 (2020).

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, Michigan State University, East Lansing, MI, USA

    Zachary P. Neal & Jennifer Watling Neal

  2. School of Psychological Sciences, Macquarie University, Sydney, New South Wales, Australia

    Miriam K. Forbes

  3. College of Business, Florida State University, Tallahassee, FL, USA

    Michael J. Brusco

  4. Psychology Department, University of Minnesota, Minneapolis, MN, USA

    Robert Krueger

  5. Department of Psychological and Brain Sciences, University of Iowa, Iowa City, IA, USA

    Kristian Markon

  6. Department of Psychological Sciences, University of Missouri, Columbia, MO, USA

    Douglas Steinley

  7. Department of Psychological and Brain Sciences, Indiana University, Bloomington, IN, USA

    Stanley Wasserman

  8. Department of Psychology, University of Pittsburgh, Pittsburgh, PA, USA

    Aiden G. C. Wright

Authors
  1. Zachary P. Neal
    View author publications

    Search author on:PubMed Google Scholar

  2. Miriam K. Forbes
    View author publications

    Search author on:PubMed Google Scholar

  3. Jennifer Watling Neal
    View author publications

    Search author on:PubMed Google Scholar

  4. Michael J. Brusco
    View author publications

    Search author on:PubMed Google Scholar

  5. Robert Krueger
    View author publications

    Search author on:PubMed Google Scholar

  6. Kristian Markon
    View author publications

    Search author on:PubMed Google Scholar

  7. Douglas Steinley
    View author publications

    Search author on:PubMed Google Scholar

  8. Stanley Wasserman
    View author publications

    Search author on:PubMed Google Scholar

  9. Aiden G. C. Wright
    View author publications

    Search author on:PubMed Google Scholar

Contributions

Z.P.N. conceptualized the paper. Z.P.N., M.K.F. and J.W.N. drafted the proposal and first draft. All authors drafted and reviewed the final paper.

Corresponding author

Correspondence to Zachary P. Neal.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Reviews Methods Primers thanks Antonio Serpa-Barrientos for their contribution to the peer review of this work.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Neal, Z.P., Forbes, M.K., Neal, J.W. et al. Critiques of network analysis of multivariate data in psychological science. Nat Rev Methods Primers 2, 90 (2022). https://doi.org/10.1038/s43586-022-00177-9

Download citation

  • Published:

  • Version of record:

  • DOI: https://doi.org/10.1038/s43586-022-00177-9

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing