Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Primer
  • Published:

Systematic review and meta-analysis of preclinical studies

Nature Reviews Methods Primers volume 4, Article number: 72 (2024) Cite this article

Subjects

Abstract

The preclinical research community faces an ever-expanding corpus of biomedical literature, making it challenging to keep abreast with the latest findings. This hampers evidence-based research and informed decision-making. Thus, reliable tools are warranted to manage this evidence and maximize the global investment in research. Systematic reviews, syntheses of existing scientific evidence that address a focused question in an unbiased manner and using explicit methods, have gained momentum as an effective solution. Systematic reviews have an important role in uncovering problems in preclinical research, informing best practice guidelines, reducing research waste, promoting reproducibility and guiding translational research. Systematic reviews of preclinical studies also promote ethical animal use by maximizing the use of existing animal studies, thereby fostering animal welfare. However, poorly performed systematic reviews can produce unreliable results, leading to incorrect conclusions about the underlying literature. This Primer presents guidance for conducting a rigorous systematic review with or without meta-analysis of preclinical studies including animal and in vitro studies. It also discusses the limitations of systematic reviews and outlines current developments such as systematic review automation. By following this Primer, researchers can ensure the rigour and usefulness of their systematic reviews, ultimately benefiting decision-making and research outcomes in preclinical research.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to the full article PDF.

USD 39.95

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Evidence pyramid for human health.
Fig. 2: Ten steps of a systematic review.
Fig. 3: Exemplary PRISMA flow diagram for study inclusion.
Fig. 4: Example of a forest plot of eight estimated mean differences between the experimental and control arms, with 95% confidence intervals and random effects model weights.

Similar content being viewed by others

References

  1. de Vries, R. B. et al. The usefulness of systematic reviews of animal experiments for the design of preclinical and clinical studies. ILAR J. 55, 427–437 (2014).

    Article  Google Scholar 

  2. Higgins, J. P. et al. Cochrane Handbook for Systematic Reviews of Interventions (Wiley, 2019).

  3. Campbell, M. et al. Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline. BMJ 368, 16890 (2020).

    Google Scholar 

  4. Murad, M. H., Asi, N., Alsawas, M. & Alahdab, F. New evidence pyramid. Evid. Based Med. 21, 125–127 (2016).

    Article  Google Scholar 

  5. Chalmers, I. The Cochrane Collaboration: preparing, maintaining, and disseminating systematic reviews of the effects of health care. Ann. NY Acad. Sci. 703, 156–163 (1993).

    Article  ADS  Google Scholar 

  6. McCann, U. D., Seiden, L. S., Rubin, L. J. & Ricaurte, G. A. Brain serotonin neurotoxicity and primary pulmonary hypertension from fenfluramine and dexfenfluramine: a systematic review of the evidence. JAMA 278, 666–672 (1997).

    Article  Google Scholar 

  7. Hunniford, V. T. et al. Epidemiology and reporting characteristics of preclinical systematic reviews. PLoS Biol. 19, e3001177 (2021). This study demonstrates that a considerable number of preclinical systematic reviews are not reported according to the gold standard, which provides evidence to inform the development of guidelines for conducting and reporting preclinical systematic reviews.

    Article  Google Scholar 

  8. Ioannidis, J. P. Systematic reviews for basic scientists: a different beast. Physiol. Rev. 103, 1–5 (2022).

    Article  Google Scholar 

  9. Langendam, M. W. et al. Developing a database of systematic reviews of animal studies. Regul. Toxicol. Pharmacol. 123, 104940 (2021).

    Article  Google Scholar 

  10. Bahor, Z. et al. Development and uptake of an online systematic review platform: the early years of the CAMARADES Systematic Review Facility (SyRF). BMJ Open. Sci. 5, e100103 (2021).

    Article  Google Scholar 

  11. Ritskes-Hoitinga, M., Leenaars, M., Avey, M., Rovers, M. & Scholten, R. Systematic reviews of preclinical animal studies can make significant contributions to health care and more transparent translational medicine. Cochrane Database Syst. Rev. 2014, ED000078 (2014).

    Google Scholar 

  12. Ritskes-Hoitinga, M. & Pound, P. The role of systematic reviews in identifying the limitations of preclinical animal research, 2000–2022: part 1. J. R. Soc. Med. 115, 186–192 (2022).

    Article  Google Scholar 

  13. Ritskes-Hoitinga, M. & Pound, P. The role of systematic reviews in identifying the limitations of preclinical animal research, 2000–2022: part 2. J. R. Soc. Med. 115, 231–235 (2022).

    Article  Google Scholar 

  14. Rosso, M., Doneva, S. E., Howells, D. W., Leenaars, C. H. & Ineichen, B. V. Summer school for systematic reviews of animal studies: fostering evidence-based and rigorous animal research. Altex 41, 131–134 (2024).

    Google Scholar 

  15. Sena, E. S., Currie, G. L., McCann, S. K., Macleod, M. R. & Howells, D. W. Systematic reviews and meta-analysis of preclinical studies: why perform them and how to appraise them critically. J. Cereb. Blood Flow. Metab. 34, 737–742 (2014).

    Article  Google Scholar 

  16. Ioannidis, J. P. et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet 383, 166–175 (2014).

    Article  Google Scholar 

  17. Macleod, M. R. et al. Biomedical research: increasing value, reducing waste. Lancet 383, 101–104 (2014).

    Article  Google Scholar 

  18. Ineichen, B. V., Furrer, E., Grüninger, S. L., Zürrer, W. E. & Macleod, M. R. Analysis of animal-to-human translation shows that only 5% of animal-tested therapeutic interventions obtain regulatory approval for human applications. PLoS Biol. 22, e3002667 (2024). This work is a large systematic review on the proportions of animal to human translation of therapies, showing that only 5% of therapies tested in animals eventually obtain regulatory approval.

    Article  Google Scholar 

  19. Perel, P. et al. Comparison of treatment effects between animal experiments and clinical trials: systematic review. BMJ 334, 197 (2007).

    Article  Google Scholar 

  20. Besselink, M. G. et al. Probiotic prophylaxis in predicted severe acute pancreatitis: a randomised, double-blind, placebo-controlled trial. Lancet 371, 651–659 (2008).

    Article  Google Scholar 

  21. Hooijmans, C. R., de Vries, R. B., Rovers, M. M., Gooszen, H. G. & Ritskes-Hoitinga, M. The effects of probiotic supplementation on experimental acute pancreatitis: a systematic review and meta-analysis. PLoS ONE 7, e48811 (2012).

    Article  ADS  Google Scholar 

  22. Van Der Worp, H. B., Sena, E. S., Donnan, G. A., Howells, D. W. & Macleod, M. R. Hypothermia in animal models of acute ischaemic stroke: a systematic review and meta-analysis. Brain 130, 3063–3074 (2007).

    Article  Google Scholar 

  23. Ritskes-Hoitinga, M. & van Luijk, J. How can systematic reviews teach us more about the implementation of the 3Rs and animal welfare? Animals 9, 1163 (2019).

    Article  Google Scholar 

  24. Ioannidis, J. P. The mass production of redundant, misleading, and conflicted systematic reviews and meta‐analyses. Milbank Q. 94, 485–514 (2016).

    Article  Google Scholar 

  25. Soliman, N., Rice, A. S. & Vollert, J. A practical guide to preclinical systematic review and meta-analysis. Pain 161, 1949 (2020).

    Article  Google Scholar 

  26. Munn, Z., Stern, C., Aromataris, E., Lockwood, C. & Jordan, Z. What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences. BMC Med. Res. Methodol. 18, 1–9 (2018).

    Article  Google Scholar 

  27. Leenaars, M. et al. A step-by-step guide to systematically identify all relevant animal studies. Lab. Anim. 46, 24–31 (2012). This study presents an easy-to-use, step-by-step guide to comprehensive literature searching of animal studies, forming a solid basis for any preclinical systematic review.

    Article  Google Scholar 

  28. Bannach-Brown, A. et al. Navigating PROSPERO4animals: 10 top tips for efficient pre-registration of your animal systematic review protocol. BMC Med. Res. Methodol. 24, 20 (2024). This work presents extensive guidance to help systematic review authors develop and register their animal systematic review protocol in the leading register for animal systematic review protocols.

    Article  Google Scholar 

  29. Sideri, S., Papageorgiou, S. N. & Eliades, T. Registration in the international prospective register of systematic reviews (PROSPERO) of systematic review protocols was associated with increased review quality. J. Clin. Epidemiol. 100, 103–110 (2018).

    Article  Google Scholar 

  30. Kerr, N. L. HARKing: hypothesizing after the results are known. Pers. Soc. Psychol. Rev. 2, 196–217 (1998).

    Article  Google Scholar 

  31. Vries, R. et al. A protocol format for the preparation, registration and publication of systematic reviews of animal intervention studies. Evid. Based Preclin. Med. 2, 1–9 (2015).

    Article  Google Scholar 

  32. Popay, J. et al. Guidance on the conduct of narrative synthesis in systematic reviews. A product from the ESRC methods programme Version 1 bq2 (Lancaster University, 2006).

  33. Hooijmans, C. R., IntHout, J., Ritskes-Hoitinga, M. & Rovers, M. M. Meta-analyses of animal studies: an introduction of a valuable instrument to further improve healthcare. ILAR J. 55, 418–426 (2014).

    Article  Google Scholar 

  34. Vesterinen, H. M. et al. Meta-analysis of data from animal studies: a practical guide. J. Neurosci. Methods 221, 92–102 (2014).

    Article  Google Scholar 

  35. Hooijmans, C. R. et al. Assessment of key characteristics, methodology, and effect size measures used in meta‐analysis of human‐health‐related animal studies. Res. Synth. Methods 13, 790–806 (2022).

    Article  Google Scholar 

  36. Yang, Y., Macleod, M., Pan, J., Lagisz, M. & Nakagawa, S. Advanced methods and implementations for the meta-analyses of animal models: current practices and future recommendations. Neurosci. Biobehav. Rev. 146, 105016 (2023). This up-to-date technical tutorial about the state-of-the-art statistical methodology in meta-analysis of animal studies includes useful empirical evidence and an annotated R code to demonstrate the implementation of the presented methods.

    Article  Google Scholar 

  37. Egger, M., Juni, P., Bartlett, C., Holenstein, F. & Sterne, J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol. Assess. 7, 1–76 (2003).

    Article  Google Scholar 

  38. de Vries, R. B., Hooijmans, C. R., Tillema, A., Leenaars, M. & Ritskes-Hoitinga, M. Updated version of the Embase search filter for animal studies. Lab. Anim. 48, 88 (2014).

    Google Scholar 

  39. Hooijmans, C. R., Tillema, A., Leenaars, M. & Ritskes-Hoitinga, M. Enhancing search efficiency by means of a search filter for finding all studies on animal experimentation in PubMed. Lab. Anim. 44, 170–175 (2010).

    Article  Google Scholar 

  40. van der Mierden, S. et al. Laboratory animals search filter for different literature databases: PubMed, Embase, Web of Science and PsycINFO. Lab. Anim. 56, 279–286 (2022).

    Article  Google Scholar 

  41. Cassidy, L. C., Leenaars, C. H., Rincon, A. V. & Pfefferle, D. Comprehensive search filters for retrieving publications on nonhuman primates for literature reviews (filterNHP). Am. J. Primatol. 83, e23287 (2021).

    Article  Google Scholar 

  42. Mahood, Q., Van Eerd, D. & Irvin, E. Searching for grey literature for systematic reviews: challenges and benefits. Res. Synth. Methods 5, 221–234 (2014).

    Article  Google Scholar 

  43. Scherer, R. W. & Saldanha, I. J. How should systematic reviewers handle conference abstracts? A view from the trenches. Syst. Rev. 8, 264 (2019).

    Article  Google Scholar 

  44. Hackenbroich, S., Kranke, P., Meybohm, P. & Weibel, S. Include or not to include conference abstracts in systematic reviews? Lessons learned from a large Cochrane network meta-analysis including 585 trials. Syst. Rev. 11, 178 (2022).

    Article  Google Scholar 

  45. McGowan, J. et al. PRESS peer review of electronic search strategies: 2015 guideline statement. J. Clin. Epidemiol. 75, 40–46 (2016).

    Article  Google Scholar 

  46. Hair, K., Bahor, Z., Macleod, M., Liao, J. & Sena, E. S. The Automated Systematic Search Deduplicator (ASySD): a rapid, open-source, interoperable tool to remove duplicate citations in biomedical systematic reviews. BMC Biol. 21, 189 (2023).

    Article  Google Scholar 

  47. Yaffe, J., Montgomery, P., Hopewell, S. & Shepard, L. D. Empty reviews: a description and consideration of Cochrane systematic reviews with no included studies. PLoS ONE 7, e36626 (2012).

    Article  ADS  Google Scholar 

  48. Bannach-Brown, A. et al. Machine learning algorithms for systematic review: reducing workload in a preclinical review of animal studies and reducing human screening error. Syst. Rev. 8, 23 (2019).

    Article  Google Scholar 

  49. Bannach-Brown, A. et al. Technological advances in preclinical meta-research. BMJ Open. Sci. 5, e100131 (2021).

    Article  Google Scholar 

  50. Bannach-Brown, A. & Clark, J. Systematic review & meta-analysis: automation tools to help your review. in 16th Australian Conference on Personality and Individual Differences: Advances and Challenges in Personality and Individual Differences–Theories and Applications (ACPID, 2018).

  51. Harrison, H., Griffin, S. J., Kuhn, I. & Usher-Smith, J. A. Software tools to support title and abstract screening for systematic reviews in healthcare: an evaluation. BMC Med. Res. Methodol. 20, 7 (2020).

    Article  Google Scholar 

  52. van de Schoot, R. et al. An open source machine learning framework for efficient and transparent systematic reviews. Nat. Mach. Intell. 3, 125–133 (2021).

    Article  Google Scholar 

  53. Halman, A. & Oshlack, A. Catchii: empowering literature review screening in healthcare. Res. Synth. Methods 15, 157–165 (2024).

    Article  Google Scholar 

  54. Kellermeyer, L., Harnke, B. & Knight, S. Covidence and Rayyan. J. Med. Libr. Assoc 106, 580 (2018).

    Article  Google Scholar 

  55. Cagnoni, S. et al. Implementing AI vertical use cases—scenario 1. EFSA Support. Publ. 20, 8223E (2023).

    Google Scholar 

  56. Ouzzani, M., Hammady, H., Fedorowicz, Z. & Elmagarmid, A. Rayyan—a web and mobile app for systematic reviews. Syst. Rev. 5, 210 (2016).

    Article  Google Scholar 

  57. Van der Mierden, S., Tsaioun, K., Bleich, A. & Leenaars, C. H. Software tools for literature screening in systematic reviews in biomedical research. Altex 36, 508–517 (2019).

    Google Scholar 

  58. Goldet, G. & Howick, J. Understanding GRADE: an introduction. J. Evid. Based Med. 6, 50–54 (2013).

    Article  Google Scholar 

  59. Wei, D. et al. The use of GRADE approach in systematic reviews of animal studies. J. Evid. Based Med. 9, 98–104 (2016).

    Article  Google Scholar 

  60. Perkel, J. M. Six tips for better spreadsheets. Nature 608, 229 (2022).

    Article  ADS  Google Scholar 

  61. Broman, K. W. & Woo, K. H. Data organization in spreadsheets. Am. Statist. 72, 2–10 (2018).

    Article  MathSciNet  Google Scholar 

  62. Eggel, M. & Würbel, H. Internal consistency and compatibility of the 3Rs and 3Vs principles for project evaluation of animal research. Lab. Anim. 55, 233–243 (2021).

    Article  Google Scholar 

  63. Egger, M., Higgins, J. P. & Smith, G. D. Systematic Reviews in Health Research: Meta-Analysis in Context (Wiley, 2022).

  64. Siafis, S. et al. Trace amine-associated receptor 1 (TAAR1) agonism for psychosis: a living systematic review and meta-analysis of human and non-human data. Wellcome Open. Res. 9, 182 (2024).

    Article  Google Scholar 

  65. Lok, I. M. et al. Effects of postnatal corticosteroids on lung development in newborn animals. a systematic review. Pediatr. Res. https://doi.org/10.1038/s41390-024-03114-6 (2024).

  66. Hooijmans, C. R. et al. SYRCLE’s risk of bias tool for animal studies. BMC Med. Res. Methodol. 14, 43 (2014). This publication reports on a frequently used risk of bias assessment tool for animal preclinical systematic reviews.

    Article  Google Scholar 

  67. Munn, Z., Moola, S., Lisy, K., Riitano, D. & Tufanaru, C. in JBI Reviewer's Manual Ch. 5 (eds Aromataris, E. & Munn, Z.) (JBI, 2017).

  68. Campbell, J. M. et al. in JBI Manual for Evidence Synthesis Ch. 9 (eds Aromataris, E. & Munn, Z.) (JBI, 2020).

  69. Hooijmans, C. R. et al. Facilitating healthcare decisions by assessing the certainty in the evidence from preclinical animal studies. PLoS ONE 13, e0187271 (2018).

    Article  Google Scholar 

  70. Zeiss, C. J. et al. Menagerie: a text-mining tool to support animal–human translation in neurodegeneration research. PLoS ONE 14, e0226176 (2019).

    Article  Google Scholar 

  71. Wang, Q., Liao, J., Lapata, M. & Macleod, M. Risk of bias assessment in preclinical literature using natural language processing. Res. Synth. Methods 13, 368–380 (2021).

    Article  Google Scholar 

  72. Menke, J., Roelandse, M., Ozyurt, B., Martone, M. & Bandrowski, A. The rigor and transparency index quality metric for assessing biological and medical science methods. Iscience 23, 101698 (2020).

    Article  ADS  Google Scholar 

  73. Chi, K.-Y., Li, M.-Y., Chen, C. & Kang, E. Ten circumstances and solutions for finding the sample mean and standard deviation for meta-analysis. Syst. Rev. 12, 1–5 (2023).

    Article  Google Scholar 

  74. Riley, R. D. et al. Multivariate and network meta-analysis of multiple outcomes and multiple treatments: rationale, concepts, and examples. BMJ 358, j3932 (2017).

    Article  Google Scholar 

  75. Mavridis, D. & Salanti, G. A practical introduction to multivariate meta-analysis. Stat. Methods Med. Res. 22, 133–158 (2013).

    Article  MathSciNet  Google Scholar 

  76. Balduzzi, S., Rücker, G. & Schwarzer, G. How to perform a meta-analysis with R: a practical tutorial. BMJ Ment. Health 22, 153–160 (2019). This work discusses the R package meta, a powerful, fully open source and, thus, a recommended R package to perform meta-analyses and visualize results including forest plots.

    Google Scholar 

  77. Schwarzer, G. meta: an R package for meta-analysis. R. News 7, 40–45 (2007).

    Google Scholar 

  78. Viechtbauer, W. Conducting meta-analyses in R with the metafor package. J. Stat. Softw. 36, 1–48 (2010).

    Article  Google Scholar 

  79. Gasparrini, A. & Gasparrini, M. A. Package ‘mvmeta’. (2015).

  80. Voelkl, B., Vogt, L., Sena, E. S. & Würbel, H. Reproducibility of preclinical animal research improves with heterogeneity of study samples. PLoS Biol. 16, e2003693 (2018).

    Article  Google Scholar 

  81. Rosso, M. et al. Reliability of common mouse behavioural tests of anxiety: a systematic review and meta-analysis on the effects of anxiolytics. Neurosci. Biobehav. Rev. 143, 104928 (2022).

    Article  Google Scholar 

  82. Nakagawa, S. & Santos, E. S. Methodological issues and advances in biological meta-analysis. Evolut. Ecol. 26, 1253–1274 (2012).

    Article  Google Scholar 

  83. Rücker, G., Schwarzer, G., Carpenter, J. R. & Schumacher, M. Undue reliance on I2 in assessing heterogeneity may mislead. BMC Med. Res. Methodol. 8, 79 (2008).

    Article  Google Scholar 

  84. Hardy, R. J. & Thompson, S. G. Detecting and describing heterogeneity in meta‐analysis. Stat. Med. 17, 841–856 (1998).

    Article  Google Scholar 

  85. Riley, R. D., Higgins, J. P. & Deeks, J. J. Interpretation of random effects meta-analyses. BMJ 342, d549 (2011). This work presents a non-technical explanation of the random-effects meta-analysis model and the usefulness of prediction intervals that show the range of potential intervention effects after accounting for heterogeneity.

    Article  Google Scholar 

  86. Jackson, D., White, I. R. & Riley, R. D. Quantifying the impact of between‐study heterogeneity in multivariate meta‐analyses. Stat. Med. 31, 3805–3820 (2012).

    Article  MathSciNet  Google Scholar 

  87. Sedgwick, P. How to read a forest plot in a meta-analysis. BMJ 351, h4028 (2015). This work presents a simple explanation of how to read and interpret a forest plot.

    Article  Google Scholar 

  88. Sterne, J. A., Gavaghan, D. & Egger, M. Publication and related bias in meta-analysis: power of statistical tests and prevalence in the literature. J. Clin. Epidemiol. 53, 1119–1129 (2000).

    Article  Google Scholar 

  89. Tsilidis, K. K. et al. Evaluation of excess significance bias in animal studies of neurological diseases. PLoS Biol. 11, e1001609 (2013).

    Article  Google Scholar 

  90. Sterne, J. A. et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ 343, d4002 (2011). This non-technical paper discusses common misconceptions, pitfalls and the usefulness of funnel plots, and their role in detecting small-study effects and evaluating the risk of publication bias.

    Article  Google Scholar 

  91. Nakagawa, S. et al. Methods for testing publication bias in ecological and evolutionary meta‐analyses. Methods Ecol. Evol. 13, 4–21 (2022).

    Article  Google Scholar 

  92. Hong, C. et al. Testing small study effects in multivariate meta-analysis. Biometrics 76, 1240–1250 (2020).

    Article  MathSciNet  Google Scholar 

  93. Peters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R. & Rushton, L. Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry. J. Clin. Epidemiol. 61, 991–996 (2008).

    Article  Google Scholar 

  94. Page, M. J. et al. ROB-ME: a tool for assessing risk of bias due to missing evidence in systematic reviews with meta-analysis. BMJ 383, e076754 (2023).

    Article  Google Scholar 

  95. Page, M. J. et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 372, n71 (2021). This work presents the standard guideline to use for the reporting of a systematic review.

    Article  Google Scholar 

  96. Page, M. J. et al. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ 372, n160 (2021).

    Article  Google Scholar 

  97. Rethlefsen, M. L. et al. PRISMA-S: an extension to the PRISMA statement for reporting literature searches in systematic reviews. Syst. Rev. 10, 39 (2021).

    Article  Google Scholar 

  98. Shea, B. J. et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ 358, j4008 (2017).

    Article  Google Scholar 

  99. Whiting, P. et al. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J. Clin. Epidemiol. 69, 225–234 (2016).

    Article  Google Scholar 

  100. Samjoo, I. A. et al. Comparative efficacy of therapies for relapsing multiple sclerosis: a systematic review and network meta-analysis. J. Comp. Eff. Res. 12, e230016 (2023).

    Article  Google Scholar 

  101. Wu, X. et al. The efficacy and safety of anti-CD20 antibody treatments in relapsing multiple sclerosis: a systematic review and network meta-analysis. CNS Drugs 36, 1155–1170 (2022).

    Article  Google Scholar 

  102. Sena, E., Wheble, P., Sandercock, P. & Macleod, M. Systematic review and meta-analysis of the efficacy of tirilazad in experimental stroke. Stroke 38, 388–394 (2007).

    Article  Google Scholar 

  103. Van der Worp, H. et al. The effect of tirilazad mesylate on infarct volume of patients with acute ischemic stroke. Neurology 58, 133–135 (2002).

    Article  Google Scholar 

  104. Cannon, A. E. et al. Neuroimaging findings in preclinical amyotrophic lateral sclerosis models—how well do they mimic the clinical phenotype? A systematic review. Front. Vet. Sci. 10, 1135282 (2023).

    Article  Google Scholar 

  105. Macleod, M. R. et al. Evidence for the efficacy of NXY-059 in experimental focal cerebral ischaemia is confounded by study quality. Stroke 39, 2824–2829 (2008).

    Article  Google Scholar 

  106. Bath, P. et al. Effects of NXY‐059 in experimental stroke: an individual animal meta‐analysis. Br. J. Pharmacol. 157, 1157–1171 (2009).

    Article  Google Scholar 

  107. Zhao, Z., Cheng, M., Maples, K. R., Ma, J. Y. & Buchan, A. M. NXY-059, a novel free radical trapping compound, reduces cortical infarction after permanent focal cerebral ischemia in the rat. Brain Res. 909, 46–50 (2001).

    Article  Google Scholar 

  108. van der Worp, H. B. et al. EuroHYP-1: European multicenter, randomized, phase III clinical trial of therapeutic hypothermia plus best medical treatment vs. best medical treatment alone for acute ischemic stroke. Int. J. Stroke 9, 642–645 (2014).

    Article  Google Scholar 

  109. Holman, L., Head, M. L., Lanfear, R. & Jennions, M. D. Evidence of experimental bias in the life sciences: why we need blind data recording. PLoS Biol. 13, e1002190 (2015).

    Article  Google Scholar 

  110. Hirst, J. A. et al. The need for randomization in animal trials: an overview of systematic reviews. PLoS ONE 9, e98856 (2014).

    Article  ADS  Google Scholar 

  111. Soliman, N. et al. Systematic review and meta-analysis of cannabinoids, cannabis-based medicines, and endocannabinoid system modulators tested for antinociceptive effects in animal models of injury-related or pathological persistent pain. Pain 162, S26–S44 (2021).

    Article  Google Scholar 

  112. Landis, S. C. et al. A call for transparent reporting to optimize the predictive value of preclinical research. Nature 490, 187–191 (2012).

    Article  ADS  Google Scholar 

  113. Percie du Sert, N. et al. The ARRIVE guidelines 2.0: updated guidelines for reporting animal research. J. Cereb. Blood Flow. Metab. 40, 1769–1777 (2020).

    Article  Google Scholar 

  114. Macleod, M. et al. The MDAR (Materials Design Analysis Reporting) framework for transparent reporting in life sciences. Proc. Natl Acad. Sci. USA 118, e2103238118 (2021).

    Article  Google Scholar 

  115. Korevaar, D., Hooft, L. & Ter Riet, G. Systematic reviews and meta-analyses of preclinical studies: publication bias in laboratory animal experiments. Lab. Anim. 45, 225–230 (2011).

    Article  Google Scholar 

  116. Sena, E. S., Van Der Worp, H. B., Bath, P. M., Howells, D. W. & Macleod, M. R. Publication bias in reports of animal stroke studies leads to major overstatement of efficacy. PLoS Biol. 8, e1000344 (2010).

    Article  Google Scholar 

  117. van der Naald, M. et al. Preregistration of animal research protocols: development and 3-year overview of preclinicaltrials.eu. BMJ Open Sci. 6, e100259 (2022).

    Google Scholar 

  118. Currie, G. L. et al. Animal models of chemotherapy-induced peripheral neuropathy: a machine-assisted systematic review and meta-analysis. PLoS Biol. 17, e3000243 (2019).

    Article  Google Scholar 

  119. Suen, C. M. et al. Efficacy and safety of regenerative cell therapy for pulmonary arterial hypertension in animal models: a preclinical systematic review protocol. Syst. Rev. 5, 89 (2016).

    Article  MathSciNet  Google Scholar 

  120. Health, E. P. O. A. et al. Welfare of dairy cows. EFSA J. 21, e07993 (2023).

    Google Scholar 

  121. Health, E. P. O. A. et al. Scientific opinion concerning the killing of rabbits for purposes other than slaughter. EFSA J. 18, e05943 (2020).

    Google Scholar 

  122. Chain, E. P. O. C. I. T. F. et al. Risks for public health related to the presence of tetrodotoxin (TTX) and TTX analogues in marine bivalves and gastropods. EFSA J. 15, e04752 (2017).

    Google Scholar 

  123. Health, E. Po. A. et al. SARS‐CoV‐2 in animals: susceptibility of animal species, risk for animal and public health, monitoring, prevention and control. EFSA J. 21, e07822 (2023).

    Google Scholar 

  124. Hooijmans, C. R. et al. Remyelination promoting therapies in multiple sclerosis animal models: a systematic review and meta-analysis. Sci. Rep. 9, 822 (2019).

    Article  ADS  Google Scholar 

  125. Wilkinson, M. D. et al. The FAIR guiding principles for scientific data management and stewardship. Sci. Data 3, 160018 (2016).

    Article  Google Scholar 

  126. Clark, J. et al. A full systematic review was completed in 2 weeks using automation tools: a case study. J. Clin. Epidemiol. 121, 81–90 (2020).

    Article  Google Scholar 

  127. Scott, A. M., Glasziou, P. & Clark, J. We extended the 2-week systematic review (2weekSR) methodology to larger, more complex systematic reviews: a case series. J. Clin. Epidemiol. 157, 112–119 (2023).

    Article  Google Scholar 

  128. Bugajska, J. et al. How long does it take to complete and publish an animal systematic review? Preprint at MetaArXiv https://doi.org/10.31222/osf.io/6345f (2024).

  129. Borah, R., Brown, A. W., Capers, P. L. & Kaiser, K. A. Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open. 7, e012545 (2017).

    Article  Google Scholar 

  130. Leenaars, C. et al. Reviewing the animal literature: how to describe and choose between different types of literature reviews. Lab. Anim. 55, 129–141 (2021).

    Article  Google Scholar 

  131. Wang, Z., Nayfeh, T., Tetzlaff, J., O’Blenis, P. & Murad, M. H. Error rates of human reviewers during abstract screening in systematic reviews. PLoS ONE 15, e0227742 (2020).

    Article  Google Scholar 

  132. Chan, M. E. & Arvey, R. D. Meta-analysis and the development of knowledge. Perspect. Psychol. Sci. 7, 79–92 (2012).

    Article  Google Scholar 

  133. van der Naald, M., Wenker, S., Doevendans, P. A., Wever, K. E. & Chamuleau, S. A. Publication rate in preclinical research: a plea for preregistration. BMJ Open Sci. 4, e100051 (2020).

    Google Scholar 

  134. Bespalov, A., Steckler, T. & Skolnick, P. Be positive about negatives—recommendations for the publication of negative (or null) results. Eur. Neuropsychopharmacol. 29, 1312–1320 (2019).

    Article  Google Scholar 

  135. Stern, C. & Kleijnen, J. Language bias in systematic reviews: you only get out what you put in. JBI Evid. Synth. 18, 1818–1819 (2020).

    Article  Google Scholar 

  136. Morrison, A. et al. The effect of English-language restriction on systematic review-based meta-analyses: a systematic review of empirical studies. Int. J. Technol. Assess. Health Care 28, 138–144 (2012).

    Article  Google Scholar 

  137. Cooper, C., Bou, J. T. & Varley-Campbell, J. Evaluating the effectiveness, efficiency, cost and value of contacting study authors in a systematic review: a case study and worked example. BMC Med. Res. Methodol. 19, 45 (2019).

    Article  Google Scholar 

  138. Candal-Pedreira, C. et al. Retracted papers originating from paper mills: cross sectional study. BMJ 379, e071517 (2022).

  139. Seifert, R. How Naunyn–Schmiedeberg’s archives of pharmacology deals with fraudulent papers from paper mills. Naunyn Schmiedebergs Arch. Pharmacol. 394, 431–436 (2021).

    Article  Google Scholar 

  140. Shojania, K. G. et al. How quickly do systematic reviews go out of date? A survival analysis. Ann. Intern. Med. 147, 224–233 (2007).

    Article  Google Scholar 

  141. Ineichen, B. V., Rosso, M. & Macleod, M. R. From data deluge to publomics: how AI can transform animal research. Lab. Anim. 52, 213–214 (2023).

    Article  Google Scholar 

  142. Zurrer, W. E., Cannon, A. E., Ewing, E., Rosso, M. & Ineichen, B. V. Auto-STEED: a data mining tool for automated extraction of experimental parameters and risk of bias items from in vivo publications. Preprint at bioRxiv https://doi.org/10.1101/2023.02.24.529867 (2023).

  143. Hair, K. et al. Systematic online living evidence summaries: emerging tools to accelerate evidence synthesis. Clin. Sci. 137, 773–784 (2023).

    Article  Google Scholar 

  144. Lee, P., Goldberg, C. & Kohane, I. The AI Revolution in Medicine: GPT-4 and Beyond (Pearson, 2023).

  145. Almazrouei, E. et al. The falcon series of open language models. Preprint at https://doi.org/10.48550/arXiv.2311.16867 (2023).

  146. Tran, V.-T. et al. Sensitivity and specificity of using GPT-3.5 turbo models for title and abstract screening in systematic reviews and meta-analyses. Ann. Intern. Med. 177, 791–799 (2024).

    Article  Google Scholar 

  147. Khraisha, Q., Put, S., Kappenberg, J., Warraitch, A. & Hadfield, K. Can large language models replace humans in systematic reviews? Evaluating GPT-4’s efficacy in screening and extracting data from peer-reviewed and grey literature in multiple languages. Res. Synth. Methods 15, 616–626 (2024).

    Article  Google Scholar 

  148. Ye, H., Liu, T., Zhang, A., Hua, W. & Jia, W. Cognitive mirage: a review of hallucinations in large language models. Preprint at https://doi.org/10.48550/arXiv.2309.06794 (2023).

  149. Tang, L. et al. Evaluating large language models on medical evidence summarization. NPJ Digital Med. 6, 158 (2023).

    Article  Google Scholar 

  150. Singhal, K. et al. Large language models encode clinical knowledge. Nature 620, 172–180 (2023).

    Article  ADS  Google Scholar 

  151. Gemini Team Google et al. Gemini: a family of highly capable multimodal models. Preprint at https://doi.org/10.48550/arXiv.2312.11805 (2023).

  152. Turner, R. M., Jackson, D., Wei, Y., Thompson, S. G. & Higgins, J. P. Predictive distributions for between‐study heterogeneity and simple methods for their application in Bayesian meta‐analysis. Stat. Med. 34, 984–998 (2015).

    Article  MathSciNet  Google Scholar 

  153. Puljak, L. & Sapunar, D. Acceptance of a systematic review as a thesis: survey of biomedical doctoral programs in Europe. Syst. Rev. 6, 253 (2017).

    Article  Google Scholar 

  154. Hesen, N. A. et al. A systematic review and meta-analysis of the protective effects of metformin in experimental myocardial infarction. PLoS ONE 12, e0183664 (2017).

    Article  Google Scholar 

Download references

Acknowledgements

The authors thank A. Berger from the University Library Zurich for help with the literature database table. They thank R. Aquarius from the Radboud University Medical Centre (Radboudumc) at Nijmegen for help with the section on counterfeit data. B.V.I. discloses support for the research of this work from the Swiss National Science Foundation (No. 407940_206504) and the Universities Federation for Animal Welfare (UFAW).

Author information

Authors and Affiliations

  1. Center for Reproducible Science, University of Zurich, Zurich, Switzerland

    Benjamin Victor Ineichen

  2. Epidemiology, Biostatistics and Prevention Institute, University of Zurich, Zurich, Switzerland

    Benjamin Victor Ineichen

  3. Clinical Neuroscience Center, University Hospital Zurich, University of Zurich, Zurich, Switzerland

    Benjamin Victor Ineichen

  4. Department of Biostatistics at Epidemiology, Biostatistics and Prevention Institute, University of Zurich, Zurich, Switzerland

    Ulrike Held

  5. Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland

    Georgia Salanti

  6. Centre for Clinical Brain Sciences, Edinburgh Medical School, The University of Edinburgh, Edinburgh, UK

    Malcolm Robert Macleod

  7. Department of Anaesthesiology, Pain and Palliative Medicine, Radboud University Medical Center, Nijmegen, the Netherlands

    Kimberley Elaine Wever

Authors
  1. Benjamin Victor Ineichen
    View author publications

    Search author on:PubMed Google Scholar

  2. Ulrike Held
    View author publications

    Search author on:PubMed Google Scholar

  3. Georgia Salanti
    View author publications

    Search author on:PubMed Google Scholar

  4. Malcolm Robert Macleod
    View author publications

    Search author on:PubMed Google Scholar

  5. Kimberley Elaine Wever
    View author publications

    Search author on:PubMed Google Scholar

Contributions

Introduction (B.V.I., M.R.M. and K.E.W.); Experimentation (B.V.I., M.R.M. and K.E.W.); Results (B.V.I., U.H., G.S. and K.E.W.); Applications (B.V.I., M.R.M. and K.E.W.); Reproducibility and data deposition (B.V.I. and K.E.W.); Limitations and optimizations (B.V.I. and K.E.W.); Outlook (B.V.I. and K.E.W.); Overview of the Primer (B.V.I.). All authors critically revised the manuscript.

Corresponding author

Correspondence to Benjamin Victor Ineichen.

Ethics declarations

Competing interests

The authors have nothing to declare.

Peer review

Peer review information

Nature Reviews Methods Primers thanks Thiago França, Manoj Lalu, Julia Menon, Kieron Rooney and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Related links

CAMARADES Berlin: https://camarades.de

CAMARADES Brasil: https://camaradesbrasil.bio.br

CAMARADES Netherlands: https://www.radboudumc.nl/en/research/departments/anesthesiology/meta-research-team/camarades-netherlands

CAMARADES UK: https://www.ed.ac.uk/clinical-brain-sciences/research/camarades

CAMARADES Zurich: https://camarades.ch

Cochrane organization: https://cochrane.org

GRADE: https://www.gradeworkinggroup.org/

Joanna Briggs Institute: https://jbi.global/

Norecopa: https://norecopa.no/search?q=*&sf=all&fq=type%3ADatabases&sort=name+asc&p=2&t=strict

Preclinicaltrials.eu: https://www.preclinicaltrials.eu

PRISMA: https://www.prisma-statement.org/

PRISMA extension: https://osf.io/g4kqz/

PROSPERO: https://www.crd.york.ac.uk/prospero

PubPeer: https://pubpeer.com

Retraction Watch: https://retractionwatch.com/

Risk of Bias Tool: https://ntp.niehs.nih.gov/go/riskbias

searchRxiv: https://www.cabidigitallibrary.org/journal/searchrxiv

SYRCLE: https://syrcle.network

SYREAF: https://syreaf.org/

VetSRev: https://vetsrev.nottingham.ac.uk/

Glossary

Bias domains

Specific aspects or categories of potential bias that can affect the validity of research findings.

Citation tracking

Collecting references that have citation relationships to known relevant references.

Common effect model

A statistical model to quantitatively synthesize study findings that assumes each study estimates the exact same underlying effect, attributing any differences in study results solely to sampling error.

Free-text terms

Words and phrases used in search queries that reflect the natural language of the topic being researched.

Grey literature

Research and information produced outside traditional publishing and distribution channels, such as preprint servers, books, patent registries, conference proceedings, guidelines and theses.

Meta-analysis

A statistical technique combining the results of several studies to compute a single estimate of effect, providing a quantitative summary of the evidence.

Narrative synthesis

A method of summarizing and interpreting findings from multiple studies using text to identify patterns, themes or relationships.

Odds ratio

The ratio of the odds of an event in the treatment and control groups.

Preclinical research

Basic and translational studies conducted before human testing, involving animals and in vitro systems.

Prediction interval

The interval including the underlying effect of a new experiment with similar design, setting and methods to observed experiments.

Publication bias

The tendency for journals and researchers to publish statistically significant or favourable findings over non-significant or negative outcomes, leading to a skewed representation of research evidence in the literature.

Random effects model

A statistical model to quantitatively synthesize study findings that assumes studies estimate different underlying effects that vary from one study setting to another.

Risk difference

The absolute difference in the probability of an event occurring between two groups, indicating the change in risk in the exposed group compared with the control group.

Risk ratio

The ratio of the two event rates in the treatment and control groups.

Selective outcome reporting bias

The selective disclosure of only some results based on their nature or direction, often emphasizing favourable findings over less favourable or insignificant ones, leading to a distorted view of the actual findings.

Standardized mean difference

The mean difference divided by the common standard deviation of control and experimental groups.

Thesaurus terms

Standardized words or phrases used to unify terminology and improve the accuracy of searches across different studies and sources.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ineichen, B.V., Held, U., Salanti, G. et al. Systematic review and meta-analysis of preclinical studies. Nat Rev Methods Primers 4, 72 (2024). https://doi.org/10.1038/s43586-024-00347-x

Download citation

  • Accepted:

  • Published:

  • Version of record:

  • DOI: https://doi.org/10.1038/s43586-024-00347-x

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing