Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Correspondence
  • Published:

Towards reproducible robotics research

Nature Machine Intelligence (2025)Cite this article

Subjects

Access through your institution
Buy or subscribe

Given the complexity of robotic systems, reproducibility might be seen as less feasible in robotics than in other fields of research. However, reproducibility in robotics is necessary and possible. The community started to consider reproducibility issues in the 2000s. Some of the authors of this article were involved in a special interest group, started some 15 years ago, on Good Experimental Methodology (GEM) and Benchmarking. The group was created by a European robotics network called EURON, and the project was led by F.B., A.P.d.P. and now emeritus professor John Hallam, then at the University of Southern Denmark. The EURON GEM’s guidelines list basic requisites for reproducibility in robotics1.

It is worth noting that the large majority (>90%) of published robotics research papers could be called ‘experimental’ because they report real world trials or simulations. In our view, an ‘experimental’ paper should include at least two elements to be considered a scientific contribution. First, the results need to be reproducible in a statistically meaningful way. Though results can vary between different experiments, they have to be statistically equivalent. Second, there must be an objective comparison with competing approaches, meaning that a set of well-defined steps to perform the comparison procedure needs to be outlined. In robotics, papers often report results as ‘experimental evidence’, but given the complexities and challenges of modelling robotics and AI systems, a careful description of the methodology, comparisons and evaluations becomes very important, and often details are missing.

This is a preview of subscription content, access via your institution

Access options

Access through your institution

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Reproducible Article life cycle.

References

  1. Bonsignorio, F., Hallam, J. & del Pobil, A. Zenodo https://doi.org/10.5281/zenodo.16452480 (2008).

  2. Amigoni, F., Reggiani, M. & Schiaffonati, V. Auton. Robots 27, 313–325 (2009).

    Article  Google Scholar 

  3. Faragasso, A. & Bonsignorio, F. Front. Robotics AI 10, 1127972 (2023).

    Article  Google Scholar 

  4. Redfield, S. Nat. Mach. Intell. 1, 263–264 (2019).

    Article  Google Scholar 

  5. Pineau, J. et al. J. Mach. Learn. Res. 22, 7459–7478 (2021).

  6. Bonsignorio, F. IEEE Robotics Autom. Mag. 24, 178–129 (2017).

    Article  Google Scholar 

Download references

Author information

Author notes

  1. These authors contributed equally: Angel P. del Pobil, Enrica Zereik.

Authors and Affiliations

  1. University of Zagreb Faculty of Electrical Engineering and Computing, AIFORS Lab, Zagreb, Croatia

    Fabio Bonsignorio

  2. Heron Robots, Genoa, Italy

    Fabio Bonsignorio

  3. Universitat Jaume I, Department of Engineering and Computer Science, Castellon, Spain

    Angel P. del Pobil

  4. Institute of Marine Engineering, Italian National Research Council, Genoa, Italy

    Enrica Zereik

Authors
  1. Fabio Bonsignorio
    View author publications

    Search author on:PubMed Google Scholar

  2. Angel P. del Pobil
    View author publications

    Search author on:PubMed Google Scholar

  3. Enrica Zereik
    View author publications

    Search author on:PubMed Google Scholar

Corresponding author

Correspondence to Fabio Bonsignorio.

Ethics declarations

Competing interests

F.B. is the chief executive officer and founder of Heron Robots.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bonsignorio, F., del Pobil, A.P. & Zereik, E. Towards reproducible robotics research. Nat Mach Intell (2025). https://doi.org/10.1038/s42256-025-01114-7

Download citation

  • Published:

  • DOI: https://doi.org/10.1038/s42256-025-01114-7

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics