Visual Simultaneous Localization and Mapping for Minimally Invasive Surgery

Visual simultaneous localization and mapping (VSLAM) has become a cornerstone technology for enhancing navigation, situational awareness and augmented reality in minimally invasive surgery. By processing video streams from endoscopic or capsule cameras, VSLAM algorithms estimate the camera’s trajectory while reconstructing a three-dimensional representation of the surgical scene. Early approaches relied on sparse, feature-based methods to detect points of interest and apply bundle adjustment, but the unique challenges of endoscopic imagery—such as low texture, specular reflections, dynamic fluids and soft-tissue deformation—have driven the development of hybrid and direct methods. Monocular systems have been augmented with deep learning models to predict depth maps and improve robustness, while stereo endoscopes and structured light setups offer richer geometry at the cost of more complex hardware. Non-rigid mapping techniques account for tissue motion, creating deformable models that maintain accuracy during organ movement. Together, these advances support real-time geometry-aware overlays, automated lesion localisation and precise instrument guidance, promising to reduce operative time, improve safety and enable new applications in surgical robotics and tele-mentoring.

Research from Nature Portfolio

No recent Nature Portfolio content available.

Visual Simultaneous Localization and Mapping for Minimally Invasive Surgery publication trend

The graph below shows the total number of articles in visual simultaneous localization and mapping for minimally invasive surgery across all publications each year (not limited to Nature Index journals).

Technical terms

Simultaneous Localization and Mapping (SLAM): A computational process by which a moving camera or robot estimates its own position while building a map of an unknown environment.

Visual Odometry: The technique of determining the motion of a camera by analysing consecutive image frames without constructing a full map.

Non-rigid Mapping: An extension of SLAM that models scene deformation, allowing the map to adapt to moving or deformable structures such as soft tissues.

Feature Extraction: The identification of distinct image points or regions (for example corners or edges) that can be reliably tracked between frames.

Calibration: The process of determining a camera’s intrinsic parameters (focal length, principal point, distortion) and extrinsic relationship to other sensors or coordinate frames.

References

1. Endomapper dataset of complete calibrated endoscopy procedures. Scientific Data (2023).
2. Implicit domain adaptation with conditional generative adversarial networks for depth prediction in endoscopy. International Journal of Computer Assisted Radiology and Surgery (2019).
3. Deep EndoVO: A recurrent convolutional neural network (RCNN) based visual odometry approach for endoscopic capsule robots. Neurocomputing (2018).
4. A non-rigid map fusion-based direct SLAM method for endoscopic capsule robots. International Journal of Intelligent Robotics and Applications (2017).
5. Reconstructing a 3D heart surface with stereo-endoscope by learning eigen-shapes.. Biomedical Optics Express (2018).
6. Real‐time geometry‐aware augmented reality in minimally invasive surgery. Healthcare Technology Letters (2017).