Omnidirectional Depth Estimation in Spherical Imaging

Depth estimation from spherical imagery seeks to reconstruct three-dimensional structure across a full 360° field of view. Conventional perspective techniques must be adapted to account for the radial distortion inherent to fisheye and panoramic sensors, and novel projection models have emerged to map spherical rays into depth predictions. Geometry-based pipelines apply spherical sweeping across concentric shells to generate cost volumes for stereo or multi-view depth inference, while modern deep networks exploit equirectangular or content-aware projections to learn global and local cues simultaneously. Hybrid approaches combine geometric rectification and learned descriptors to improve feature matching for structure from motion, enabling robust camera orientation and dense reconstruction in urban, indoor and autonomous driving environments. Recent advances have reduced computational overhead by cascading coarse-to-fine estimation stages and embracing transformer-based context aggregation, opening routes to real-time performance on embedded platforms. Applications span robotics navigation, immersive virtual reality, environmental monitoring and autonomous vehicles, underscoring the global relevance of omnidirectional depth sensing for situational awareness and scene modelling.

Research from Nature Portfolio

No recent Nature Portfolio content available.

Omnidirectional Depth Estimation in Spherical Imaging publication trend

The graph below shows the total number of articles in omnidirectional depth estimation in spherical imaging across all publications each year (not limited to Nature Index journals).

Technical terms

Omnidirectional image: an image capturing a full spherical field of view around a vantage point.

Equirectangular projection: a mapping that unfolds the sphere into a rectangular image by linearly sampling latitude and longitude.

Spherical sweeping: a depth-estimation technique that evaluates cost volumes on concentric spherical shells centered at the camera.

Cost volume: a data structure storing matching costs across depth hypotheses for pixel correspondences in stereo or multi-view setups.

Vision transformer: a neural network architecture using self-attention mechanisms to capture long-range dependencies in image patches.

Patch filling: a method to augment sparse depth data by interpolating missing regions based on neighbouring context.

Structure from motion (SfM): a process to recover camera poses and sparse 3D features by matching image keypoints across views.

References

1. 3D reconstruction of spherical images: a review of techniques, applications, and prospects. Geo-spatial Information Science (2024).
2. CAPDepth: 360 Monocular Depth Estimation by Content-Aware Projection. Applied Sciences (2025).
3. OmniGlasses: an optical aid for stereo vision CNNs to enable omnidirectional image processing. Machine Vision and Applications (2024).
4. Reliable Feature Matching for Spherical Images via Local Geometric Rectification and Learned Descriptor. Remote Sensing (2023).
5. A Novel Panorama Depth Estimation Framework for Autonomous Driving Scenarios Based on a Vision Transformer. Sensors (2024).
6. CasOmniMVS: Cascade Omnidirectional Depth Estimation with Dynamic Spherical Sweeping. Applied Sciences (2024).