Immersive Video Processing and Depth Estimation Techniques
Summary
Immersive video processing combines advanced capture, encoding and rendering methods to deliver three-dimensional visual experiences that allow users to move freely within a virtual scene. At its core lies volumetric video, in which dynamic three-dimensional representations of real objects are acquired by multi-camera rigs and reconstructed as dense point clouds, meshes or Gaussian blobs. Depth estimation techniques infer per-pixel or per-point distance information from multiple viewpoints, enabling six degrees of freedom (6DoF) navigation and realistic relighting. The processing pipeline typically encompasses camera calibration, multi-view synchronisation, depth map generation and refinement, efficient compression of geometry and texture, and interactive streaming to end devices. Recent advances leverage machine learning to improve depth accuracy and temporal consistency, while novel coding schemes such as point cloud compression and dynamic Gaussian representations reduce bandwidth demands. Applications span virtual reality telepresence, cultural heritage preservation, medical simulation and immersive entertainment, offering globally scalable experiences that bridge physical and digital realms.
Research from Nature Portfolio
No recent Nature Portfolio content available.
Immersive Video Processing and Depth Estimation Techniques publication trend
The graph below shows the total number of articles in immersive video processing and depth estimation techniques across all publications each year (not limited to Nature Index journals).
Technical terms
Immersive video: A form of video that allows users to move or look around freely in six degrees of freedom within a three-dimensional scene.
Volumetric video: A dynamic three-dimensional representation of real objects captured from multiple viewpoints and reconstructed as point clouds, meshes or Gaussian primitives.
Depth map: A two-dimensional image where each pixel value encodes the distance from the camera to the scene surface at that point.
6DoF: Six degrees of freedom describing three translational (x, y, z) and three rotational (pitch, yaw, roll) axes of movement.
MPEG Immersive Video (MIV) standard: An ISO/IEC specification for coding and streaming immersive video content using patch-based atlases and legacy video codecs.
Decoder-side depth estimation: Techniques that reconstruct depth maps at the client or cloud without transmitting them in the bitstream, relying on metadata and decoded views.
References
- V^3: Viewing Volumetric Videos on Mobiles via Streamable 2D Dynamic Gaussians. ACM Transactions on Graphics (2024).
- MPEG Immersive Video Coding Standard. Proceedings of the IEEE (2021).
- On Alpha-Expansion-Based Graph-Cut Optimization for Decoder-Side Depth Estimation. Applied Sciences (2024).
About these summaries
This Nature Research Intelligence Topic summary is created with the cited references and a large language model. We take care to ground generated text with facts, and have systems in place to gain human feedback on the overall quality of the process in line with our AI principles. We strive to create accurate and useful summaries for people unfamiliar with the research topic and that supports this goal. These pages are a beta release and will be updated as we learn how best to help people gain value from a research topic summary.
Turn complex research questions into confident strategic decisions
When you're under pressure to set direction, justify investment, or understand your competitive position, you need more than raw data — you need trusted insights you can act on.
Benchmark your performance against global peers using robust, methodologically sound analysis.
Combine quantitative metrics with qualitative expert insight to uncover strengths, gaps and emerging opportunities.
Gain tailored, decision-ready recommendations aligned to your strategic priorities.
Talk to us to learn more about our data dashboards and bespoke strategy reports.
Grow research skills, confidence and careers with training built for every stage of the research lifecycle.
Developed with Nature Portfolio journal Editors and internationally renowned experts. Discover three ways to learn:
Self-paced, online courses in convenient bite-sized units, covering key skills across scientific writing, publishing, grant writing, data analysis, and more.
Expert trainer-led workshops with hands-on exercises and real-time feedback across core research skills, delivered via interactive group sessions.
Editor-led workshops combining core principles in writing and publishing, personalised 1:1 feedback from Nature Portfolio Editors and hands-on exercises.
Explore course catalogues and workshop agendas, enquire about the options or request institutional pricing.