Table 2 A summary table outlining the proposed solutions for enabling technologies in digital twin creation, along with the purpose of each solution.

3D reconstruction

Blender

Creation of 3D model.

Carla simulator and Unity3D

Simulation of 3D models and

environments, generate what

-if scenarios

Data enhancement

3D geometry encoded into 2D images,

enabling efficient processing using

Swin Transformers

Encoding 3D data into 2D images

enables advanced 2D processing

techniques to operate on the

encoded representation

Data compression

Image-based volumetric video

compression pipeline

Reduction of the amount of

information required to represent

volumetric content

Scene understanding

Mask2Former

2D Semantic segmentation

A combination of Mask R-CNN

and ODISE

2D panoptic segmentation for

semantic enhancement.

Sensor Fusion

H-SLAM, spatial mapping and FGICP

Fusion of multiple depth maps

from different sensors

Fusion of LiDAR-based and visual

-based H-SLAM

Localization improvement

Integration of 2D camera-based

segmentation results into the LiDAR-

based 3D reconstructed point cloud

Enriching the 3D map with

semantic information

Localization

Depth map estimation model trained on

huge heterogeneous image

datasets³⁰

Depth map estimation from

RGB images

IBL and NeRF-based localization method

Incorporation of NeRF as a

mapping model in the update step

Rendering

Nerfacto model within the Nerfstudio

framework

Construction of a 3D outdoor

scene representation using NeRF

COLMAP or integration of the capture

car’s GPS information along with

camera calibration parameters

Image placement within the

3D environment

DietNeRF or DS-NeRF

Reduction of the number of

viewpoints needed

MVSNeRF or ENeRF

Generalizable NeRF algorithms

to avoid overfitting

XR rendering

UNITY

Real-time 3D rendering, enabling

immersive virtual, augmented, and

mixed reality experiences