Cross-Domain multi-channel transformer for point cloud registration in cultural heritage digital preservation

Abstract

Point cloud registration plays a crucial role in preservation and digitization of cultural heritage by accurately aligning multiple point cloud datasets to create a complete 3D model. However, the complexity and diversity of objects lead to low-overlap and significant variations, posing challenges in achieving high accuracy, robustness and generalizability. This study proposes a Cross-Domain Multi-Channel Transformer (CDMCT) to address these challenges. The multi-channel dynamic encoding to enhance model’s sensitivity to local structures and angular relationships. The cross-domain convergence network preserves the global relationships within point cloud and the overall structural information. The integration of graph networks allows for flexible handling of local feature variations. We trained on 3DMatch and KITTI, and validated on cultural heritage datasets Terracotta Warriors and WHU-TLS ancient buildings. Experimental results show that CDMCT achieves significant improvements in registration accuracy and robustness, demonstrating its broad application potential in the digital preservation of cultural heritage.

Introduction

Cultural heritage encompasses historically and culturally significant assets such as ancient buildings and Terracotta Warriors. Their preservation, restoration, and digitization represent crucial tasks in modern technology. Traditional digitalization efforts of cultural heritage require precise 3D reconstruction and detailed structural analysis. Point cloud registration^1,2,3 technology addresses the task of aligning point cloud data obtained from different viewpoints or times to achieve a consistent coordinate system and geometric structure. This process enables the construction of unified 3D models and facilitates the integration and analysis of multi-source point cloud data. It contributes to achieving high-precision digitalization of cultural heritage, thereby supporting the long-term preservation, restoration, and exhibition of ancient buildings and Terracotta Warriors. Point cloud registration plays a pivotal role in various fields including 3D modeling^4,5, classification⁶, artifact restoration, autonomous driving⁷, robot navigation⁸, virtual reality, medical imaging, and cultural heritage preservation⁹. However, challenges such as low-overlap, large-scale scenes and across datasets generalization make point cloud registration a highly demanding research direction.

Over the past few decades, various point cloud registration methods have been proposed, including feature-based learning methods^10,11,12 and End-to-End Learning Methods^13,14. In feature-based learning point cloud registration methods, key features are extracted from point cloud data and matched and aligned using machine learning or deep learning algorithms to obtain high-precision 3D data. This process not only aids in the protection and restoration of ancient buildings and the Terracotta Warriors but also provides robust support for the preservation, restoration, and display of cultural heritage. In unsupervised point cloud registration, due to the absence of accurate label information, traditional inlier evaluation methods struggle to effectively differentiate inliers from outliers, leading to unstable and inaccurate registration results. To address this, Huang et al.¹⁵ proposed an innovative unsupervised learning probabilistic point cloud registration method. This method takes advantage of statistical models to overcome noise and density variation and uses alignment scores in a unified Gaussian mixture model to supervise network training while reducing reliance on supervisory information, achieving better registration accuracy and efficiency. To enhance the effectiveness of multi-view point cloud registration, Wang et al.¹⁶ proposed a multi-view point cloud registration algorithm. By utilizing reliable pose graph initialization and history re-weighting, this method resolves issues of misalignment and instability in multi-view point cloud registration, enhancing robustness and accuracy. Firstly, an initial pose graph is constructed through preprocessing and feature extraction. The pose graph represents relative pose relationships between point cloud from different viewpoints. Subsequently, global optimization is applied to the pose graph using optimization algorithms to further enhance its accuracy and consistency. Lastly, a historical re-weighting mechanism is introduced to re-weight pose graphs from previous registration steps. To address accidents and fatalities in scaffolding operations, Kim et al.¹⁷ proposed a novel method for scaffold point cloud registration considering safety nets. The method comprises three steps: firstly, LiDAR-camera calibration and SLAM-based point cloud data acquisition; secondly, object detection and coordinate transformation; and finally, identification of scaffold installation stages and point cloud registration. Experimental validation demonstrated that this method effectively achieves scaffold point cloud registration in scenarios without safety nets. Traditional Iterative Closest Point (ICP) methods perform poorly in low-overlap scenarios and have high algorithmic complexity. Feature-based methods are often sensitive to feature selection and extraction, leading to poor generalization. Additionally, some existing neural network methods often require a large amount of data labeling and model adjustment when dealing with different datasets, limiting their flexibility in practical applications.

End-to-end Learning methods directly learn and optimize the registration process through deep learning networks or optimization algorithms, thereby enhancing the overall performance and accuracy of the system. This approach typically handles the complexity within data more effectively and is suitable for various cultural heritage scenarios requiring point cloud registration. FCGF¹⁸ employs a fully convolutional network to learn geometry feature representations end-to-end, enhancing the representation capacity and automation of geometric features. Specifically, the network takes three-dimensional point cloud data as input and extracts features through a series of convolutional and pooling layers. Unlike traditional convolutional neural networks, this network performs convolution operations across the entire input point cloud rather than only on local regions. Through this approach, the network captures global geometric information and generates fully convolutional geometry feature maps. Existing point cloud descriptors rely on structural information and ignore texture information, yet texture information is crucial for distinguishing scene components. To solve this problem, Huang et al.¹⁹ proposed a method based on multi-modal fusion to generate point cloud registration descriptors that consider structural and texture information. This method achieves state-of-the-art accuracy and improves descriptor dissimilarity. To address the limitation of point cloud registration methods in terms of generalization ability, which often leads to performance degradation when applied to unseen datasets with distributions different from the training set, Yuan et al.²⁰ proposed a novel end-to-end registration method based on random networks. The method utilizes multiple randomly initialized networks for feature extraction and correspondence establishment. Additionally, a collaborative integration strategy is introduced to prune outliers from the correspondences established by random networks, making full use of spatial consistency. Through collaborative integration pruning, the majority of outliers can be removed, enabling robust registration within an economical RANSAC iteration.

In recent years, the Transformer architecture has achieved remarkable success in natural language processing and computer vision fields^21,22,23. Its powerful sequence modeling capabilities and self-attention mechanism demonstrate great potential in point cloud registration tasks. Yew et al.²⁴ using Transformers for point cloud correspondence. The core idea is to treat point cloud datasets as sets of points and use the Transformer model to learn point-to-point correspondences. Each point is represented as a feature vector, and attention mechanisms are employed to establish relationships between points. Through the use of a Transformer model, point cloud correspondences can be learned in an end-to-end manner without relying on traditional manual feature design or the two-step process of correspondence point extraction and matching. To address the impact of geometric structure descriptors in point cloud registration, Yuan et al.²⁵ proposed an Enhanced Geometric Structure Transformer method to improve the modeling of geometric structures within point clouds. By employing this enhanced transformer, they aimed to learn three explicit geometric structure features and utilize them to extract reliable correspondences for point cloud registration. In contrast to conventional approaches, this method eliminates the need for explicit positional embeddings or additional feature exchange modules, thus simplifying the network architecture. Yu et al.²⁶ introduced RIGA to address the limitations of existing neural descriptors in point cloud registration. Existing descriptors either suffer from performance degradation under large rotations or lack sufficiently unique local geometric information. To tackle this, they designed descriptors with rotation invariance and global awareness. By extracting point pair features from sparse local regions, RIGA encodes rotation-invariant local geometry into geometric descriptors and globally incorporates the 3D structure and geometric context of the entire framework in a rotation-invariant manner. Compared with traditional methods that require manual feature design, Transformer can capture global dependencies between point cloud data through self-attention, enhancing the automation and accuracy of registration. However, directly applying Transformer to point cloud registration still faces several challenges, including low-overlap between point cloud data, limited generalization ability across datasets, and insufficient robustness. These issues restrict its effectiveness in practical applications.

To address these issues, this paper proposes an end-to-end point cloud registration method based on Cross-domain Multi-channel Transformer to tackle challenges such as low-overlap, large-scenes, and across datasets generalization. We focus primarily on four representative point cloud datasets: 3DMatch²⁷, KITTI²⁸, Terracotta Warriors and WHU-TLS²⁹. In terms of method innovation, this paper introduces multi-channel dynamic encoding and cross-domain convergence networks, which effectively handle registration problems between low-overlap, large-scenes, and different datasets.

The contributions of this paper are mainly reflected in the following aspects: Multi-channel Dynamic Encoding: By introducing multi-channel angular relationship modeling into the task of cultural heritage point cloud registration. By repeatedly applying ternary angular positional encoding, it generates direction-sensitive multi-channel angle matrices, and incorporates dynamic positional weighting to adapt to the complex surfaces and local structural heterogeneity of heritage point clouds. By integrating angular encodings from different channels, the model’s ability to discriminate local geometric features is significantly enhanced.

Cross-domain Convergence Network: By combining the global modeling capability of Transformer with the local topological representation power of graph neural networks, the proposed method ensures both global structure preservation and robust expression of local feature variations in cultural heritage point clouds. When facing common challenges in heritage scenarios such as occlusion and incomplete scanning, it demonstrates superior robustness and enables high-precision registration.

Algorithm Generalization Performance: CDMCT is trained on general-purpose datasets and validated on representative cultural heritage datasets, including the Terracotta Warriors and WHU-TLS ancient buildings. Experimental results show that the proposed method exhibits strong cross-domain generalization capability and can adapt to registration tasks across different heritage types and under low-overlap conditions, aligning well with the complexity and diversity of real-world digital preservation scenarios.

Methods

Similar to the structures of D3Feat³⁰ and Predator³¹, our approach also employs a hierarchical framework, as illustrated in Fig. 1.

Fig. 1: The network architecture of CDMCT.

The figure illustrates the pipeline of the Cross-Domain Multi-Channel Transformer (CDMCT) model. Key modules include Encoder, Transformer and Decoder.

Problem setting

For two point cloud, the source point cloud $P=\{{p}_{i}\in {{\mathbb{R}}}^{3}| i=1,2,\ldots ,N\}$ and the target point cloud $Q=\{{q}_{i}\in {{\mathbb{R}}}^{3}| i=1,2,\ldots ,M\}$, where N and M represent the number of points in point cloud P and Q, respectively, the objective of point cloud registration is to align the two point cloud using an unknown 3D rigid transformation RT = {R, T}, comprising of a rotation matrix R ∈ SO(3) and a translation vector $T\in {{\mathbb{R}}}^{3}$. The transformation matrix can be defined as:

$$\mathop{\min }\limits_{R,T}\sum _{({p}_{i},{q}_{i})\in \vartheta }{\left\Vert R\cdot {p}_{i}+T-{q}_{i}\right\Vert }_{2}^{2}$$

(1)

Where, ϑ represents the ground truth correspondences between the points in P and Q. The notation $\left\Vert \bullet \right\Vert$ denotes the Euclidean norm.

In our method, the correspondences between points are output. Subsequently, the optimal rigid transformation is estimated from these correspondences using RANSAC³² to achieve the final point cloud registration.

Encoder-Decoder

Encoder: For the denser source point cloud $P\in {{\mathbb{R}}}^{N\times 3}$, $P=\{{p}_{i}\in {{\mathbb{R}}}^{3}| i=1,2,\ldots ,N\}$. and target $Q\in {{\mathbb{R}}}^{M\times 3}$, $Q=\{{q}_{i}\in {{\mathbb{R}}}^{3}| i=1,2,\ldots ,M\}$. we utilize the KPConv³³ module as the backbone, which consists of a series of residual modules and strided convolutions, to perform downsampling and reduce the number of keypoints to ${P}^{{\prime} }\in {{\mathbb{R}}}^{N^{\prime} \times 3}$ and ${Q}^{{\prime} }\in {{\mathbb{R}}}^{M^{\prime} \times 3}$, respectively (where $N > N^{\prime}$ and $M > M^{\prime}$).

Building upon this, we further design a shared encoder to simultaneously process the key points of the source and target point clouds. The architecture of this shared encoder consists of the following steps:

First, the input passes through the KPConv layer, which uses a kernel point convolution to convolve the local neighborhood of each key point, thereby extracting local geometric information. Next, residual connections are applied to alleviate the vanishing gradient problem in deep networks and enhance feature representation capability. This is followed by Batch Normalization and ReLU activation, which accelerate convergence and improve nonlinear modeling capacity. Finally, a shared parameter mechanism is employed: the source point cloud ${P}^{{\prime} }$ and the target point cloud ${Q}^{{\prime} }$ are simultaneously fed into the encoder, sharing all convolutional layers and weights to extract consistent local structural features across both domains.

Through the shared encoder, we obtain the feature representations of the source and target point clouds as ${F}_{{P}^{{\prime} }}^{{\prime} }\in {{\mathbb{R}}}^{N^{\prime} \times D}$ and ${F}_{{Q}^{{\prime} }}^{{\prime} }\in {{\mathbb{R}}}^{M^{\prime} \times D}$, where D represents the feature dimension. This structure ensures that the source and target domains have a consistent feature space during the encoding phase, which is helpful for the subsequent registration task.

Decoder: In this module, set up in the usual manner, there is a 3-layer network structure that primarily consists of upsampling, linear transformation, and skip connections.

Transformer

Multi-channel Dynamic Encoding (MCDE): In traditional Transformer models, positional encoding is typically applied in a fixed manner. However, the geometric structure and distribution characteristics of point cloud data may make it challenging for traditional positional encoding methods to capture changes in local features. MCDE addresses this issue by iteratively applying ternary angle positional encoding and incorporating dynamic positional encoding, allowing each point to more fully express different angular relationships and effectively capture the geometric information of the point cloud. This enhances the model’s ability to perceive the local structure and angular relationships of the point cloud. In this process, each channel corresponds to different angular relationships, forming multiple channels of angle encoding through weight-adjusted angle encoding to ensure that the model can fully understand and utilize the various angle information present in the point cloud. The network architecture of Transformer is illustrated in Fig. 2.

Fig. 2: The network architecture of Transformer.

MCDE: Multi-channel Dynamic Encoding. GCN: Graph Convolutional Networks. MC: Multi-head Cross-attention.

For each point P_i in the point cloud, the three angular relationships between it and the points in the neighbor set N_i are first computed and denoted as θ_ij, as follows:

$${\theta }_{ij}=arccos\left(\frac{({p}_{i}-{p}_{j})\cdot ({p}_{i}-{p}_{k})}{\parallel {p}_{i}-{p}_{j}\parallel \cdot \parallel {p}_{i}-{p}_{k}\parallel }\right)$$

(2)

Where, p_i, p_j, and p_k represent three different points, arccos represents the inverse cosine function, and ∥ ⋅ ∥ represents the modulus of the vector.

To adapt to the local structure of the point cloud, dynamic positional encoding is utilized to adjust the weights for optimizing the local perception of the point cloud. For each point P_i, the weights of its neighboring points are computed as α_i,j:

$${\alpha }_{i,j}=\Phi (Cat({p}_{i},PE({p}_{i}-{p}_{j})))$$

(3)

Where, Φ represents a multi-layer perceptron, Cat denotes the concatenation operation, and PE represents the relative position encoding.

By iteratively applying ternary angle positional encoding, multiple-channel angle encoding is generated. Each channel corresponds to different angular relationships, ensuring that the model can fully understand and utilize various angle information present in the point cloud. Multi-channel dynamic encoding is achieved through weight-adjusted angle encoding to capture richer geometric features. The specific formula for multi-channel dynamic encoding is:

$$MCD{E}_{c}(p)=\mathop{\sum }\limits_{i=1}^{K}\mathop{\sum }\limits_{j=i+1}^{K}PE({\alpha }_{ij}\cdot {\theta }_{{{p}_{i}}_{j}})$$

(4)

Where, MCDE_c represents the multi-channel dynamic encoding for the Cth channel, K is the number of neighboring points, and PE represents the relative position encoding.

Finally, the multi-channel dynamic encoding is fused with the original features of the point cloud to form the final feature representation, integrating the original features and angle information of the point cloud.

$$MCDE({p}_{i})=Cat\left({F}_{{P}^{{\prime} }}^{{\prime} },MCD{E}_{c}(p)\right)$$

(5)

Where, ${F}_{{P}^{{\prime} }}^{{\prime} }$ represents the original features of point ${P}^{{\prime} }$ in the point cloud.

Overall, ternary angle positional encoding computes the angle relationships between each point and its neighbors, encoding this angle information into each point’s feature representation. This angle information provides important insights into the local geometric structure of point clouds. Meanwhile, dynamic positional encoding adjusts the influence of each neighboring point on the current point through weighting, allowing the model to better adapt to different local features within point clouds. Their combination enables a more comprehensive utilization of geometric information and angle relationships in point cloud data, collectively enhancing the model’s ability to capture the geometric structure and angle relationships of point clouds.

Cross-domain Convergence Network (CC-Net): The CC-Net aims to address the issue of insufficient capture of local features in point cloud registration tasks by traditional Transformer models. Traditional Transformer models typically capture global features through cross-attention mechanisms. However, in point cloud registration tasks, local features are equally crucial. Therefore, CC-Net adopts the design concept of cross-domain convergence networks, integrating multi-head cross-attention mechanisms with graph convolutional networks at different levels.

CC-Net utilizes multi-head cross-attention mechanisms to preserve the global relationships of point cloud data, which helps the model capture overall structural information. Simultaneously, the introduction of graph convolutional networks better handles the variations in local features because graph convolutional networks excel at capturing local connectivity relationships between nodes in graph-structured data. Thus, by integrating multi-head cross-attention mechanisms and graph convolutional networks at different levels, CC-Net can more comprehensively utilize both global and local information of point cloud data, thereby improving the generalization and robustness of the registration model.

Firstly, we define the position relation MCDE embedding for the cross-attention module. We define a 4-layer multi-head cross-attention module, which inherits the feature information from the previous module while focusing on the mutual information between two point clouds.

$$MHA(Q,K,V)=(H{e}_{1}\oplus \ldots \oplus H{e}_{H})\cdot {W}^{o}$$

(6)

$$H{e}_{H}=Attn(Q\cdot {W}_{h}^{Q},K\cdot {W}_{h}^{K},V\cdot {W}_{h}^{V})$$

(7)

Where, Q, K, and V represent Query, Key, and Value matrices, respectively. The operation He₁ to He_H involves multiple attention heads (Head₁ to Head_H) which capture different aspects of the input.

Each attention head uses learnable transformation matrices ${W}_{h}^{Q}$, ${W}_{h}^{K}$, and ${W}_{h}^{V}$ to linearly transform the input Q, K, and V matrices. The transformed values are then used to compute attention scores.

The dimensions of Q, K, V, and each attention head (d_Q, d_K, d_V, d_Head) are related, where d_Q = d_K = d_V = d_Head = D/H. Here, D is the model’s dimensionality and H is the number of attention heads.

Given a graph G = (V, E), where V is the set of nodes and E is the set of edges, Graph Convolutional Network (GCN) can perform convolution operations on each node v, and its output representation h_v is given by:

$${h}_{v}=\sigma\left(\sum _{u\in N(v)}\frac{1}{{c}_{v}}{W}_{G}{h}_{u}\right)$$

(8)

Where, N(v) is the set of neighboring nodes of node v, c_v is a normalization factor, W_G is a learnable weight matrix, and σ is the sigmoid activation function.

Moving forward, let’s denote input matrices as ${F}_{{P}^{{\prime} }}^{{\prime} }=({x}_{1}^{{P}^{{\prime} }},{x}_{2}^{{P}^{{\prime} }}\cdots {x}_{N^{\prime} }^{{P}^{{\prime} }})$ and ${F}_{{Q}^{{\prime} }}^{{\prime} }=({x}_{1}^{{Q}^{{\prime} }},{x}_{2}^{{Q}^{{\prime} }}\cdots {x}_{N^{\prime} }^{{Q}^{{\prime} }})$, and the output matrix as $Z^{\prime} =({z}_{1}^{{P}^{{\prime} },{Q}^{{\prime} }},{z}_{2}^{{P}^{{\prime} },{Q}^{{\prime} }}\cdots {z}_{N^{\prime} }^{{P}^{{\prime} },{Q}^{{\prime} }})$. The computation involves summing weighted contributions of $MHA({F}_{{P}^{{\prime} }}^{{\prime} },{F}_{{Q}^{{\prime} }}^{{\prime} },{F}_{{Q}^{{\prime} }}^{{\prime} })$ based on attention scores:

$${z}_{i}^{{P}^{{\prime} },{Q}^{{\prime} }}=\mathop{\sum }\limits_{j=1}^{N^{\prime} }soft\max (\mathop{\alpha}\nolimits_{i,j}^{Cross-}){x}_{j}^{{Q}^{{\prime} }}{W}^{V,{Q}^{{\prime} }}+{h}_{v}$$

(9)

Where, the weight coefficient ${\alpha }_{i,j}^{Cross-}$ is defined as:

$${\alpha }_{i,j}^{Cross-}=\frac{1}{\sqrt{{d}_{Head}}}({x}_{i}^{{P}^{{\prime} }}{W}^{Q,{P}^{{\prime} }}+MCDE({p}_{i})){({x}_{j}^{{Q}^{{\prime} }}{W}^{K,{Q}^{{\prime} }})}^{T}$$

(10)

Finally, the output encapsulates the correspondence between point cloud as follows:

$${F}_{i}=MLP({\rm{cat}}[{F}_{{P}^{{\prime} }}^{{\prime} },{z}_{i}^{{P}^{{\prime} },{Q}^{{\prime} }}])$$

(11)

Loss function

The network we propose is based on end-to-end training and supervised training using ground truth. The specific loss function is as follows:

Feature Loss: We utilize a circular loss function to evaluate feature loss and constrain the per-point feature descriptors training for 3D point cloud. Its definition is as follows:

$${{\mathcal{L}}}_{FL}^{P}=\frac{1}{{N}_{P}}\mathop{\sum }\limits_{i=1}^{{N}_{P}}\log\left[1+\sum _{j\in {\varepsilon }_{p}}{e}^{c{\beta }_{p}^{j}({d}_{i}^{j}-\Delta p)}\bullet \sum _{k\in {\varepsilon }_{n}}{e}^{\lambda {\beta }_{p}^{k}(\Delta n-{d}_{i}^{k})}\right]$$

(12)

Where, ${d}_{i}^{j}$ represents the Euclidean distance between features, ${d}_{i}^{j}={\Vert {f}_{{p}_{i}}-{f}_{{q}_{j}}\Vert }_{2}$. ε_p and ε_n respectively represent the matching and unmatching points of the point set P_RS (random sampling points of the source point cloud), which correspond to the positive and negative areas, respectively. Δp and Δn represent positive and negative regions, and λ is a predefined parameter. Similarly, the feature loss ${{\mathcal{L}}}_{FL}^{P}$ of the target point cloud is calculated in the same way. The total feature loss is given by ${{\mathcal{L}}}_{FL}=\frac{1}{2}({{{\mathcal{L}}}_{FL}}^{P}+{{\mathcal{L}}}_{FL}^{Q})$.

Overlap Loss: We use a binary cross-entropy loss function for supervised training, defined as follows:

$${{\mathcal{L}}}_{OL}^{P}=\frac{1}{N}\mathop{\sum }\limits_{i=1}^{N}{O}_{{p}_{i}}^{label}\log ({O}_{{p}_{i}})+(1-{O}_{{p}_{i}}^{label})\log (1-{O}_{{p}_{i}})$$

(13)

Where, ${O}_{{p}_{i}}^{label}$ represents the overlapping mark of ground truth at point p_i, which is defined as follows:

$${O}_{{p}_{i}}^{label}=\left\{\begin{array}{l}1,\,\left\Vert {T}_{P,Q}^{GT}({p}_{i})-NN({T}_{P,Q}^{GT}({p}_{i}),Q)\right\Vert < {\tau }_{1}\\ 0,{\rm{otherwise}}\end{array}\right.$$

(14)

Where ${T}_{P,Q}^{GT}$ represents the ground truth rigid transformation between the overlapping point cloud, and NN represents the nearest neighbor. τ₁ is the overlap threshold. Similarly, the overlap loss ${{\mathcal{L}}}_{OL}^{Q}$ of the target point cloud is calculated in the same way. The total overlap loss is defined as ${{\mathcal{L}}}_{OL}=\frac{1}{2}({{\mathcal{L}}}_{OL}^{P}+{{\mathcal{L}}}_{OL}^{Q})$.

Matching loss: For any point p_i in the source point cloud, a corresponding feature can be found at point q_i in the target point cloud, indicating that p_i and q_i form a pair of matching points. To handle the sparsity of ground truth points after downsampling, we use a matching loss function to address this issue.

$${{\mathcal{L}}}_{ML}^{P}=\frac{1}{N}\mathop{\sum }\limits_{i=1}^{N}{M}_{{p}_{i}}^{label}\log ({M}_{{p}_{i}})+(1-{M}_{{p}_{i}}^{label})\log (1-{M}_{{p}_{i}})$$

(15)

Where, ${M}_{{p}_{i}}^{label}$ represents the ground truth label of the registration point p_i, defined as follows:

$${M}_{{p}_{i}}^{label}=\left\{\begin{array}{l}1,\,\left\Vert {T}_{P,Q}^{GT}({p}_{i})-NN({T}_{P,Q}^{GT}({p}_{i}),Q)\right\Vert < {\tau }_{2}\\ 0,{\rm{otherwise}}\end{array}\right.$$

(16)

Where, τ₂ represents the overlap threshold. Similarly, the matching loss ${{\mathcal{L}}}_{ML}^{Q}$ of the target point cloud is calculated in the same way. The total matching loss is given by ${{\mathcal{L}}}_{ML}=\frac{1}{2}({{\mathcal{L}}}_{ML}^{P}+{{\mathcal{L}}}_{ML}^{Q})$.

In summary, the overall loss function is ${\mathcal{L}}={{\mathcal{L}}}_{FL}+{{\mathcal{L}}}_{OL}+{{\mathcal{L}}}_{ML}$.

Ethics approval and consent to participate

Written informed consent has been obtained from the School of Information Science and Technology of Northwest University and all authors for this article, and consent has been obtained for the data used.

Results

Dataset, Implementation Details and Evaluation Metrics

To evaluate the effectiveness of the proposed algorithm in handling issues such as low-overlap, generalization, and large-scale scenes, we conducted extensive experiments on various datasets, including 3DMatch²⁷ and 3DLoMatch³¹ for indoor real scenes, and Odometry KITTI²⁸ for outdoor large-scale scenes, and validated it on various cultural heritage datasets, including the Terracotta Warriors and WHU-TLS²⁹.

3DMatch: The 3DMatch dataset comprises depth images from 62 different scenes collected from datasets like 7-Scenes and SUN3D. These scenes encompass various types such as houses, buildings, etc. Among these, 46 scenes are used for training, 8 for validation, and another 8 for testing. 3DLoMatch is a dataset derived from the 3DMatch dataset. Notably, the overlap ratio for 3DMatch and 3DLoMatch datasets is greater than 30 % and between 10 % to 30 %, respectively.

Odometry KITTI: The Odometry KITTI dataset is captured in real-time using the Velodyne HDL-64E S3 LiDAR scanner, covering urban, rural, and highway environments across 11 large scenes. The dataset is split into 0-5 scenes for training, 6-7 for validation, and 8-10 for testing.

Terracotta Warriors: The Terracotta Warriors dataset contains three-dimensional scans of the Terracotta Warriors excavated from the mausoleum of the First Emperor of China, Qin Shi Huang. These data encompass various quantities and types of Terracotta Warriors, including soldiers, chariots, and horses.

WHU-TLS: The WHU-TLS ancient architecture dataset was acquired using a RIEGL VZ-400 laser scanning system and a 5D Mark II camera. It consists of 9 sites and approximately 240 million points. The dataset includes unstructured features such as eaves and tiles, making it suitable for 3D model reconstruction of ancient buildings and digital preservation of cultural heritage.

Implementation Details: Our algorithm is deployed on a system with an NVIDIA RTX 4090 GPU, an Intel(R) Core(TM) i7-13700KF CPU @ 3.40GHz, and 128GB RAM, running on a Windows 10 operating system. We use PyTorch for training.

Evaluation Metrics: Following the metrics used in Predator³¹, REGTR²⁴, and GMCNet³⁴, we evaluate the datasets using Relative Rotation Error (RRE) and Relative Translation Error (RTE). Additionally, we use Registration Recall (RR) to evaluate the 3DMatch dataset, The definitions of RRE and RTE are as follows:

$$RTE={\left\Vert t-{t}^{GT}\right\Vert }_{2}$$

(17)

$$RRE=\arccos\left(\frac{trace({R}^{T}{R}^{GT})-1}{2}\right)$$

(18)

Where, R^GT and t^GT represent the ground truth rotational error and translational error, respectively.

Registration recall(RR) is defined as the root mean square error of the transformation being less than 0.2 m. The formula is shown below:

$$RMSE=\sqrt{\frac{1}{\left\vert {C}_{ij}^{GT}\right\vert }\sum _{(p,q)\in {C}_{ij}^{GT}}{\left\Vert {T}_{P,Q}^{GT}(p)-q\right\Vert }_{2}^{2}}$$

(19)

Where, ${C}_{ij}^{GT}$ represents the set of ground truth correspondences.

For Terracotta Warriors dataset, we use Chamfer Distance (CD) and Hausdorff Distance (HD) evaluate, The definitions of CD and HD are as follows:

$${\rm{CD}}(P,Q)=\frac{1}{| P| }\sum _{p\in P}\mathop{\min }\limits_{q\in Q}\parallel p-q{\parallel }_{2}^{2}+\frac{1}{| Q| }\sum _{q\in Q}\mathop{\min }\limits_{p\in P}\parallel q-p{\parallel }_{2}^{2}$$

(20)

$${\rm{HD}}(P,Q)=\max \left\{\mathop{\sup }\limits_{p\in P}\mathop{\inf }\limits_{q\in Q}\parallel p-q{\parallel }_{2},\mathop{\sup }\limits_{q\in Q}\mathop{\inf }\limits_{p\in P}\parallel q-p{\parallel }_{2}\right\}$$

(21)

3DMatch and 3DLoMatch

To validate the registration performance of the proposed algorithm under low-overlap scenarios, we followed the training methodology of the Predator approach and conducted validation on the 3DMatch and 3DLoMatch datasets. Additionally, we compared the proposed algorithm against other state-of-the-art methods, including FCGF¹⁸, D3Feat³⁰, Predator³¹, OMNet³⁵, REGTR²⁴, GeoTrans³⁶, RoReg³⁷, UDPReg³⁸, MAC³⁹, and RIGA²⁶. Figure 3 shows the registration visualization of the 3DMatch and 3DLoMatch datasets, respectively.

Fig. 3: Registration visualization on 3DMatch, 3DLoMatch.

3DMatch example shows accurate alignment of overlapping fragments, 3DLoMatch example demonstrates performance under low-overlap.

From Table 1, it can be observed that our proposed algorithm outperforms other methods in terms of registration results. When compared to UDPReg, RoReg, and RIGA algorithms, our proposed algorithm demonstrates improvements of 11 %, 4.1 %, and 11.2 % respectively in the RR metric for scenarios with low-overlap. In terms of time complexity, our algorithm demonstrates excellent performance in handling data of different scales. Compared to RoReg, GeoTrans, Predator, and D3Feat, our algorithm reduces the Time metric by 2.52 s, 0.08 s, 0.39 s, and 0.25 s respectively in low-overlap scenarios.

Table 1 Performance on 3DMatch and 3DLoMatch datasets

Odometry KITTI

To verify the registration performance of our proposed algorithm in large-scale scenes, we adopted the training methodology of the Predator approach. We then conducted validation on the outdoor Odometry KITTI dataset, which represents significant environments. Furthermore, we compared our proposed algorithm against other advanced methods, including FCGF¹⁸, D3Feat³⁰, Predator³¹, SpinNet⁴⁰, HRegNet⁴¹, GeoTrans³⁶, SHM_DGR⁴², GeDi⁴³, MAC³⁹, SC² -PCR++⁴⁴, RNC²⁰ and RIGA²⁶. Figure 4 shows the registration visualization of the Odometry KITTI datasets.

Fig. 4: Registration visualization on Odometry KITTI.

Visualizations show registration results under straight-road frame alignment and curved-road frame alignment.

From Table 2 and Fig. 4, it is evident that our proposed algorithm achieves superior registration results on the Odometry KITTI dataset compared to other methods. Through comparison with the performance of other algorithms, our proposed method demonstrates a distinct advantage in addressing point cloud registration tasks in large-scale scenes.

Table 2 Performance on Odometry KITTI dataset

Terracotta Warriors

To validate the accuracy and generalization ability of our algorithm on the Terracotta Warriors dataset, we used real-world fragments collected from the Mausoleum of the First Qin Emperor. This dataset offers a rich variety of cultural heritage point cloud samples, including fragments of different parts such as the head, upper body, arms, legs, and lower body. In addition, we compared our proposed algorithm with other advanced methods, including D3Feat³⁰ and Predator³¹.

As shown in Table 3, our proposed method demonstrates superior registration performance on the Terracotta Warriors dataset compared to methods D3Feat and Predator. Specifically, in terms of evaluation metrics, our method achieves the best results in both Chamfer Distance (CD) and Hausdorff Distance (HD). The CD value of 0.0066 is significantly lower than that of D3Feat (0.0089) and Predator (0.0081), indicating a smaller average distance between point clouds and thus higher registration accuracy. Additionally, the HD value of 0.2814 is also better than those of the other methods, suggesting reduced worst-case alignment error. These results strongly validate the robustness and accuracy of our algorithm in complex cultural heritage scenarios.

Table 3 Performance on Terracotta Warriors dataset.

In the experiment, we evaluated the performance of the algorithm by conducting registration tests on partial and complete Terracotta Warrior models and visualizing the data. Figures 5 and 6 show the registration results of different fragments, while Fig. 7 illustrates the registration visualization of the entire Terracotta Warrior model. The registration results demonstrate that our method effectively aligns fragments of the Terracotta Warriors and maintains geometric consistency even in areas of low-overlap. This indicates that our method has good applicability and robustness in handling complex heritage data such as the Terracotta Warriors.

Fig. 5: Registration visualization of small partial fragments on Terracotta Warriors.

It presents the registration results of the front and back sides of different parts, mainly including small fragments of the head, upper body, and lower body.

Fig. 6: Registration visualization of large partial fragments on Terracotta Warriors.

It presents the registration results of the front and back sides of different parts, mainly including large fragments of the arm, upper body, and leg.

Fig. 7: Registration visualization on entire Terracotta Warriors.

It presents the registration results of the front and back sides of entire parts, mainly including large fragments of the arm and leg.

Terracotta Warriors and similar relic fragments often exhibit wear, missing parts, and other issues due to prolonged burial, resulting in low-overlap between point cloud data and significant morphological differences between fragments. Under these conditions, traditional methods often produce noticeable errors at edges and in local details. The CDMCT method enhances the model’s perception of local geometric information through multi-channel dynamic encoding, maintains global structural consistency through a cross-domain convergence network, and adapts to local feature variations with the graph network. CDMCT provides an effective tool for 3D digital preservation in the field of archaeology, supporting the refined restoration and digital preservation of relics and offering promising potential for broader application in the preservation of cultural heritage.

WHU-TLS

To further validate the generalization capability of our proposed algorithm, we used real-world cultural heritage building data from the WHU-TLS dataset. This dataset contains high-precision point cloud data of multiple ancient buildings, covering complex geometries and large-scale scenes, with significant diversity and complexity.

Figures 8 and 9 show the application of our algorithm in independent experiments, including the registration of point cloud data for main gate and the both sides of the ancient building. Specifically, the registration of the main gate requires handling more geometric details and structural complexity, while the registration of the sides involves larger-scale geometric scenes. The results indicate that the algorithm not only accurately aligns the geometric shapes of different parts of the building but also maintains structural consistency, particularly in regions with low-overlap, successfully achieving global geometric alignment. This demonstrates that our method can address geometric differences between different data sources and maintain accurate perception of global structure.

Fig. 8: Registration visualization for main gate on WHU-TLS.

It presents the registration results of No. 6 and No. 7, as well as No. 4 and No. 5 for main gate.

Fig. 9: Registration visualization for both sides on WHU-TLS.

It presents the registration results of No. 8 and No. 9, as well as No. 2 and No. 1 for both sides.

Figure 10 shows the overall digital model of the WHU-TLS ancient building. It is evident that the algorithm successfully handles point cloud registration in large-scale scenes and can effectively merge point cloud data from different sources to construct a high-precision and detailed digital model. This performance is achieved thanks to the innovative modules introduced in our method, including multi-channel dynamic encoding and cross-domain convergence networks, which enable the model to process diverse geometric structures while maintaining global consistency.

Fig. 10: Digital modeling of WHU-TLS.

It presents the overall modeling result of the nine fragmented parts.

In light of the demands of cultural heritage digitization, the results of this experiment demonstrate the versatility of our algorithm when faced with complex and fragmented cultural heritage data. For large-scale architectural digital modeling, the CDMCT method provides reliable and accurate results, offering strong support for the preservation and restoration of cultural heritage.

Ablation Study

To validate the effectiveness of the individually selected modules in our model, we conducted ablation experiments on the 3DMatch and 3DLoMatch datasets.

From Table 4, it can be observed that all the proposed modules contribute to improving the registration accuracy of point cloud to varying degrees. In particular, compared to using only the Cross-domain Convergence Network (CC-Net), combining it with Multi-Channel Dynamic Encoding (MCDE) reduces the Relative Rotation Error (RRE) from 1.633° to 1.523° and the Relative Translation Error (RTE) from 0.057 m to 0.041 m on the 3DMatch dataset. On the more challenging 3DLoMatch dataset, the Registration Recall (RR) increases by 1.4 percentage points to 75.3 %, demonstrating the enhanced robustness of this module in low-overlap scenarios. In addition, the introduction of Multi-Channel Angle Encoding (MAE) also yields stable improvements, indicating that angular information serves as a valuable supplement for structurally complex targets. This performance enhancement is mainly attributed to the integration of MCDE on top of the original cross-domain convergence attention module, which effectively improves the algorithm’s sensitivity to local structural and angular variations. Moreover, the inclusion of local information within the attention mechanism also positively contributes to the overall performance of the algorithm.

Table 4 Ablation of different modules on the 3DMatch, 3DLoMatch datasets

Robustness Study

To assess the model’s robustness performance, we conducted noise experiments based on the indoor 3DMatch and 3DLoMatch datasets, as well as the outdoor Odometry KITTI dataset.

As shown in Table 5 and Fig. 11, the proposed CDMCT algorithm demonstrates strong robustness when subjected to varying levels of noise interference, both in indoor datasets (3DMatch and 3DLoMatch) and in the large-scale outdoor Odometry KITTI dataset.

Table 5 Results on 3DMatch and 3DLoMatch datasets

Fig. 11: Results on Odometry KITTI dataset.

a Comparison of RRE under different noise levels (b). Comparison of RTE under different noise levels.

In the indoor datasets, when the noise level increases from 0.005 to 0.04, the registration recall (RR) of CDMCT decreases by only about 1.8 % (from 94.1 % to 92.3 %), which is significantly better than methods such as D3Feat and Predator. Under the same noise variation, Predator’s performance drops by 3.6 %, while D3Feat sees a larger decline of 5.0 %. On the more challenging 3DLoMatch dataset, CDMCT also outperforms both Predator and D3Feat significantly in terms of RR.

Furthermore, as illustrated in Fig. 11, the relative rotation error (RRE) experiments on the Odometry KITTI dataset show that CDMCT maintains an RRE below 0. 4° across noise levels ranging from 0.1 to 0.5, whereas the RRE of Predator exceeds 0. 5°, indicating that CDMCT is more resilient to noise in large-scale scenes. Similarly, in terms of relative translation error (RTE), CDMCT consistently stays within the 6-8 cm range, while Predator exceeds 9 cm, further highlighting its instability under noisy conditions.

In summary, CDMCT exhibits excellent robustness and stability across diverse datasets, maintaining high registration accuracy even in low-overlap, complex, or high-noise scenarios, thereby validating its broad applicability and reliability in real-world applications.

Discussion

The proposed cross-domain multi-channel Transformer point cloud registration method (CDMCT) is designed to address common challenges in cultural heritage scenarios, such as low-overlap, partial data loss, large-scale scenes, and cross-domain generalization difficulties. It is particularly suitable for representative cultural heritage datasets such as the Terracotta Warriors and WHU-TLS ancient buildings.

In terms of multi-channel dynamic encoding, we introduce multi-channel angular relationship modeling, which leverages ternary angular positional encoding and dynamic weighting mechanisms to enhance the model’s sensitivity to complex local structures. In our experiments, the Terracotta Warriors dataset includes numerous curved surfaces, fine decorative details, and damaged cracks and features that traditional methods struggle to align accurately. CDMCT achieves high-precision registration on datasets, achieving the best performance in terms of Chamfer Distance and Hausdorff Distance, which validates the effectiveness and adaptability of the multi-channel dynamic encoding in handling complex surface structures in cultural heritage.

The cross-domain convergence network integrates the global modeling capabilities of Transformer with the local topological representation strength of graph neural networks. On one hand, it preserves the global spatial relationships between point clouds. On the other, it enhances robustness to local feature variations. In the cultural heritage domain, such a combination is uncommon, as most existing methods struggle to simultaneously maintain global consistency and handle local incompleteness. Our experiments on the WHU-TLS dataset demonstrate that CDMCT maintains high-quality registration even in the presence of occlusions and missing scans, indicating its clear advantages in complex heritage scenarios.

CDMCT is trained on public general-purpose datasets such as 3DMatch and KITTI and is validated on cultural heritage datasets. This choice is based on two considerations: first, heritage data are expensive to obtain, small in scale, and difficult to annotate, making them unsuitable for large-scale training; second, we aim to evaluate the model’s zero-shot cross-domain generalization to improve its practicality. Experimental results show that CDMCT achieves strong registration performance on previously unseen heritage datasets, demonstrating good domain transfer capability. However, this strategy has its limitations, as the model may not fully adapt to some extreme structural features. In future work, we plan to fine-tune the model with a small amount of heritage-specific data to further enhance its adaptability to cultural heritage scenarios.

In conclusion, the CDMCT method not only incorporates multi-channel geometric modeling and cross-domain structural fusion mechanisms tailored to the complexities of cultural heritage, but also demonstrates strong practicality and robustness through validation on representative datasets. It offers a promising solution for heritage point cloud registration with both strong performance and application value.