An automatic congenital radio-ulnar synostosis deformity evaluation method (CRUS-DE): integrating TLT-SAM and GPMM-R for landmark identification

Abstract

Congenital Radio-Ulnar Synostosis (CRUS) causes difficulty in forearm rotation, which is treated using osteotomy. Effective preoperative planning of osteotomy requires automatic and objective quantification of deformity angles. In this paper, an automatic Congenital Radio-Ulnar Synostosis deformity evaluation method (CRUS-DE) is proposed. Initially, a method using threshold-layer tracking (TLT) and the segment anything model (SAM) is designed to recognize and segment the forearm from CT images. Subsequently, the Gaussian Process Morphable Model, in conjunction with refinement based on anatomical characteristics is developed to accurately identify forearm landmarks. Finally, the model automatically estimates deformity angles for quantitative assessment of CRUS. The forearm landmarks and deformity angles were successfully obtained from CT images based on CRUS-DE with average errors ranging from 0.98 to 1.55 mm and from 0.7° to 2.4°, respectively. No significant differences existed between the automatic method and manual method. The CRUS-DE, as an explainable method, presented excellent performance in the quantification of deformity angles. This method can be used in preoperative planning and postoperative evaluation of osteotomy for forearm deformities.

Introduction

Congenital Radio-Ulnar Synostosis (CRUS) clinically presents as the fusion of the proximal radius and ulna, with an incidence rate of approximately 1/600¹. CRUS has serious influence on the forearm rotation function and range of motion², profoundly affecting the patient’s daily life, work, and psychological state³. Consequently, patients with CRUS are usually diagnosed and treated during childhood.

Computer-assisted medicine has rapidly developed in the past decade, playing a crucial role in various fields, including auxiliary diagnosis, preoperative planning and robot-assisted surgery^4,5,6,7,8,9. Osteotomy is the main treatment method for forearm deformities^{4,5,10,11,12,13}. Numerous studies have affirmed that computer-assisted preoperative planning for osteotomy effectively improve its accuracy^14,15,16. A key issue in preoperative planning is quantifying deformity angles, which can assist surgeons in comprehensively diagnosis the degree of deformities^17,18. The manual quantification exists several defects, the consistency is difficult to ensure, and for complex deformities, such as CRUS, which includes angular deformities in each anatomical plane and internal rotation deformity (Fig. 1a), manual quantification may be difficult to accurately capture all the details. Automatically quantifying deformities during the preoperative planning of osteotomy can evaluate osteotomy preoperative planning quickly and accurately, which can guide surgeons to improve osteotomy preoperative planning. Meanwhile, it can reduce subjectivity and individual differences in measurement results. In addition, the postoperative outcomes can be predicted. Therefore, automatic and objective quantification of deformity angles is imperative.

Fig. 1

Presentation of CRUS in medical images. (a) X-ray image and 3D model of forearm with CRUS, highlighting the proximal synostosis in red boxes. (b) Forearm landmarks in X-ray image. (c) Anatomical landmarks in the distal forearm: (i) Lister node (LN); (ii) dorsal endpoint of radial styloid process (DS); (iii) palmar endpoint of radial styloid process (PS); (iv) palmar endpoint of a sigmoid notch (SN); (v) ulnar styloid process (US). (d) Anatomical landmarks in the proximal forearm: (vi) radial tuberosity (RT); (vii) olecranon tip (OT); (viii) tip of coronoid process (CP); (ix) ulnar tip of olecranon (UTO); (x) radial tip of olecranon (RTO).

Quantification of CRUS cannot be separated with the identification of forearm anatomical landmarks. The forearm anatomical landmarks are situated in regions of bony tissue with distinctive characteristics, as illustrated in Figs. 1b-d. The anatomical coordinate system can be established based on forearm anatomical landmarks, and the anatomical planes can be determined as a benchmark to ensure consistency in deformity quantification. Besides, landmarks can be used to align the normal and CRUS forearms, and the deformity angles can be measured sequentially¹⁷.

Many studies concentrated on the automatic identification of anatomical landmarks and other anatomical characteristics. Baek et al.¹⁹. used the anatomical deformation method to construct a femoral statistical shape model to predict femoral anatomical landmarks. Phan et al.²⁰. utilized a femoral reference model with anatomical landmarks and surface segmentation maps as a single atlas to aid in the prediction of femoral anatomical landmarks. Subburaj et al.²¹. used a method based on Gaussian and mean curvatures to identify all feasible landmarks and a spatial adjacency matrix to discern anatomical landmarks. Kai et al.²². exploited the inertial principal axis, contour extraction, and spherical least squares fitting of the femoral head and posterior condyle to construct the femoral coordinate system. By doing so, the researchers identified the femoral coordinate system specified by Wu²³. However, these researches suffer from several inherent limitations: firstly, the measurements are subject to the operator’s experience and subjective judgment, leading to significant intra- and inter-observer variability¹⁷; secondly, for complex conditions like CRUS, which involves angular and rotational deformities in the coronal, sagittal, and axial planes, manual measurement struggles to accurately and consistently capture all details in three-dimensional space. Furthermore, existing automated orthopedic measurement tools are mostly developed for lower limb bones like femur, tibia^19,20,21,22 or are limited to 2D image analysis, and cannot be directly applied to the 3D deformity quantification of forearm bones.

Besides, there are also several challenges exist in forearm landmarks identification. Firstly, obviously differences exist in bony morphology and rotational posture. Secondly, forearm landmarks are located on both ends of forearm with low bone mineral density, automatic segmentation of forearm tends to be incomplete. What’s more, CRUS is often diagnosed and treated at childhood, pediatric forearm landmarks are less prominent compared to adult forearm landmarks²⁴.In this study, an automatic Congenital Radio-Ulnar Synostosis deformity evaluation method (CRUS-DE) is proposed to automatically quantify deformities from original CT images is proposed. First, the threshold-layer tracking and Segment Anything Model is developed to segment the forearm from original CT images (TLT-SAM), even under challenges like low bone mineral density and partial volume effects. Subsequently, a method, referred to as Gaussian Process Morphable Model and Refinement (GPMM-R), is established to accurately identify the subtle and less prominent landmarks in pediatric bones. Lastly, the model automatically quantifies the deformity angles. The accuracy is verified compared with the “Global Truth” labeled by senior surgeons, aiming to reduce subjectivity and improve the efficiency and objectivity of preoperative diagnosis, planning, and postoperative evaluation.

Method

Overview

The main research comprises three steps: Firstly, the forearm is segmented from original CT images after performing the forearm CT scanning, involving threshold-layer tracking to recognize forearms and accurate segmentation by segment anything model. Next, the forearm anatomical landmarks are identified based on GPMM-R method for coarse registration and landmark refinement. Finally, the deformity angles can be quantified using recognized forearm landmarks. Our method is applicable to preoperative diagnosis, planning and postoperative evaluation of osteotomy.

Subject cases

This study follows the ethical standards for research using human subjects established in the Declaration of Helsinki. This case series study was approved by the ethical review board of Beijing Jishuitan Hospital (202003-02). All data accessed in this study were fully anonymized. In addition, ethics committee waived the requirement for informed consent.

This study is a retrospective investigation. Twenty children treated at Beijing Jishuitan Hospital from March 2020 to March 2021 are included in the study, the demographic information is shown in Table 1. The original CT data was obtained during their examination. The data was collected and annotated by clinical surgeons after acquiring the original CT data. For research purposes, the data was accessed in 29/03/2021. Among these pediatric patients, case 1 to case 10 are suffered from bilateral deformity, while case 11 to case 20 are suffered from unilateral deformity. And our research was performed after that.

Table 1 Demographic data of cases included in the study.

All children underwent bilateral forearm CT examinations (Aquilion 64, TOSHIBA). During examination, the children were placed in the prone position with both forearms raised Abduction and elevation, palms down and elbows slightly flexed.Bilateral forearms were simultaneously scanned, ranging from the distal humerus to the carpometacarpal joint. The CT scans were conducted with a voltage and current set at 100 kV and 65 mA, respectively, featuring an image resolution of $\:512\times\:512$ pixels and a voxel size ranging from 0.3 to 0.9 mm, and a slice thickness of 0.8 mm. The CT series were saved in DICOM files.

Automatic forearm segmentation

The masks are generated using connected domain algorithm following the segmentation of the middle layer CT image through the threshold algorithm. A loss function is formulated to assess the similarity between each mask and the predicted shape of the forearm bone Sect. 

$$\:los{s}_{i}=\left[\begin{array}{ccc}{w}_{P}&\:{w}_{S}&\:{w}_{R}\end{array}\right]{\left[\begin{array}{ccc}\left|{P}_{i}-\widehat{P}\right|&\:\left|{S}_{i}-\widehat{S}\right|&\:\left|{R}_{i}-\widehat{R}\right|\end{array}\right]}^{T}$$

(1)

The weight parameters were set to $\:{w}_{P}=0.4$, $\:{w}_{S}=0.4$, and $\:{w}_{R}=0.2$ based on leave-one-out cross-validation. The predicted values $\:\widehat{P}$, $\:\widehat{S}$, and $\:\widehat{R}$ were obtained by averaging the geometric features of the middle-layer forearm bone masks from the healthy forearms in the data set.

where, the $\:{P}_{i}$, $\:{S}_{i}$ and $\:{R}_{i}$ represent the perimeter, area and regional density of the mask $\:{\varOmega\:}_{i}$, respectively, with weights assigned as $\:{w}_{P}$,$\:{w}_{S}$ and $\:{w}_{R}$. The symbols with “^” indicate the corresponding predicted values.

Four masks with the minimum $\:los{s}_{i}$are denoted as $\:{\varvec{L}}_{1}$,$\:{\varvec{L}}_{2}$,$\:{\varvec{R}}_{1}$and$\:{\varvec{R}}_{2}$. A threshold-layer tracking (TLT) algorithm, a layer-by-layer mask propagation method that uses shape and density similarity to track forearm bones across CT slices, is designed to track the masks. The algorithm details are explained in Tab.S1. As shown in Fig. 2a, recognition of the CRUS forearm when two separated masks merge into a single mask at a certain layer. To enhance segmentation accuracy, the result of the TLT method is integrated with the segment anything model (SAM)²⁵, which is termed of TLT-SAM in this article, to improve the incomplete segmentation. The midpoint of radius is set as the “add mask”, and several reference points along with the midpoint of ulna are designated as the “remove area”. Finally, the forearm bones are reconstructed based on marching cube algorithm.

Fig. 2

Details of forearm bone segmentation and reconstruction. (a) Adjacent layer tracking of forearm masks using an adaptive rectangular box. (b) Automatic division begins at the distal layer of synostosis (in the purple plane), and progresses toward the proximal end layer by layer. The green plane indicates the kth layer of the synostosis region.

The forearm is subdivided into radius and ulna to aid in landmark recognition. This subdivision begins at the distal layer of the synostosis region, where the fusion mask is separated into two groups using a clustering algorithm. To preserve the continuity between layers and the integrity of the bone section arc, the optimization objective considers the movement distance of the dividing line endpoint and the length of the dividing line. The corresponding loss is expressed as follows:

$$\:{f}_{k}({p}^{k},{q}^{k})={f}_{k1}\cdot\:{f}_{k2}={e}^{-(\frac{{\left(‖{p}^{k},{c}_{1}^{k-1}‖+‖{q}^{k},{c}_{2}^{k-1}‖\right)}^{2}}{2{\sigma\:}_{1}^{2}}+\frac{(‖{p}^{k},{q}^{k}‖-{d}_{min})}{2{\sigma\:}_{2}^{2}})}$$

(2)

$$\:{{d}_{min}={min}\left\{‖{p}^{k},{q}^{k}‖\right\}}_{,}\forall\:{p}^{k},{q}^{k}\in\:{\varvec{C}}^{k}$$

(3)

where, $\:{\varvec{C}}^{k}$ represents a set of contour points, $\:{c}_{1}^{k-1}$ and $\:{c}_{2}^{k-1}$ denote endpoints of the dividing line in the previous layer, $\:{\sigma\:}_{1}$ and $\:{\sigma\:}_{2}$ are Gaussian variances, which are determined by leave-one-out cross validation, as shown in Fig. 2b. The synostosis region is subsequently divided based on the obtained dividing points by minimizing Eq. (2).

Landmarks identification

Utilizing the reconstructed ulna and radius, a slice operation is conducted. The largest slice section perpendicular to the distal radius shaft axis, fitted by the midpoints of the distal 1/3 radius, is identified as the distal radius platform, which contour is denoted as $\:{\varvec{C}}_{T}$ (Fig. 3a).

The template for distal radius platform is established as a 2D template using Gaussian Process Morphable Model (GPMM)²⁵. Thirty healthy pediatric radii, with landmarks (LN, DS, PS, SN) labeled by a senior surgeon, are employed for template establishment. For each radius, the contour $\:\varvec{C}\:$at the distal radius platform is calculated once the distal radius platform is identified. $\:\varvec{C}$ is divided and interpolated by landmarks to normalize it. $\:O$ is defined as the center of $\:\varvec{C}$, and the radioulnar direction is defined as the line connecting $\:O$ to the midpoint of DS and PS, aiding in contour alignment. The GPMM for the distal radius platform is established using the contour points:

$$\:c\left(x\right)\sim\:\mu\:\left(x\right)+{\sum\:}_{i=1}^{r}{\alpha\:}_{i}\sqrt{{\lambda\:}_{i}}{\varphi\:}_{i}\left(x\right),{\alpha\:}_{i}\in\:N\left(\text{0,1}\right)$$

(4)

where, $\:\mu\:\left(x\right)$ is the reference contours, and $\:\left({\lambda\:}_{i},{\varphi\:}_{i}\right)$are the eigenvalue/eigenfunction pairs. So, registration of the distal radial platform and the GPMM template can be regarded as an optimization problem:

$$\:\underset{{\alpha\:}_{1},{\alpha\:}_{2},\cdots\:,{\alpha\:}_{r}}{argmin}D\left[{\varvec{C}}_{T},\mu\:+{\sum\:}_{i=1}^{r}{\alpha\:}_{i}\sqrt{{\lambda\:}_{i}}{\varphi\:}_{i}\right]+\eta\:{\sum\:}_{i=1}^{r}{\alpha\:}_{i}^{2}$$

(5)

where, $\:D$ is a metric to measure the similarity of $\:{\varvec{C}}_{T}$ and the deformation template, optimized iteratively using the Iterative Closest Point algorithm. And $\:\eta\:\in\:\varvec{R}$ is a weighting parameter which was set to 0.1 to balance the goodness-of-fit and the regularization of model deformation. The points on $\:{\varvec{C}}_{T}$ closest to the landmarks on the template are assigned as the corresponding landmarks, which are denoted as $\:l{m}_{LN}^{temp}$,$\:l{m}_{DS}^{temp}$,$\:l{m}_{PS}^{temp}$and $\:l{m}_{SN}^{temp}$, respectively (Fig. 3b).

Fig. 3

Details of identifying radius landmarks. (a) Identification of distal radius platform. (b) Registration of distal radius platform (depicted by the dark blue curve) and the template (shown in red curve). In Fig, LN, DS, PS and SN are indicated by brown, red, orange, and yellow square points. (c) Refinement of radius landmarks in the neighborhood. The dark blue and red points represent the landmarks before and after refinement. (d) Recognition of RT for a normal radius. (e) Evaluation of RT for a radius with CRUS.

However, two facts must be considered. First, statistical shapes are provided from the GPMM template, which is limited in the representation of individual differences. Besides, the landmarks are not strictly located on the distal radius platform. Therefore, it is essential to refine radial landmarks. A reference point $\:{O}_{RP}$ is obtained by moving $\:O$ (midpoint of $\:{\varvec{C}}_{T}$) to proximal orientation with $\:d$, which is approximately 1/3 of the distance between $\:O$ and $\:l{m}^{temp}$, and the final landmark $\:lm$ can be calculated as

$$\:lm=\underset{p}{argmax}{‖p-{O}_{RP}‖}^{2},p\in\:\varvec{U}\left(l{m}^{temp}\right)$$

(6)

where, $\:\varvec{U}\left(l{m}^{temp}\right)$is neighborhood of $\:l{m}^{temp}$ (Fig. 3c). For normal radius, RT located on the ulnar side of the proximal radius, and appears as a proximal apophysis. The slice in proximal radius and with the largest area is recorded as $\:T$. Corresponding to each point on $\:T$, The longitudinal section including the proximal radius axis and the point is established, and the degree of convexity is calculated compared with the adjacent bone cortex ($\:\delta\:$). The point with the largest bulge is recorded as the RT (Fig. 3d). Considering the loss of bony protrusion in the proximal radius with CRUS, RT is estimated based on the relative position of the ulna and radius, which is located on the midplane of the synostosis region and the line connecting ulnar and radius shaft axis point (Fig. 3e).

Considering the relative dispersion of proximal ulnar landmarks, a 3D GPMM template is established for the entire ulna. Thirty ulnar surface models are reconstructed, with the surface meshes simplified using a quadric based edge collapse strategy. Registration is performed using the Coherent Point Drift (CPD) algorithm to align the surface points. The GPMM of the ulna is obtained using Eq. (4), and the first four modes of variation of the ulna are shown in Fig. 4a. These modes mainly effect: (1) the length of the ulna; (2) the width of the ulnar olecranon and the angular bending of the distal ulna; (3) the convexity of the CP and the longitudinal depth of the olecranon; (4) the width of ulnar scoliosis. The registration of the target ulna and ulnar deformation model is achieved through Eq. (5), and the temporary ulnar landmarks are subsequently identified, followed by a refinement process. The proximal ulnar shaft axis is determined by fitting by midpoints on 1/5 to 1/2 proximal ulnar slices, while the vertical point of CP is recorded as the reference point $\:{O}_{RP}$. Equation (6) is used for ulnar landmarks refinement (Fig. 4b). At the distal ulna, a point situated far from the radius at the long axis endpoint of the ellipse, which is fitted by surface points adjacent to the ulnar distal plane, is recognized as US.

Fig. 4

Details of ulnar landmark identification. (a) Influence of the first four modes on ulnar shape. (b) Recognition and refinement of ulnar landmarks.

The automatic identification of forearm landmark involves GPMM template registration and refinement based on anatomical characteristics, referred as GPMM-R in the subsequent sections of the paper.

Deformity quantification

The quantification of CRUS is inseparable from the establishment of the forearm axes and coordinate systems. The axis points are recorded as the fitting center of circle on each slice except for the proximal ulna and fusion segment of radius, For the proximal ulna, the dorsal arc is used to fit the circle, while for radius fusion segment, the radial arc is used to determine the axis point. Sequentially, Distal radial and proximal ulnar coordinate systems are established, which are specified in Table 2 and depicted in Figs. 5a and b. The distal radius platform is fitted by four distal radius landmarks (LN, DS, PS, and SN). The coronal plane of the ulna is defined using UTO, RTO, and ulnar proximal reference point (UPR). In this step, UPR is defined as ulnar axis points at 1/5 of the proximal ulna.

Table 2 Forearm local coordinate systems.

Finally, the quantification of deformity angles of the forearm with CRUS can be calculated based on the landmarks, axis and coordinate system. The angular deformities, reflect the bending degree of the bone axis. The alignment between the normal forearm and CRUS forearm is performed after the local coordinate system is established. The angular deformity is evaluated by the angle between shaft axes of two forearm bones, including radial and dorsal angles of radius (RAR and DAR), and ulnar and palmar angles of ulna (UAU and PAU). The angular deformity is depicted in Fig. 5c using PAU as an example. The positive directions are designated as ulnar, dorsal, radial and palmar side for RAR, DAR, UAU and PAU, respectively. Considering the degree of CRUS exists individual differences, the axis of partial deformity forearm is used to compare with the normal forearm. And the selection of axis points is defined by two parameters: (1) Distance from the normal forearm axis to CRUS forearm axis ($\:{\delta\:}_{u}$,$\:{\delta\:}_{r}$); (2) The proportion of the shortest axis used to fit the axis ($\:{\eta\:}_{u}$,$\:{\eta\:}_{r}$). IRAR and IRAU describe the internal rotation angles, which are automatically quantify by RT and US, respectively (Figs. 5d-e).

Fig. 5

Details of quantification of deformity angles. (a) Distal radial coordinate system. The brown, red, orange and yellow points are LN, DS, PS and SN, respectively. (b) Proximal ulnar coordinate system. The grey, pink, blue and purple points are UPR, CP, UTO and RTO, respectively. (c) Quantification of angular deformities. (d) Quantification of IRAR. The green square point is RT. (e) Quantification of IRAU. The pink square point is US.

Results

Validation of forearm segmentation and reconstruction

The accuracy of forearm segmentation and reconstruction is essential for the rationality of subsequent landmark identification. Original CT images of one case were randomly selected, and forearm bone segmentation and reconstruction were conducted by both TLT-SAM method and manual method. The manual method was done by a senior hand surgeon using Mimics 20.0 software. The IOU was applied to verify the degree of similarity with manual segmentation, which is expressed as follows

$$\:IOU=\frac{A\cap\:B}{A\cup\:B}$$

(7)

Figure 6a intuitively showed the comparison of segmentation results, including manual masks, TLT masks and TLT-SAM masks. To clearly quantify the segment result, the forearm was divided into 10 segments, and the result of IOU was shown in Fig. 6b. The result showed that the IOU for TLT ranged from 0.567 ~ 0.999, and for TLT-SAM was 0.929 ~ 0.991, which meant the TLT method could identify the forearm bones stably, but the accuracy of segmentation was poor. For example, at proximal 10% of the forearm, the minimum IOU is only 0.681, indicating the BMD of proximal forearm was low and the partial volume effect was extremely serious, which is shown in Fig. 6c.

Fig. 6

Results of automatic forearm segmentation. (a) Comparison of forearm masks from manual, TLT, and TLT-SAM methods. From left to right, each two columns corresponded to the proximal, middle and distal CT slices. (b) IOU of forearm bone segmentation. The X-axis represented the proportion of the distance to forearm proximal end. (c) Comparison of bone reconstruction by TLT and TLT-SAM method.

The Dice Similarity Coefficient (DSC) and the 95th percentile Hausdorff Distance (HD95) were calculated to evaluate the segmentation performance. The TLT-SAM method achieved an average DSC of $\:0.902\pm\:0.021$(0.891~0.954) and an average HD95 of $\:1.23\pm\:0.45\text{\hspace{0.17em}}\text{mm}$(0.812~1.773), across all forearm bones.

Compared with the division results of the manual method and automatic method, the proximal surface points of radius and ulna were divided into four classes, and a confusion matrix was used to measure the division result. From Tables 3 and 93.8% for radial and 96.1% for ulnar surface points were classified correctly compared with the manual method. The misclassification region was mainly located in the posterior forearm and far from the proximal landmarks.

Table 3 Confusion matrix of surface points by two methods.

Accuracy of forearm landmark identification

The anatomical landmarks of 40 radii and ulnas were identified using GPMM-R method. To assess the accuracy of the recognition method, five senior hand surgeons independently marked the landmarks three repetitions on the CT slices. The average value of each landmark was used as the “Global Truth” (GT) to minimize subjective errors. Subsequently, the landmarks labeled by the surgeons and identified through GPMM-R method were transformed into the local coordinate system established based on the GT.

Comparing the identified landmarks with the GT landmarks revealed normally distributed errors. The mean Euclidean distance errors for LN, DS, PS, SN and RT were 1.14, 0.98, 1.01, 1.09, and 1.55 mm, respectively, and for OT, CP, UTO, RTO and US, the mean errors were 1.27,1.72,1.35, 1.21, and 1.14 mm, respectively. No significant difference was observed between the two methods. Figures 7a-b provide a qualitative representation of the identification result for a normal forearm and a CRUS forearm. Notably, landmarks marked by senior surgeons exhibited high subjectivity and variability. Intraclass correlation coefficients (ICC) were calculated to assess inter-observer and intra-observer reliabilities (two-way mixed effects, single measures, and absolute agreement) of the manual method, yielding results ranging from 0.053 to 0.868 and 0.317 to 0.681, respectively. However, the mean landmarks (GT) closely aligned with those identified using the automated method. Consequently, the automatic method provided reproducible results with significantly greater stability compared to the manual method.

We compared the GPMM-R method with the curvature-based method²⁰ to evaluate the performance of GPMM-R. The GPMM-R method achieved a significantly lower mean identification error than curvature-based method ( $\:p<0.05$).

For further analysis, we calculated the error projections in the distal-proximal (DP), medial-lateral (ML), and posterior-anterior (PA) directions (Fig. 7c). The T-test was performed, and the results were presented in Tab.S2. Moreover, there was no significant difference in projection errors between the two methods (P > 0.05,$\:\:\alpha\:$ = 0.05), except for the ML directions of PS and CP, and the PA direction of UTO.

Fig. 7

Results of forearm landmark recognition. (a) Normal forearm with recognized landmarks. (b) CRUS forearm with recognized landmark. (c) Boxplot of errors in different direction.

Among distal radius landmarks, LN in the ML direction exhibited the maximum error range from − 2.8 to 1.8 mm. This discrepancy was attributed to LN being convex in the PA direction, while its structure appeared relatively smooth in the ML direction. The most significant difference (P = 0.002) in the radius landmarks was observed in the error of PS in the ML direction. PS displayed an arc-like shape on the distal radius platform, contributing to an error in the tangent direction. In the labeling results by senior surgeons, the dispersion of PS was notably apparent compared to other radius landmarks. Additionally, PS is situated at the intersection of the distal radius growth plate and metaphysis, where the growth plate morphologically extended in the proximal direction. For that, PS, being closer to the optimized reference point, exhibited sequentially increasing recognized errors. Among ulnar landmarks, the CP in the ML direction exhibited the maximum error among all landmarks, and it also differed significantly from the manual methods. This discrepancy could be attributed to the relatively flat nature of CP in pediatric orthopedics, with an unobvious convex in the PA direction. In such cases, CP tended to be identified to other convex points on the arc in this situation, producing errors in the ML direction of CP. Dominant errors were noted in the PA direction of UTO (-3.1 to 1.2 mm) and RTO (-2.4 to 1.8 mm), attributed to the small gradient in the DP direction of proximal ulnar olecranon.

The forearms were categorized into normal and abnormal groups, and a T-test was performed to confirm the robustness of the method. The detailed results can be found in Tab.S3. DS exhibited significant differences in the ML direction (P = 0.004), attributed to the radial styloid epiphysis protruding proximally and DS being flatter for the abnormal radius compared to the normal radius. In CRUS, the development of the distal radius epiphysis might be retarded. Moreover, a smooth shape of the distal dorsal radius was observed at the distal radio-ulnar joint when the ulnar head was luxated to the dorsal radius. Additionally, CRUS caused the radial side of the olecranon to develop a straight morphology in the sagittal plane, significantly altering the RTO in the DP direction, leading to the significant difference between the two groups (P = 0.008).

Quantification of bony deformity in CRUS

The deformity angles were further quantified based on identified anatomical landmarks, axes and coordinate systems. Simultaneously, five senior surgeons with more than five years of clinical experience measured the deformity angles manually after employing Multi Planar Reconstruction (MPR)^26,27 to transform the CT images into local coordinate systems, and the mean angles of these measurements were recorded as the “Global Truth” (GT).

The RAR, DAR, IRAR, UAU, PAU and IRAU of 40 forearms were calculated. Compared with GT, the results of the quantification of deformities are normally distributed. A T-test (CI = 95%) was performed between the results calculated by automatic method and GT, indicating no statistical difference between the two methods, the mean error of the six deformity angles ranged from 0.7 ~ 2.4°, suggesting that the automatic quantification results exhibit good consistency with the manual methods (Table 4).

Table 4 Error of deformity angle quantification.

The difference between normal and CRUS forearms was further analyzed based on the quantified deformity angles. Forty forearms were classified into normal and CRUS groups. Figures 8a-b showed the quantification angles for radii and ulnas, and T-test was performed between the deformity angles of normal and CRUS groups, the results were listed in Table 5.

Fig. 8

Comparison of quantification angles between normal forearm and forearm with CRUS. (a) Boxplot of radius deformity angles. (b) Boxplot of ulnar deformity angles. (c) Three-dimensional illustration of deformity angles of radius with CRUS. (d) Three-dimensional illustration of ulnar quantification angles.

Table 5 T-test for deformity angles between normal and CRUS forearms.

Obviously, the deformity angles of normal forearms were smaller than that of CRUS forearms, and all the deformity angles of CRUS forearms were significantly different from those of normal forearms. Compared to normal forearm, the CRUS forearm had a larger degree of angular deformity, indicating the distal end of the forearm bone deviated from the ideal position, which has a certain impact on hand function. The internal rotation angles exist within a large range of variation, indicating the individual differences in the degree of deformity.

Discussion

Forearm landmarks identification and deformity quantification are prerequisite of preoperative planning for osteotomy. Though the clinical qualitative diagnosis of CRUS is based on whether the proximal ulna and radius are fused on X-ray or CT images, the quantification is essential for evaluating the degree of deformity. The CRUS is complex with the deformity appearing in multiple directions, which is difficult to quantify on X-ray images. Therefore, the diagnosis and preoperative planning are always performed based on CT images. This process is performed by senior surgeons, which involves three steps: (1) aligning the distal radius or proximal ulna with a normal forearm bone; (2) The pair of bones are transformed into the coronal, sagittal and transverse planes, respectively; (3) The angles between CRUS and normal forearm are measured manually in anatomical planes. However, the manual operation is subjective and heavily relies on clinical experience. Therefore, the study proposed an automatic Congenital Radio-Ulnar Synostosis deformity evaluation method (CRUS-UE) to realize the deformity quantification from original CT images, including forearm segmentation using TLT-SAM method, pediatric forearm landmarks identification using GPMM-R method and deformity quantification based on recognized landmarks.

While our study had a limited number of subject cases, which is insufficient for training a 3D deep learning network. We leveraged the inter-layer continuity of forearm masks. Initially, threshold segmentation and bone morphology prediction were applied to the middle layer of CT images to identify the forearm masks. Subsequently, a layer-by-layer tracking method was utilized to sequentially segment the entire forearm. To improve the recognition of the landmarks, the SAM was then employed for fine segmentation. Our method circumvented the complexity of training a deep learning network, and the results demonstrated satisfactory segmentation. Importantly, we observed that the segmentation at both ends of the forearm was relatively complete, having little effect on landmark identification.

In addition to quantifying deformities, the accurate identification of forearm anatomical landmarks is crucial in preoperative planning for osteotomy. This process involves aligning anatomical landmarks to determine whether the bone fragment is corrected into the ideal position. Because of the limited samples’ number, the GPMM is used. Landmarks labeled by senior surgeons are subjective, as shown in Fig. 7, introducing uncertainty in osteotomy preoperative planning. Meanwhile, pediatric bony landmarks are relatively smooth, the refinement after template matching is included to fully consider the anatomical characteristics of these landmarks, reflecting individual differences. Once the clinical surgeon or therapist input the DICOM file of CRUS forearm, a visual CRUS forearm 3D-reconstruted model with highlight landmarks and a chart with the deformity values present. Therefore, our method is explainable, which is important for surgeons, and the automatic landmark identification method is theoretically highly accurate.

Our automatic deformity quantification method is reliable, because it had good consistency with the manual methods. Significant differences were found in the deformity angles between normal and CRUS forearm. For deformity angles in the coronal plane, the RAR was always positive, illustrating the radial head leans towards the ulna, forming the proximal radioulnar joint, while the RAR was much bigger for CRUS radius, since that CRUS led to the fusion of ulnar side of the radial head and the proximal ulna, and the radius and ulna shared the same medullary cavity in severe cases, which made the missing in the ulnar side of the proximal radius. The radial side of the radius presented a pointed cone, and attached to the shaft of the proximal ulna. The UAU is much bigger for CRUS ulna, because the distal radio-ulnar joint was dislocated for patients with CRUS, and the ulnar head deviated from the ideal position. For deformity angles in the sagittal plane, The DAR and PAU were always positive (from 17.9 to 24.8° and from 12.0 to 17.9°, respectively), which indicated for the CRUS forearms, the distal forearm tended towards the palmar side. That’s for the radius in CRUS patients was usually in a pronation posture, the radius axis rotated from the proximal radial side across the ulna to the distal ulnar side. It had been shown in the literature that DAR was the deformity that had the greatest impact on CRUS¹⁶.

Surgeons could evaluate the degree of deformity objectively and repeatable, which is meaningful for clinical diagnostics, especially for surgeons with limited experience. Besides, correcting the deformity angles to achieve the osteotomy goal is a necessary consideration in the planning of osteotomy. Meanwhile, the evaluation of postoperative correction of bony deformities would also be realized based on the deformity angles. Furthermore, our method is expected to guide the robotic system to execute personalized and precise osteotomy. With the help of the automatic-generated optimal osteotomy method and patient specific instrument which are based on this CRUS-DE,^28,29 the robotic system can adjust operations based on real-time feedback, ensuring the accurate achievement of surgical goals. We anticipated that this integrated approach will offer a more advanced and intelligent treatment of CRUS, insights from human-exoskeleton cooperative control and personalized assistance strategies^30,31 could inform the development of more intuitive and adaptive robotic osteotomy procedures.

By combining the automated quantitative results with patient-specific instruments²⁹ and real-time navigation, we aim to enable more personalized and precise osteotomies. Simultaneously, exploring the extension of this method to the evaluation of long bone deformities in other anatomical locations is a valuable future direction.

However, the study has a few limitations. Firstly, CRUS is categorized into four types, and osteotomy is necessary for both type III and type IV, however, in clinical experience, CRUS is a rare congenital malformation with extremely uneven classification. Type 3 patients account for the vast majority of all patients, especially those seeking for treatment. So, our primary focus was on the type III due to the limited number of cases. Secondly, our research did not encompass children who had experienced fractures or undergone osteotomy. And due to the limited overall number of patients with CURS as a rare disease, the dataset included in the study is relatively small. As a result, in this research, there is a certain degree of overlap between the Global truth data and the training-testing data, though this problem is mostly solved by using GPMM-R, but may still introduce some level of bias.

Regarding computational efficiency, the current CRUS-DE need manually input the CT data, which may cost a lot of time, with segmentation requiring about 2 min and landmark identification and angle quantification requiring about 3 min. This demonstrates potential for clinical practice, but further speed improvements through code optimization and parallel computing are planned.5 Conclusion.

Congenital Radio-Ulnar Synostosis (CRUS) is comprehensively congenital deformities with deformity in all directions, manual quantification is subjective and variability. For overcome this issue, an innovative Congenital Radio-Ulnar Synostosis deformity evaluation method (CRUS-DE) is proposed in our study for quantifying deformity angles for forearm with CRUS, which is essential for preoperative diagnostics, planning and postoperative evaluation.

The radius and ulna were recognized and segmented from original CT images by the TLT-SAM method, and then a method combined with GPMM and refinement (GPMM-R) was proposed to recognize forearm anatomical landmarks. Finally, the deformity angles were quantified to evaluate the degree of deformity based on recognized landmarks. The conclusions were as follows:

• 1. The segmented forearm bone masks by the TLT-SAM method have a high consistency with the manual method, with IOU ranging from 0.929 ~ 0.991.

• 2. The accuracy of recognized landmarks is verified by comparing with the “Global Truth” labeled by senior surgeons, and the average error ranges from 0.98 mm to 1.55 mm.

• 3. The automatic deformity quantification method is reliable, and the automatic calculated results have no significant differences compared with those of manual measurement.

In conclusion, automatic deformity quantification is realized based on CRUS-DE, which is potential to be used in preoperative diagnostics, planning, and postoperative evaluation of osteotomy. Furthermore, it would be applied in robot-assisted surgery.