Introduction

Thyroid cancer is the most common cancer in the endocrine system and the tenth most prevalent cancer in the world1,2. Thyroid nodules are increasingly assessed for the detection of malignancies using ultrasonography due to its non-invasive nature, cost-effectiveness, and availability2,3. However, the low diagnostic accuracy of ultrasonography due to low specificity in thyroid cancer diagnosis often results in an increased number of fine-needle aspiration biopsies for benign cases1,3. The diagnostic specificity of ultrasonography can be increased by using contrast-enhanced ultrasound4 or newer imaging modalities including strain and shear wave elastography1,4,5,6,7. Additionally, ultrasound-based power Doppler imaging has demonstrated effectiveness in detecting malignancy in thyroid nodules based on the blood flow patterns and vascularity3,8,9. Recently, contrast-free high-definition microvasculature imaging (HDMI) was introduced to visualize the tumor microvasculature structures at a sub-millimeter scale10,11. This technique quantifies the morphological features of tumor microvessels which help in distinguishing malignant from benign thyroid nodules1. The microvessel images are generated from the acquired raw ultrasound data consisting of a sequence of frames, using morphological filtering and vessel enhancement, making them suitable for quantifying these features10,13,14,16,17,18,19,20.

Recent published research used the quantitative HDMI (qHDMI) features as new biomarkers for differentiating benign and malignant thyroid nodules1. However, the proximity of thyroid nodules to the pulsating carotid artery introduces physiological motion between successive image frames in the acquired ultrasound data. This motion can contaminate the resulting power Doppler HDMI images3,12, leading to degradation in the microvessel image quality and alterations in the quantitative morphological features12. Therefore, in large scale clinical datasets, motion can lead to misclassification of the thyroid nodules resulting in inaccurate diagnosis.

The objective of the present study is to classify the thyroid nodules using quantitative HDMI biomarkers derived from motion-containing and motion-corrected power Doppler images and to evaluate performance in both scenarios. A comprehensive analysis is presented for groups with high motion to demonstrate the effectiveness of motion-corrected HDMI in thyroid nodule classification. We hypothesize that motion correction can enhance the vessel visualization and improve the accuracy of thyroid nodule classification.

Methods

In-vivo thyroid dataset

This prospective study was conducted in accordance with the Health Insurance Portability and Accountability Act (HIPAA) and under the guidelines and regulations of the Mayo Clinic institutional review board (IRB)-approved protocol (IRB#: 08-008778). Prior to participation, each patient provided a written Mayo Clinic IRB-approved informed consent, including permission for publication. Participants were 18 years or older with suspicious thyroid nodule(s) referred to fine-needle aspiration biopsy (FNAB). HDMI study was performed prior to FNAB. For nodules with positive or indeterminate FNAB results, surgical procedures were conducted. Histopathological findings from FNAB and/or surgical pathology served as the gold standard reference. Final diagnoses for all malignant cases, irrespective of nodule size, were based on surgical pathology results. For benign cases, diagnoses were based on standard clinical findings and FNAB results, as surgery was not required for benign nodules.

High-definition microvasculature imaging

Thyroid nodules were assessed using an ultrasound platform with plane wave imaging capabilities, specifically the Alpinion E-Cube 12R ultrasound scanner (Alpinion Medical System Co., Seoul, South Korea), equipped with an L3-12H linear probe operating at 8.5 MHz. To reduce motion artifacts, patients were asked to remain still and hold their breath for approximately three seconds during data acquisition. However, the physiological motion due to pulsating carotid artery remains inevitable. Once the nodule was located in B-mode ultrasound, a sequence of raw in-phase quadrature-phase (IQ) data was acquired and processed as described in10,11. Detailed methodologies for HDMI image formation, vessel extraction, denoising, morphological filtering, and the steps for vessel quantification can be found in3,10,13,14,15.

Inter-frame motion correction of HDMI

We utilized our recent framework, IFMoCoNet12, for correcting motion between the successive IQ frames prior to be processed for extracting the power Doppler, as shown in Figure 1. The framework used a low-complex deep learning network comprising two blocks of convolutional and depth-wise separable convolutional layers, followed by a hybrid adaptive attention block, two depth-wise separable convolutional layers, a squeeze-and-excite attention block, and a final convolutional layer for prediction12. Batch normalization was utilized after each convolutional layer, except the final layer. The input motion containing IQ frames were fed along with the ground truth/reference frames to the first convolutional layer which used 64 kernels/filters each with a size of 3 × 3. He uniform initialization was used for the filter weights, and Gaussian error linear unit (GELU) activation was applied to the convolved output. The corresponding 64 output feature maps were presented as inputs to the second layer, i.e., depth-wise separable convolutional layer. This layer used 128 filters, GELU activation, and performed a computationally less intensive two-step operation including a depth-wise and a pointwise convolution12. The 128 feature map outputs were presented to another block of convolutional and depth-wise separable convolutional layer, each with 64 filters with size of 3 × 3. The corresponding outputs were further given as inputs to a hybrid adaptive attention block which combined dilated convolutions (3 × 3 and 5 × 5) with channel and spatial-level attention mechanisms for extracting robust features with different receptive fields and scales12. The corresponding output was further enriched with channel-level features using a squeeze-and-excite-attention block consisting of global average pooling and GELU activation as described in 12. The attention map output was fed to the final layer, i.e., convolution with a single filter and GELU activation for predicting the motion corrected data/frame(s). A more detailed description of IFMoCoNet layers, their operations can be found in12.

Fig. 1
figure 1

Thyroid nodule differentiation using quantitative features derived from motion containing and motion-corrected HDMI obtained after using the recently proposed motion correction framework, IFMoCoNet.

Full size image

The network was trained using the motion containing IQ frames and the corresponding reference/ground truth frames extracted as the frames with the maximum correlation with all other frames in the IQ sequence. We used the frames of 80% of the subjects for training, 10% for validating, and the remaining 10% for testing the proposed network. Optimal hyper-parameter tuning was used in Keras tuner for selection of hyper-parameters. Based on that, we use the number of filters/kernels as 1, 64, 128, 256 for the convolutional layers, each with a size of 3 × 3 and a stride of 1, along with batch sizes of 2-8 and epochs as 40-100. Adam optimizer with a learning rate of 0.0005 was used for minimizing the loss function, i.e., mean square error, and maximizing the similarity metric, i.e., cosine similarity in this work. The frames which had low correlation with rest of the frames in the sequence were identified as having more motion. The mean inter-frame correlation (MIFC) was used as a measure to assess the amount of motion in the sequences. MIFC was computed as the mean of the mean of correlation of each frame with each other frame in the sequence. Detailed description of the reference frame selection scheme can be found in12. The trained network predicted the motion corrected IQ frames for the thyroid dataset, which were further processed to obtain the power Doppler using the spatio-temporal restructuring, singular value decomposition for clutter removal followed by background removal using top hot filtering and vessel enhancement using Hessian filtering10. As an example, we show the microvasculature Doppler image in Figure 1 for motion-containing as well as corrected frames obtained after these post-processing steps. It can be observed that the vessels are relatively less blurred after correcting motion. Full analysis of the results of IFMoCoNet can be found in12.

Representative HDMI images obtained from IQ data, along with the corresponding spatio-temporal correlation matrices (MIFC), before and after motion correction for five participants are shown in Fig. 2. This figure illustrates the improvements in both the HDMI images and the MIFC matrices following motion correction. Due to the presence of real motion artifacts in the in-vivo thyroid dataset, ground truth HDMI images were not available. Therefore, we computed the MIFC values based on the spatio-temporal matrices to assess the quality of images prior to and after motion correction. It is notable that IFMoCoNet substantially improved the MIFC scores from 0.59, 0.29, 0.42, 0.53, and 0.42 to 0.92, 0.76, 0.78, 0.90, and 0.78, respectively. Further, the microvasculature appears to be superior in terms of visual assessment, and background noise was substantially suppressed after motion correction.

Fig. 2
figure 2

Spatio-temporal correlation matrices and HDMI images from 5 representative cases. (A and B) are correlation matrices and HDMI images before motion correction in cases 1–5. (C and D) are correlation matrices and HDMI images after motion correction in cases 1–5. MIFC: means inter-frame correlation.

Full size image

Morphology-based features of HDMI

We extracted microvessel morphological features of the motion-containing and motion corrected power Doppler HDMI images to be used as quantitative biomarkers for differentiating benign and malignant thyroid nodules. In a previous study, these features have been used to classify thyroid nodules1. These vessel morphological features include vessel density (VD)13,21,22, number of vessel segments (NV), number of branch points (NB)13. We estimated the features NV and NB per unit area of the nodule region for neutralizing any bias. Murray’s deviation (MD) has shown to present a diameter mismatch which makes it a crucial biomarker for thyroid nodule differentiation1. Detailed description of the calculation of MD can be found in14. Some other features are also included in this study for additional diagnostic information as follows. Microvessel fractal dimension (mvFD) signifies the structural complexity of the network of vessels. Complete details are mentioned in14. The vessel density ratio (VDR) is computed as the ratio of the vessel densities of the center of the tumor and the periphery. Finally, intratumoral or peritumoral vascular distribution pattern is identified by the spatial vascularity pattern which is estimated by the VDR14,23. The detailed proposed steps for morphological operations and quantification are well described in our previous studies13,14. In total, nine features were considered for subsequent analysis.

Machine learning-based analysis and classification

For analysis and classification of malignant vs. benign nodules, we applied support vector machine (SVM) classifiers with a radial basis function kernel trained with the HDMI biomarkers as the input feature set. We trained two separate models based on the usage of motion-containing HDMI and motion-corrected HDMI biomarkers. Given the relatively small sample size, we adopted a nested cross-validation (NCV) design, such that the dataset was randomly divided into ten different outer folds with each fold having approximately 10% of the full dataset. Support Vector Machine (SVM) models were then trained using each unique combination of the nine outer folds, with the remaining one outer fold serving as the test set. Model implementation was performed based on the library libsvm in Python Scikit learn machine learning framework with default parameter settings as follows. L-2 regularization parameter of 1.0, squared hinge loss function, and RBF kernel gamma value of 2.17 ×10-18 as estimated from the default ‘scale’ value in libsvm.

For classification performance assessment, different benchmark classification measures were estimated for models derived from both the motion-containing and motion-corrected feature sets. These include classification accuracy (ACC), sensitivity (SEN), specificity (SPE), Mathew’s correlation coefficient (MCC) and the F-1 measure. Detailed description of these metrics can be found in24. Measures were aggregated across the NCV outer fold test sets using mean and standard deviation (SD). All implementation was done in Python 3.10. We further estimated the permutation importance of the biomarkers using SVM, to analyze the relevance of all features for classification, before and after motion correction.

Results

Study cohort

The study cohort included a total of 347 enrolled participants. Based on surgical pathology and FNAB results, the outcome of 25 participants was indeterminate. Thereby, we excluded them from this study and performed the analysis for 322 participants with a mean (SD) age of 55.04 (15.18) years. This study cohort consisted of 123 male and 199 female participants. According to the histopathological results of FNAB, 197/322 (61%) of thyroid nodules were confirmed to be benign, and 125/322 (39%) of the nodules were confirmed to be malignant by surgical pathology. Of the total number of benign nodules, 112/197 nodules belonged to females with a mean (SD) age of 56.03 (13.99) years and 85/197 nodules belonged to males with a mean (SD) age of 58.75 (13.31) years. Of the total number of malignant nodules, 87/125 nodules belonged to females with a mean (SD) age of 47.66 (14.44) years and 38/125 nodules belonged to males with a mean (SD) age of 49.87 (15.33) years.

Motion correction

The distribution of qHDMI images of nodules with respect to the mean inter-frame correlation values of the IQ frames before and after motion correction are shown in Figure 3. We observed that 19 out of 322 images (6%) had the lowest MIFC (< 0.2) before motion correction, which decreased to 6 out of 322 (1.86%) after correction. Further, the proportion of nodules with qHDMI images having MIFC values above 0.6 increased from 149/322 (46%) to 184/322 (57%) after motion correction. The number of nodule images in the MIFC range of 0.2–0.6 fluctuate between the motion containing and corrected cases, likely reflecting varying rates of improvement. For example, motion correction improved the correlation for 13 of 19 nodules that initially had low MIFC values (0.2–0.4) in their HDMI images. Based on these results, nodule images with MIFC values below 0.5 in the original data were identified as exhibiting “high” motion.

Fig. 3
figure 3

Participant data/image distribution with respect to (w.r.t.) MIFC values for motion containing (X-axes) and motion-corrected (Y-axes) IQ frames obtained after using the recently proposed motion correction framework, IFMoCoNet.

Full size image

We present the distribution of biomarkers for benign and malignant nodules of all participants before and after motion correction in Table 1. The values of all biomarkers are averaged over the participants, for both benign and malignant nodules. Further, to assess the statistical significance of the discrimination between the benign and malignant nodules by these biomarkers, we estimate the corresponding p-values using the paired t-test in MATLAB, as mentioned in the table. Our analysis revealed that the biomarkers NV, NB, VD, mvFD, MD and BA were significantly higher (p < 0.05) for malignant nodules, while VDR did not show significant increase. Similar trends were observed after motion correction, except for BA. Notably, as shown in Table 1, BA values decreased in malignant nodules, further accentuating the difference between benign and malignant lesions.

Table 1 qHDMI Biomarker Values (Mean (SD)) and Corresponding p-Values for All 322 Subjects Before and After Motion Correction.
Full size table

Biomarker importance for classification

In this study, we analyzed the statistical significance in terms of relevance of different biomarkers in discriminating between the malignant and benign thyroid nodules by using SVM classifier before and after motion correction. Figure 4 displays the permutation importances of all biomarkers25. A permutation importance greater than 0 indicates that a biomarker impacts the model’s classification if shuffled in a random order, and thereby, is statistically significant. Before motion correction, the biomarkers VDR, BAmax, VD, BAmean and NB showed positive permutation importances in the descending order, indicating that these biomarkers affect the model’s performance notably if shuffled in random order, with VDR and BAmax affecting the most. While NV exhibited a negative permutation importance, suggesting that the model’s performance is not affected by this biomarker at all. After motion correction, the significance of BAmax and BAmean increased, as evidenced by their higher permutation importance values, and NV adjusted to 0.

Fig. 4
figure 4

Permutation importance of biomarkers used for training SVM before and after motion correction.

Full size image

Classification performance analysis

Table 2 illustrates the mean classification performance measures across all NCV folds (i.e., all 322 subjects) using the original (motion-containing data), motion corrected data obtained from our proposed framework along with that obtained from an existing two-stage image registration method. We observed that the mean overall accuracy increased by 2% when using the motion-corrected HDMI biomarker, with similar modest increases for the other performance measures after motion correction by the proposed framework. Given the predominance of participants with high inter-frame correlation, we additionally summarized the classification measures for the high-motion subset in Table 2. We included the data from this group in four of the ten folds.

Table 2 Mean (SD) cross-validated classification performance for all participants (N = 322) and high motion containing data subset using original, proposed and two-stage motion corrected data.
Full size table

The corresponding relative changes in metric values following motion correction for both cases are shown in Fig. 5. The percentage of relative change was estimated as the difference between the post- and pre- correction metrics with respect to original (pre-correction) value. It can be observed that for the 116 cases, there was a considerable increase in the metric values, ranging from 4.8 to 13%, while the change was lower for the full dataset, as expected.

Fig. 5
figure 5

Percentage improvement in the classification metric after motion correction relative to original value, for full dataset of 322 subjects and high motion cases.

Full size image

We computed the confusion matrices for the 116 cases to evaluate the classification performance after and before motion correction, as illustrated in Fig. 6 (a) and (b) respectively. The matrices reveal that fewer cases were classified as falsely malignant after motion correction and the number of cases falsely detected as benign were reduced from eleven to eight.

Fig. 6
figure 6

Confusion matrices for (a) Motion-containing and (b) Motion-corrected cases, for high-motion participants.

Full size image

AUC comparison using DeLong’s test

To analyze the statistical significance of the difference in the AUC values before and after motion correction of the high motion subset, we performed the DeLong’s test25. This test is commonly used to assess whether the difference between the AUCs for two different machine learning models (M1 and M2) is statistically significant or not25. It estimates the z-score for the AUC’s as follows26:

$$z=(AUC\left({M}_{1}\right)-AUC\left({M}_{2}\right)\text{)/}{\text{sqrt}}\text{(}{\text{var}} \, \text{{}\text{AUC(M}{1}\text{)}} \, \text{+} \, {\text{var}} \, \text{{}\text{AUC(M}{2}\text{)}} \, - \, {2} \, {\text{cvar}} \, \text{{}\text{AUC(M}{1}\text{)} \, \text{,} \, \text{AUC(M}{2}\text{)}}\text{)}$$

where: sqrt denotes the square root function, var and cvar denote the variance and covariance respectively. In this study, we considered one model trained using two different datasets, i.e., motion-containing and motion-corrected data. So, we performed the DeLong’s test for comparing the AUC’s obtained by the model trained with these two cases. The z-score was translated to the p-value for determining the statistical significance as follows.

$$p = {2} \times {\text{normcdf}}\left( { - |z|} \right)$$
(1)

where normcdf denotes the normal cumulative distribution function and denotes the absolute value. In this study, we used MATLAB 2023a to compute the p-value. The AUC values were 0.81 (confidence interval (CI): [0.76, 0.97]) and 0.92 (CI: [0.87, 0.98]) before and after motion correction respectively, along with the corresponding DeLong’s test p-value of 0.07. It can be observed that there was a noticeable difference between the AUC values, although the p-value was slightly higher than the benchmark threshold of 0.05 for statistically significant.

Performance comparison

In our previous work12, we demonstrated the superiority of IFMoCoNet with respect to the two-stage method in terms of performance as well as time complexity. In this work, we performed a comparative evaluation study of the thyroid nodule classification using our proposed inter-frame motion correction framework with that using an existing motion correction method based on two-stage image registration approach30. For this purpose, we used the motion corrected data obtained from the two-stage method to reconstruct the HDMI and extract the qHDMI biomarkers for classifying the thyroid nodules. We observed that 12 cases were misclassified as compared to the uncorrected data, while those 12 cases were either correctly classified or remained same in the case of IFMoCoNet. This resulted in marginal changes in the overall performance by using two-stage method as compared to uncorrected data. We added the two-stage method performance metrics for comparative evaluation in Table 2. The two-stage motion corrected data performed almost the same as the original/uncorrected data in terms of accuracy, i.e., 0.86, however, it achieved relatively lower specificity (0.87), and F-1 score (0.79). Further, for the high motion subset, marginal improvements were achieved in the metrics with the same specificity as the original data.

Discussion

In this study, we demonstrated that our recently proposed novel deep learning-based motion correction technique improves the discrimination between benign and malignant thyroid nodules. This improvement is achieved by deriving more accurate quantitative biomarkers from motion-corrected HDMI data. We observed a substantial increase in the mean inter-frame correlation (MIFC) of the spatio-temporal matrices following motion correction, which resulted in improved quality of HDMI images. Further, there was a considerable increase in the number of HDMI images with higher MIFC values, further supporting the effectiveness of our approach. Additionally, we identified distinct changes in the qHDMI biomarkers following motion correction. We identified different trends in the qHDMI biomarkers after motion correction. The values of the biomarkers NV, NB, VD, mvFD, MD and Dmax further decreased for benign nodules and increased for malignant nodules after motion correction, making them statistically more significant. This trend aligns with the expectation that malignant nodules typically exhibit a higher number of vessel segments and branch points, along with denser vasculature, greater fractal dimension, and higher Murray’s deviation1,14. Although further testing is needed across larger and diverse datasets, these findings still indicate the adeptness of the deep learning-based motion correction framework in retrieving the microvasculature image quality. Further, we noticed that the bifurcation angle was significantly higher for malignant nodules before motion correction, which was unusual since it is expected that malignant nodules usually exhibit lower bifurcation angles14,27,28. This could be attributed to the presence of motion. However, we observed that it decreased for malignant nodules and increased for benign nodules after motion correction, lowering the statistical significance pre-correction. We also observed from the permutation importance analysis that after motion correction, bifurcation angle became the most important biomarker for affecting the model’s discriminatory performance. Another finding suggested that the vessel density ratio significantly affected the model performance before as well as after motion correction, although the p-value was not significant in both cases. This aligns with the fact that VDR can distinguish benign and malignant nodules29. The number of vessels did not affect the model before correction, rather, its negative permutation importance value indicated that it is merely noisy data for the model, however, its permutation importance increased to 0 after correction, indicating the capability of motion correction framework. This analysis also demonstrated that lower p-values do not necessarily imply the discriminatory nature of the biomarkers for the model. The classification performance metrics showed a substantial improvement after motion correction for the high motion subset as compared to that for the full data. This was expected since the amount of motion was more prior to correction in that subset which led to degradation in the microvasculature and possible misclassification. The motion correction framework was able to improve the sensitivity of the classification model notably, more than the specificity which was almost comparable to that before correction. We further examined that the AUC difference between the models trained with motion-containing and corrected data was apparent with the confidence interval becoming narrower with higher values for motion corrected data. Although DeLong’s estimate of the p-value was slightly over the standard threshold, the higher AUC confidence interval values after motion correction were noticeable. We observed that for cases with low motion, the classification performance was similar after motion correction. Importantly, these results demonstrate that motion correction enhances classification accuracy in dataset with higher level of motion, while not adversely affecting performance in dataset with negligible motion. This is evident as data with lower initial correlation typically associated with higher motion show greater improvements after correction. In contrast, data with initial higher correlation, reflecting lower motion, exhibit minimal improvement. Nevertheless, our novel deep leaning-based motion correction tool performs effectively even on low-motion data, maintaining or slightly improving results without degradation. In comparison, the existing two-stage image registration method provides only a miniscule improvement in classification performance and, in fact, degrades the results on low-motion containing datasets. Furthermore, the comparative analysis of time complexity of our proposed framework with the existing two-stage method demonstrated a remarkable reduction in the execution time to predict a motion corrected frame from 75 s to 15 ms12. Finally, the comparative analysis with the two-stage method and a few cutting-edge deep learning techniques demonstrated the superiority of our novel motion correction framework in terms of computational complexity and performance12.

The primary aim of this study was to validate our recently proposed inter-frame motion correction framework for effectively extracting the qHDMI biomarkers to classify benign and malignant thyroid nodules using HDMI data. However, the current dataset is relatively small to comprehensively assess our framework in all respects. Therefore, in future work, we plan to test the model on a larger and more diverse dataset to ensure that the proposed motion correction framework can handle variations across different patient populations and imaging conditions. Another future study may include the validation of our framework for correcting motion in three-dimensional HDMI data.

Conclusion

This study presents a rigorous analysis of thyroid nodule classification performance using quantitative HDMI biomarkers derived from motion-containing and motion-corrected power Doppler images. A comprehensive evaluation demonstrates the effectiveness of the deep-learning-based motion correction framework in enhancing quantification of qHDMI biomarkers. Finally, the results highlight the potential of deep learning-based motion correction methodology for improving thyroid nodule classification toward more accurate cancer diagnosis.