Introduction

Breast cancer has become a cancer disease that imposes a global public burden. However, breast cancer is often found to have progressed to the middle or advanced stages, thereby losing the opportunity for surgical intervention. Neoadjuvant therapy (NAT) has become a standard conversion treatment strategy for middle and advanced breast cancer, aiming to reduce tumor staging, shrink tumor volume, alleviate lymph node metastasis, and create surgical opportunities for patients with middle and advanced stages1. Pathologic complete response (pCR) is the desired goal of NAT. Patients who achieve pCR can adopt breast-conserving strategies or even avoid surgical intervention. pCR is based on the pathological tissue of the tumor after surgery, which is evaluated by immunohistochemical techniques, including the Miller-Payne (MP) grading system and the residual cancer burden (RCB) grading system, to visually reflect the true changes in the lesions compared with the puncture tissue pathology2,3. However, some studies had shown that about 37% of breast cancer patients could not achieve the desired goal through NAT, and about 5% of them would therefore increase treatment costs or suffer adverse reactions4. Accurately identifying breast cancer patient populations that can benefit from NAT is currently a hot clinical challenge to avoid excessive NAT.

The imaging biomarker features of medical images have been proven to be correlated with NAT tumor response. For instance, changes in background parenchymal enhancement observed in breast MRI before and after NAT revealed that the pCR group exhibited significantly greater changes compared to the non-pCR group5; the ultrasound models developed based on posterior acoustic shadowing, margin, and calcification features of breast lesions predicted pCR, achieving an area under the curve (AUC) of 0.769, with sensitivity (SEN) and specificity (SPE) of 0.83 and 0.57, respectively6. Another model utilizing ultrasound size, posterior acoustic shadowing, and shape features predicted pCR, yielding an AUC of 0.758, SEN of 0.67, and SPE of 0.787. Furthermore, the diagnostic performance of predicting NAT-pCR using the rising time parameter obtained from contrast-enhanced ultrasound imaging was good, with an AUC reaching 0.718. Nevertheless, the predictive accuracy of such methods only attains moderate effectiveness, prompting the need to explore imaging biomarkers with higher utilization value. Deep learning (DL), an artificial intelligence (AI) technology capable of extracting imaging features representative of tumor heterogeneity from medical images, holds potential for achieving precision diagnosis and treatment9. Convolutional neural network (CNN) DL algorithms include ResNet50 and VGG1610,11. For example, the ResNet50 model could identify breast cancer patients who were likely to achieve pCR from NAT among 603 multi-center breast cancer ultrasound images, achieving an AUC of 0.8812. The pCR prediction models developed using VGG16 features from pathological images based on support vector machine demonstrated optimal performance, with an AUC of 0.7913. Recently, vision transformer (VIT) has become a more valuable DL algorithm for image utilization, which has been widely applied in image feature extraction and prediction tasks. For example, the predictive model developed using VIT algorithm based on CT images had been utilized to assess postoperative recurrence of lung cancer, achieving an AUC of 0.9014. Compared to CNN algorithms, VIT employs an attention mechanism to increase the weight of important image parts, potentially enabling better prediction of breast NAT-pCR. For instance, when VIT and seven CNN algorithms were used to develop pCR prediction models based on CT images, VIT features performed best in both training and validation cohorts, while VGG16 features performed the worst15. Although VIT has begun to be applied in breast cancer NAT-pCR prediction, studies based on ultrasound images remains scarce. Moreover, the current comparisons of various deep learning features often rely on a single machine learning algorithm to develop models, raising questions about whether the chosen algorithm is optimal and whether this may lead to biased comparison results. Cross-validation methods for feature selection and model development using various machine learning algorithms have been widely applied in exploring the applicability of models in the past, for instance, 12 machine learning algorithms demonstrated good performance in developing multi-combination models for predicting lymph node metastasis in cholangiocarcinoma based on radiomics features.

Based on the above background, the aim of this study is to explore the value of 113 multi-combination models generated by 12 machine learning algorithms, using ultrasound DL features extracted by VIT, ResNet50, and VGG16 algorithms, in predicting cancer NAT-pCR response. In addition, the clinical significance of VIT features was interpreted based on radiomics features.

Materials and methods

Ethical approval

The Ethics Committee of Maoming People’s Hospital waived the need for obtaining informed consent because the study was retrospective. This retrospective study had obtained ethical approval from the Ethics Review Committee of Maoming People’s Hospital (PJ2024MI-K040-01). All analyses were conducted in accordance with relevant guidelines and regulations, including the Helsinki Declaration.

Breast cancer patients undergoing NAT

This study initially recruited 501 patients with breast space-occupying lesions who underwent ultrasound-guided puncture histopathology biopsy at Maoming People’s Hospital from January 2020 to December 2023. Inclusion Criteria: (1) Patients with primary breast space-occupying lesions classified as the Breast Imaging Reporting and Data System (BI-RADS) category 4 or 5 based on pre-biopsy ultrasonography. (2) Patients confirmed as breast cancer by puncture histopathology. (3) Patients who received NAT to obtain surgical opportunities, including chemotherapy, endocrine therapy, and targeted therapy. (4) Patients who underwent surgical resection after NAT to assess the tumor response. Exclusion Criteria: (1) Patients with bilateral lesions or multiple unilateral lesions. (2) Lesions that were too large to be fully captured or with suboptimal image quality for analysis. (3) Tumor response assessment not performed using the Miller & Payne (MP) grading system. (4) Patients with missing follow-up data or severing lack of clinical information (Fig. 1).

The clinical pathological data were sourced from the medical record system, encompassing age, menopausal status, lesion size, BI-RADS classification, white blood cell (WBC), hemoglobin count, fibrinogen, platelet distribution width (PDW), neutrophil-to-lymphocyte ratio (NLR), platelet-to-lymphocyte ratio (PLR), systemic immune inflammation index (SII), pan-immune inflammation value (PIV), human epidermal growth factor receptor 2 (HER2), Ki67 expression41. A total of 124 patients with breast disease were finally included, aged from 32 to 82 years old, with an average age of (52.55 ± 9.41) years old. Among them, there were 23 cases of BI-RADS 4a/b, 51 cases of 4c, and 50 cases of 5. The overall cohort was randomly split into a training cohort (n = 87) and a validation cohort (n = 37) at a ratio of 7:3.

Fig. 1
Fig. 1The alternative text for this image may have been generated using AI.
Full size image

Flow chart of study cohort.

Tumor response assessment

Tumor response to NAT in breast cancer, assessed by the MP system, was graded as follows: Grade 1 (no change), Grade 2 (≤ 30% reduction), Grade 3 (30-90% reduction), Grade 4 (> 90% reduction with scattered cells), and Grade 5 (no residual invasive cancer, with intraductal carcinoma remnants)42. Grades 1–3 were classified as non-pCR, while Grades 4–5 were considered as pCR.

Ultrasound image acquisition and deep learning feature analysis

The ultrasound physicians involved in image acquisition all possessed 3 years of experience in diagnosing breast lesions, ensuring that the captured images were the largest and clearest representations of the lesions. The BI-RADS classification was independently assigned by an ultrasound physician with 5 years of experience in diagnosing breast lesions, ensuring the accuracy and reliability of the diagnostic outcomes. The image acquisition process was facilitated by high-end ultrasound diagnostic systems, including Mindray Resona7, GE LOGIQ E9/10, and Philips EPIC7C, equipped with high-frequency linear array transducers (L11-3U, ML9L, L12-5, L12-3) operating at frequencies ranging from 5 to 13 MHz. Appropriate machine parameters, such as gain, depth, acoustic window, mechanical index, and focal zone, were meticulously adjusted to optimize image quality. The acquired images of breast space-occupying lesions were saved in the Digital Imaging and Communications in Medicine (DICOM) format and subsequently loaded into the ITK-SNAP (3.80 version) software for further processing. An augmentation strategy involving random horizontal and vertical flipping was applied to the original ultrasound images to enhance their diversity. The regions of interest (ROIs) for breast space-occupying lesions were manually segmented in collaboration by two ultrasound physicians, each with 5 years of experience in breast diagnosis, using the ITK-SNAP software. The ComBat method was applied to calibrate for the variabilities in radiomic feature changes caused by differences in ultrasound equipment43. This was achieved by utilizing the PyRadiomics package to extract 1316 radiomics features from both the original ultrasound images and their corresponding transformed ROIs images, including: shape features, first-order statistical features, gray-level co-occurrence matrix (GLCM) features, gray-level run length matrix (GLRLM) features, gray-level size zone matrix (GLSZM) features, gray-level dependence matrix (GLDM) features, and neighboring gray tone difference matrix (NGTDM)44. With the ROI as the centerpiece, the smallest bounding rectangle encompassing the ROI’s information was extracted and resized to 224 × 224 pixels, format was “PNG”, serving as the input for the deep learning model. Supervised DL models, pre-trained on the ImageNet dataset, had been frequently leveraged for medical image analysis45. Consequently, from each ROIs image, deep learning features were extracted utilizing various deep learning models, including VIT, ResNet50, and VGG16, all of which were initialized with ImageNet pre-trained model parameters46. Given that deep learning harnesses image information at a scale of tens of thousands of dimensions, a compression process was applied to the image information, ultimately yielding 108 deep learning features47. These features encapsulate the essential information from the ROIs images, facilitating their utilization in downstream medical image analysis tasks (Fig. 2).

Fig. 2
Fig. 2The alternative text for this image may have been generated using AI.
Full size image

Image feature extraction and analysis.

Development and validation of multi-combination machine learning models

The Z-score method initially standardized the overall cohort. The Wilcoxon test was then employed to identify high-value features that were indicative of favorable NAT-pCR. Subsequently, 12 distinct algorithms were leveraged to develop machine learning prediction models based on these high-value features, encompassing least absolute shrinkage and selection operator (LASSO), Ridge, elastic net (Enet), Stepglm, support vector machine (SVM), glmBoost, linear discriminant analysis (LDA), plsRglm, random forest (RF), gradient boosting machine (GBM), XGBoost, and naive bayes (NB)37. Within a cross-validation framework, the strategy was adopted wherein one algorithm was used for feature selection while another 11 algorithms were deployed to construct the classification prediction model, resulting in a maximum of 113 algorithm combinations. The best algorithms developed based on deep learning features were used to construct the radiomics machine learning models. The interpretation of clinical significance of deep learning features was based on correlated radiomics features. The receiver operating characteristic curve (ROC) was applied to evaluate the performance of the models, and AUC, SEN, and SPE were calculated. Calibration curve analysis and Hosmer-Lemeshow (HL) test were used to evaluate models calibration and goodness of fit. Decision Curve Analysis (DCA) was used to evaluate the net clinical benefit or utility of models.

Literature review on artificial intelligence based on ultrasound or pathology imaging

Before November 10, 2024, the literature was retrieved based on the PubMed database with the keyword “breast and neoadjuvant and (radiomics or deep learning)”. The literature on tumor response assessment of NAT for breast cancer was comprehensively reviewed and sorted out in terms of deep learning study types. The main author, publication year, study population, tumor response standards, model type, AUC, SEN, SPE were extracted according to the full text of the literature. In addition, the literature based on pathological image analysis was also summarized and analyzed. A meta-analysis was conducted based on the Stata (version 12.0) software.

Statistical analysis

SPSS (17.0 version) and R (3.68 version) software were used for statistical analysis and plotting. Continuous variables were represented by mean values ± standard deviations, and t-test was used for comparison; categorical variables were represented by examples (%), and Chi square test was used for comparison. P < 0.05 was considered statistically significant.

Results

The baseline characteristics of breast cancer cohorts

44 cases (50.57%) in the training cohort showed pCR, while 19 cases (51.35%) in the validation cohort demonstrated non-pCR. The baseline characteristics of the two cohorts were summarized in Table 1. No significant differences in baseline characteristics were observed between the two cohorts.

Table 1 The baseline characteristics of breast cancer cohorts.
Full size table

Development and validation of multi-combination machine learning models

Among the overall cohort, 9, 7, and 7 high-value features were identified from 108 features extracted by the VIT, Resnet50, and VGG16 deep learning algorithms, respectively, based on the Wilcoxon test (Fig. 3). These features were then input into 113 combinations of algorithms, and predictive models consisting of 3 or more than 3 features were selected, resulting in 111, 87, and 82 models constructed from VIT, Resnet50, and VGG16 features, respectively (Fig. 4). The AUC of the validation cohort was used as the criterion for selecting the optimal model. For VIT, Resnet50, and VGG16 features, the highest AUCs were achieved in the Stepglm [forward], NB, and glmBoost + Ridge combination algorithms, respectively. The AUCs of the optimal algorithms in the training and validation cohorts were 0.872 and 0.839 for VIT, 0.837 and 0.804 for Resnet50, and 0.835 and 0.804 for VGG16. The HL test demonstrated a good consistency between the best algorithm models and pathological outcomes (VIT_Stepglm [forward], training cohort, P = 0.45, validation cohort, P = 0.61; Resnet50_NaiveBayes, training cohort, P = 0.44, validation cohort, P = 0.88; VGG16_glmBoost + Ridge, training cohort, P = 0.50, validation cohort, P = 0.27). DCA demonstrated that the VIT_Stepglm [forward] models possessed good potential clinical application value (Fig. 5). Among the three deep learning features, VIT features exhibited the best predictive performance, while Resnet50 and VGG16 features showed similar predictive effects. The Stepglm [forward] machine learning algorithm based on VIT features performed the best (AUC = 0.839), followed by VGG16 features (AUC = 0.787), and Resnet50 features performed the worst (AUC = 0.754). For the NB algorithm, Resnet50 features resulted in the best performance (AUC = 0.804), followed by VGG16 features (AUC = 0.678), and VIT features performed the worst (AUC = 0.667). Ridge algorithm achieved the best performance with VGG16 features (AUC = 0.804), followed by VIT features (AUC = 0.778), and Resnet50 features yielded the third-highest AUC of 0.772. Both the Stepglm (direction = forward) and Ridge algorithms demonstrated moderate predictive performance when applied to the quantitative features extracted from the three deep learning algorithms. Various other machine learning models also showed moderate performance in predicting favorable tumor responses to NAT for breast cancer, indicating the predictive stability of these models. Table 2 summarized the optimal combination algorithms for different machine learning algorithms.

Table 2 Development and validation of multi-combination machine learning models.
Full size table
Fig. 3
Fig. 3The alternative text for this image may have been generated using AI.
Full size image

Identification of high-value VIT features related to pCR. (A) Feature utilization rates across 111 machine learning models based on VIT features. (B) Heatmap of correlations among 9 VIT high-value features.

Fig. 4
Fig. 4The alternative text for this image may have been generated using AI.
Full size image

The pCR prediction model employed the 111 combination machine learning models based on VIT features. VIT_Stepglm [forward] exhibiting the best performance, while AUCs in the training and validation cohorts were 0.872 and 0.839, respectively.

Fig. 5
Fig. 5The alternative text for this image may have been generated using AI.
Full size image

The optimal algorithm combination results of VIT, ResNet50, and VGG16 features. For VIT, Resnet50, and VGG16 features in the training cohort (A) and validation cohort (B), the highest AUCs were achieved in the Stepglm [forward], NB, and glmBoost+Ridge combination algorithms, respectively. In the training cohort (C) and validation cohort (D), DCA demonstrated that the VIT_Stepglm [forward] models possessed good potential clinical application value.

Interpreting potential clinical significance represented by deep learning features based on radiomics

Twenty-six high-value features were identified from 1316 radiomics features based on the Wilcoxon test. Predictive models were developed using the optimal algorithms of lasso + Stepglm [forward], NB, and Ridge. The AUC, SEN, and SPE of the training cohort and validation cohort were 0.763, 0.80, 0.67 and 0.626, 1.00, 0.39; 0.694, 0.41, 0.95 and 0.722, 0.95, 0.56; 0.697, 0.75, 0.60 and 0.722, 0.89, 0.56, respectively (Fig. 6A and 6B). The correlation between the nine VIT deep learning features and the 26 radiomics features was presented in Fig. 6C. The results showed that 7 radiomics features were correlated with deep learning features, with correlation coefficients ranging from − 0.19 to 0.21. Among them, there were three GCLM features, two GLSZM features, one GLRLM feature, and one GLDM feature, representing the heterogeneity and complexity of texture distribution within ultrasound images in breast cancer.

Fig. 6
Fig. 6The alternative text for this image may have been generated using AI.
Full size image

Interpreting potential clinical significance represented by deep learning features based on radiomics. The predictive models developed based on 26 high-value radiomics features, Ridge algorithm performed the best, while AUCs in the training and validation cohorts were 0.697 (A) and 0.722 (B), respectively. (C) Correlation analysis between 9 VIT features and 26 radiomics features. It was discovered that the DL_103 feature correlated with GLCM (r=0.21) and GLSZM (r=0.20) features, indicating its representation of the complexity of lesion texture distribution, which aligned with the enhanced image weighting by the attention mechanism of VIT.

Literature review on artificial intelligence based on ultrasound or pathology imaging

Among the 276 literature retrieved, 14 were deep learning literature and 7 were radiomics literature based on ultrasound imaging related to NA therapy for breast cancer, 73,4,12,17,18,19,20 and 517,21,22,23,27 of which were enrolled meta analysis, respectively. 8 literature on deep learning based on pathology imaging, of which 513,24,25,26,27 were included in the meta analysis (Table 3). The meta analysis revealed that the pooled AUC, SEN, and SPE of the deep learning ultrasound models were 0.91 (0.89–0.93), 0.86 (0.83–0.89), and 0.84 (0.78–0.88), respectively. In contrast, the pooled AUC, SEN, and SPE of the radiomics ultrasound models stood at 0.81 (0.78–0.85), 0.78 (0.68–0.85), and 0.72 (0.62–0.80), respectively. Interestingly, the deep learning pathological models achieved the pooled AUC of 0.81 (0.77–0.84), a SEN of 0.68 (0.59–0.77), and a SPE of 0.85 (0.72–0.92), indicating that the ultrasound models seemed to perform better in predicting breast cancer NAT tumor response compared to the pathological model (Fig. 7).

Table 3 Literature review on artificial intelligence based on ultrasound or pathology imaging.
Full size table
Fig. 7
Fig. 7The alternative text for this image may have been generated using AI.
Full size image

Literature review on artificial intelligence based on ultrasound or pathology imaging. The meta-analysis results showed that DL features based on ultrasound images (AUC=0.91) were superior to radiomics features based on ultrasound images (AUC=0.81) and DL features based on pathological images (AUC=0.81).

Discussion

Accurately identifying the population of breast cancer patients who benefit from NAT is conducive to minimizing unnecessary over-treatment. DL features have demonstrated high value in predicting NAT-pCR responses. This study aimed to explore the value of 113 multi-combination models generated by 12 machine learning algorithms, using ultrasound features extracted by VIT, ResNet50, and VGG16, in predicting breast cancer NAT-pCR responses. The results indicated that the predictive model developed based on VIT features had a higher value in evaluating the efficacy of NAT for breast cancer compared to features quantified by other deep learning methods. The application of multi-combination models allowed for the selection of the optimal algorithm to achieve higher prediction performance.

Literature review on deep learning in predicting NAT-pCR for breast cancer

AI technology serves as a groundbreaking force in transcending traditional image vision, enabling high-throughput quantification and enhanced utilization of heterogeneous imaging features in medical images, encompassing MRI, CT, ultrasound, and pathological images. High-throughput imaging features have been widely applied in predicting NAT-pCR in breast cancer. MRI-derived radiomics features had been extensively validated by numerous meta-analyses to exhibit favorable performance in NAT-pCR prediction, with pooled AUC values ranging from 0.78 to 0.8528,29,30,31. Meanwhile, MRI-derived DL features had shown even more promising results, achieving a pooled AUC of 0.92 through meta-analyses. Similarly, ultrasound-derived radiomics features, when subjected to a meta-analysis encompassing eight studies, exhibited a pooled AUC, SEN, and SPE of 0.86, 0.87, and 0.78, respectively32. However, the application value of ultrasound-based DL features in predicting NAT-pCR remained under-explored. The PubMed search uncovered 14 studies pertaining to the use of DL features in predicting NAT outcomes, with eight of these studies collectively demonstrating a pooled AUC of 0.91, SEN of 0.87, and SPE of 0.83, mirroring the outcomes observed with MRI-DL features and outperforming both MRI and ultrasound radiomics features, surpassing the meta-analysis of the four ultrasound studies included in this study. Furthermore, despite the potential of pathological image-based DL features, a comprehensive analysis of their utility has yet to be conducted. The search identified eight studies on pathological image-related DL features, of which five reported a pooled AUC, SEN, and SPE of 0.80, 0.68, and 0.85, respectively, suggesting room for improvement. This subpar performance could be attributed to the inherent complexity of pathological images and the maturity level of AI technologies employed. Notably, VIT features had already been explored for NAT-pCR prediction in CT and pathological images15,24, underscoring their potential. Nevertheless, the application of VIT features to ultrasound images for this purpose remained an area ripe for further investigation.

Multi-combination machine learning models for predicting NAT-pCR

The utilization of DL feature extraction followed by their presentation to machine learning algorithms for developing predictive models has been widely embraced in medical imaging. Nevertheless, the question of which machine learning algorithm best suits DL features remains an area requiring further investigation. For instance, when developing predictive models using DL features extracted from chest CT images via algorithms such as ResNet50, ResNet101, AlexNet, VGG16, VGG19, GoogLeNet, SqueezeNet, Xception, and presenting these features to machine learning algorithms including SVM, k-nearest neighbors, RF, decision tree, and NB, it was observed that the combination of ResNet50 features and SVM machine learning algorithm yielded the best results, achieving an accuracy of 0.96333. Similarly, in another study where DL features were extracted from chest CT images using ResNet18 and GoogleNet, and presented to RF, SVM, fast decision tree, and NB algorithms for predictive model development, it was found that the ResNet18 features combined with SVM also demonstrated optimal performance34. This suggested that SVM algorithms may be more compatible with ResNet neural network algorithms. However, when considering radiomics features extracted from chest CT images, a multi-center study indicated that RF performed better, whereas a single-center study found SVM to be superior, among logical regression, RF, and SVM algorithms35,36. The emergence of these results is also correlated with feature selection strategies. Thus, whether developing several models can truly represent the best approach for assessing feature suitability is a matter of debate. Against this backdrop, the application of multi-combination models has gradually been explored for its potential value. For example, in predicting pancreatic lymph node metastasis, 77 ensemble models were developed based on 10 algorithms, with the Stepglm [backward] algorithm performing the best, achieving an AUC of 0.8537. Similarly, in predicting gastric cancer prognosis, 101 mitochondrial-related scoring models were developed based on 12 genes, with the Cox + random survival forest combination yielding the best performance38.

Against this backdrop, the present study, based on single-center data, first explored the value of VIT features and CNN features in predicting breast cancer NAT-pCR. It was found that VIT features performed better, achieving an AUC of 0.839. Notably, the optimal machine learning approaches for VIT, ResNet50, and VGG16 features differed, being Stepglm [forward], NB, and glmBoost + Ridge, respectively. Encouragingly, the Stepglm [forward] model based on CNN features achieved moderate performance in predicting NAT-pCR in the validation cohort, with AUCs of 0.754 and 0.787, respectively. After reviewing numerous similar publications, it was discovered that the Stepglm algorithm actually performed exceptionally well in many feature selection and prediction tasks, which may be attributed to the algorithm itself. In numerous studies that employed various algorithms for feature selection and model development, for instance, the plsRglm prediction model developed subsequent to feature selection by the Stepglm algorithm was found to be superior in predicting endometriosis compared to other algorithms39. Furthermore, multicenter data indicated that the Stepglm model developed after feature selection by the LASSO algorithm was more advantageous in predicting drug-resistant epilepsy40. Despite the variation in the most suitable machine learning algorithms for different DL features, there exist algorithms that could excel across multiple DL feature sets. From a feature utilization perspective, the DL_103 feature was utilized a total of 111 times in model development. Through correlation analysis of radiomics features, it was discovered that the DL_103 feature correlated with GLCM (r = 0.21) and GLSZM (r = 0.20) features, indicating its representation of the complexity of lesion texture distribution, which aligned with the enhanced image weighting by the attention mechanism of VIT. Additionally, when the aforementioned top three machine learning algorithms were employed to develop predictive models based on radiomics features, it was found that the Ridge algorithm performed the best, followed by NB, with Stepglm [forward] performing the worst. However, even the optimal Ridge model (AUC = 0.722) was inferior to several dozen ensemble models based on DL features.

Limitations

Despite the relatively good performance of VIT features in predicting NAT-pCR compared to other CNN algorithms, this study still had certain limitations. Firstly, it was a retrospective study with a small sample size from a single center, and further validation with multi-center large-scale data will be needed to confirm the clinical applicability and robustness of the models. Secondly, the resolution of ultrasound imaging was limited by individual patient differences, such as breast density type and adipose thickness, as well as the acquisition parameters set by physicians. Although the Combat method was used to remove inter-instrument differences, it was still difficult to assess whether different parameters would affect the effectiveness of models. Thirdly, VIT features were extracted based on pre-trained model parameters, which still required large-scale data for model training and VIT feature extraction. Finally, while attempts had been made to interpret the clinical significance of VIT features based on radiomics features, causal relationships still need to be validated through molecular-level analysis.

Conclusions

In conclusion, most of the machine learning models developed based on VIT features extracted from ultrasound images have shown satisfactory performance. Compared to other CNN features, VIT features exhibit superior performance. However, different machine learning models utilize distinct DL features, resulting in variations in their effectiveness. Consequently, the application of multi-combined models can potentially lead to the discovery of more suitable prediction models. While VIT features hold potential application value, their clinical interpretability still requires further exploration at the molecular level.