Introduction

Several trials have established the safety and efficacy of endovascular therapy (EVT) in treating large vessel occlusion stroke in up to 24 h from symptom onset1,2,3,4. Accurate estimate of ischemic core volume (ICV) and penumbra volume (PV) is important in decision-making for EVT and predicting the patient’s clinical outcome5. However, prompt and quantitative evaluation of ICV and PV increasingly requires the support of automated artificial intelligence (AI) software6, aiding clinicians in rapid decision-making for precise diagnosis and treatment7.

The UGuard® Intelligent Stroke Imaging Diagnosis System (Version1.6, Qianglianzhichuang Technology, Beijing) software is a novel fully automated post-processing software that runs on a NVIDIA GPU server. UGuard® comprises a fully automated platform (registration, segmentation, and motion correction) that utilizes a machine learning algorithm for quantification of ICV and PV on CT perfusion (CTP) images.

In this study, we aimed to examine the agreement and correlation of ICV and PV measured by UGuard and RAPID, an FDA approved and widely validated CTP-processing software8,9, and assess the predictive value of UGuard measurements regarding favorable outcomes in patients receiving EVT.

Methods

Patients

Consecutive AIS patients receiving EVT treatment between January 2020 and September 2022 were retrospectively recruited from Shenyang First People’s Hospital and Zhangzhou Affiliated Hospital of Fujian Medical University. All patients underwent CTP examination before treatment. CTP images were post-processed by RAPID and UGuard softwares. All patients were eligible for enrolment if they were (1) age ≥ 18 years; (2) clinically diagnosed with AIS caused by large vessel occlusion (LVO) of the anterior circulation, with or without ipsilateral ICA occlusion.

Patients were excluded from the study if there was (1) inadequate quality of imaging for analysis, (2) hemorrhage or suspected hemorrhage on neuroimaging, (3) pre-morbid modified Rankin Scale (mRS) score ≥ 2, (4) inadequate clinical data or, (5) patients transferred to other hospitals or died within 24 h after admission.

Ethical approval and consent to participate

This study was approved by the Ethics Committees of Shenyang First People’s Hospital Medical Ethics Committee (Shenyang, China) and Zhangzhou Affiliated Hospital of Fujian Medical University Clinical Trial Ethics Committee (Zhangzhou, China). The need for informed consent was waived by the Ethics Committees due to the retrospective nature of the study. All patient-related information was de-identified in statistical analysis and did not involve any additional intervention or direct patient contact. All data were handled in strict compliance with privacy protection laws and institutional guidelines.

Clinical assessment

Clinical profiles of the patients included age, gender and medical history (hypertension, diabetes, heart disease, hyperlipidemia, smoking and drinking history). Stroke severity and patient outcomes were assessed by certified raters using the National Institutes of Health Stroke Scale (NIHSS) and the modified Rankin Scale (mRS) scores, respectively. The NIHSS assessment included the scores at admission, 24 h after EVT and at the time of discharge. The mRS scale included the scores at admission prior to EVT and 90 days after EVT (0 - 2 for favorable clinical outcome and 3 - 6 for unfavorable outcome)8,9. Stroke etiology was determined with the Trial of ORG 10,172 in the Acute Stroke Treatment (TOAST) classification. Recanalization status was defined by the modified Thrombolysis in Cerebral Infarction (mTICI) score with mTICI 2b - 3 considered as successful.

Imaging acquisition protocols

Shenyang First People’s Hospital was equipped with a 200-mm wide detector (SOMATOM Definition AS, Siemens Healthcare, Forchheim, Germany). The scanning parameters were as follows: slice thickness 5 mm, total dose-length product 2886.8 mGy*cm, tube voltage 80 kV, tube current 400 mAs, matrix 512 × 512, total scan time 55.62 s, and detector collimation 128 × 0.625 mm. Contrast was administered intravenously with 70 ml iodixanol injection (Visipaque 100 ml:32 g(I) GE Healthcare Ireland) at an injection rate of 5 ml/s.

Zhangzhou Affiliated Hospital of Fujian Medical University was equipped with a Revolution CT scanner (GE Healthcare, Waukesha, WI, USA) with a 160-mm wide detector. The scanning parameters were as follows: CT dose index volumes ranging 64.11 mGy, tube voltage 80 kV, tube current 120 mAs, total scan time 60s, detector collimation 256 × 0.625 mm, gantry rotation speed 1s, field of view (FOV) 220 mm × 220 mm. The CTP protocol involved acquisition of 25 scans, each with 32 images with 5-cm resolution. Reconstructed CTP volumes were provided within 5 min from the start of scanning. The contrast was 30 – 35mL of iohexol (Iomeron 50 ml:20 g(I) Bracco Imaging Italia S.r.L.) at a 5 – 6 mL/s injection rate.

Pre- and post-processing of computed tomographic perfusion images

RAPID (Version 7.0, iSchemaView, Menlo Park, CA, United States, https://www.rapidai.com/) is a fully automated processing software using a delay-insensitive algorithm. RAPID uses a deconvolution algorithm to construct quantitative perfusion maps such as cerebral blood flow (CBF), cerebral blood volume (CBV), mean transit time (MTT) and time-to-max (Tmax). RAPID defined the regions with rCBF < 30% as ICV and Tmax > 6s regions as PV.

UGuard®Stroke Intelligent Stroke Imaging Diagnosis System (Version1.6, Qianglian Zhichuang (Beijing) Technology) software automatically identifies locations at the input artery and output vein. It uses the delay insensitive Block-Circulant SVD deconvolution algorithm to calculate several perfusion function parameters (CBV, CBF, Tmax, MTT, etc.). The software automatically extracts and compares voxels from the affected and contralateral brain tissue to calculate rCBF. Similar to RAPID, UGuard also sets the threshold values to rCBF < 30% and Tmax > 6s. Specifically, during the image preprocessing step, multiple deep convolutional networks were employed in UGuard to segment the cerebrospinal fluid (ventricles and cisterns), brain hemorrhage, and distinctive cerebral regions. This facilitates the removal of non-interest areas and the statistical analysis of perfusion parameter values in different cerebral regions. Due to variable image quality acquired at multiple centers, adaptive anisotropic filtering networks were used to remove the noise, which achieved a filtering performance comparable to traditional BM3D filtering algorithm while in a reduced computation time.

In order to automatically locate the artery and vein from images, we trained a deep convolutional model for artery and vein segmentation using maximum intensity projection images of perfusion sequence data as input, through which basilar artery (BA), internal carotid artery (ICA), middle artery segment 1 (M1), middle artery segment 1 (M2), anterior artery segment 1 (A1), anterior artery segment 2 (A2) and cerebral venous sinuses were segmented. Curve scoring was then used to find the most suitable points on the arterial and venous segments, for which amplitude, time to peak, curve width, and curve shape were primarily considered in calculating the curve score.

Statistical analysis

Continuous variables were described as mean ± SD if data were normally distributed, or medians ± interquartile range if the data had a non-normal distribution. Categorical variables are presented as n (%). Categorical variables were examined using the chi-square test, while continuous variables were checked using the independent t test or the Mann-Whitney Utest. Two distinct CTP measurements were compared using the Wilcoxon signed rank test. The Spearman correlation coefficient, ICC and Bland-Altman analysis were used to determine the correlation and agreement between CTP measurements. The agreement categories included poor (ICC < 0.5), moderate (ICC 0.5 - 0.75), good (ICC 0.75 - 0.9) and strong reliability (ICC > 0.9)10. Univariable logistic regression and penalized logistic regression (PLR) were performed between CTP measurements and favorable clinical prognosis, and odds ratio (OR) and 95% confidence intervals (CIs) were calculated. The area under the curve (AUC) measured by CTP measurements was calculated using the receiver operating characteristic curve (ROC) and the DeLong test was performed to compare the predictive abilities of RAPID versus UGuard measurements11. The above-mentioned model with the maximum Youden Index was used to determine the sensitivity and specificity. The categories of subgroups are classified according to the results of LASSO regression and different clinical centers. Subgroup results presented as forest plots, with P values for interaction for each pair of subgroups. All statistical differences were based on a two-sided test and p < 0.05 was defined as significant. IBM SPSS Statistics 24.0 (SPSS Inc an IBM Company, Chicago, IL, United States, https://www.ibm.com/products/spss-statistics) and R (R Core Team v4.1.3; R Foundation for Statistical Computing, Vienna, Austria, https://www.r-project.org/) were used for statistical analysis.

Results

Clinical characteristics

A total of 175 patients were enrolled for this study. After excluding 9 patients with motion artifact on the images, 6 patients who were lost to follow-up and 1 patient who did not have a preoperative CTP examination after clinical deterioration, 159 patients who met the inclusion criteria were included in the study. Table 1 summarizes the baseline characteristics of the study population. Of these, 104 male and 55 female patients were included respectively, with a mean age of 66.4 ± 12.9 years old. Based on 90-day mRS scores, 78 patients were categorized in the favorable outcome group and 81 patients were in unfavorable outcome group. Patient comorbidities of atrial fibrillation (50.6% vs. 24.4%, P = 0.001) and coronary artery disease (14.8% vs. 5.1%, P = 0.042) were more frequent in the unfavorable outcome group than in the favorable group. The median (interquartile range, IQR) baseline NIHSS score higher in the unfavorable compared to favorable group (16 [12.0–20.5] vs. 10 [7.0–14.0], P < 0.001). There were 122 (76.7%) atherosclerotic stroke, 35 (22%) cardioembolic stroke and 2 (1.3%) undetermined stroke etiologies, and a greater proportion of patients with atherosclerotic stroke were in the favorable outcome group (P = 0.045). After treatment with EVT, successful recanalization was achieved in 144 (90.6%) patients (mTICI 2b, n = 27; mTICI 2c, n = 14; mTICI 3, n = 103). The final mTICI and clinical outcome were significantly correlated (P = 0.003).

Table 1 Baseline demographic and clinical characteristics (n = 159).
Full size table
Table 2 Univariable logistic regression between CTP measurements and favorable clinical outcome.
Full size table
Table 3 ROC analysis regarding favorable outcome.
Full size table
Table 4 Six multivariable models to predict favorable clinical outcome.
Full size table

Correlation and agreement of computed tomographic perfusion measurements between RAPID and UGuard

In estimating core infarct and penumbra volumes, UGuard and RAPID packages showed high-level agreement. RAPID software perfusion maps from three AIS patients are shown in Supp Fig. 1 with ischemic core (purple) and penumbra (light green), and UGuard perfusion maps with ischemic core (red) and penumbra (dark green). The ischemic core volumes were 3 mL, 26 mL and 73 mL respectively with UGuard compared to those of 0 mL, 24 mL and 76 mL with RAPID. Similarly, the penumbra volumes were 57 mL, 179 mL and 140 mL with UGuard compared to those of 57 mL, 181 mL and 145 mL with RAPID.

Fig. 1
figure 1

(A) Scatter plot of ischemic core volume measured by RAPID and UGuard. (B) Scatter plot of penumbra volume measured by RAPID and UGuard. The dotted oblique line represents the reference line (x = y) (A, B). (C) Bland-Altman plot of ischemic core volume. (D) Bland-Altman plot of penumbra core volume. Dotted lines represent 95% limits of agreement (C, D).

Full size image

The median (IQR) ischemic core volume measured by RAPID was 7 (0–23) mL, and that of UGuard was 12 (3–29) ml. A similar distribution can be seen according to Supp Fig. 2, but the Wilcoxon test showed that these values were significantly different (P = 0.003). The median penumbra volume measured by RAPID and UGuard were 132 (78–190) mL and 123 (81–174) mL, with no significant difference in the measurement (P = 0.078).

Fig. 2
figure 2

Comparison of ischemic core volume and penumbra volume measured by two software packages between patients with favorable and unfavorable clinical outcomes after endovascular thrombectomy (EVT).

Full size image

The Spearman correlation coefficient for ICV as measured by UGuard and RAPID was 0.82 (P < 0.001) and for PV measurement was 0.84 (P < 0.001), both of which suggest strong correlations (Fig. 1A, B).

The ICC between the ICVs measured by UGuard and RAPID was 0.92 (95% CI 0.89–0.94, P < 0.001), and ICC between the PV measurements of the two software packages was 0.8 (95% CI 0.73–0.85, P < 0.001). The Bland-Altman plot (Fig. 1C, D) shows the good to strong agreements of the two software packages’ measurements.

Prognostic value of baseline characteristics and computed tomographic perfusion measurements

The ICV and PV measured by two software packages were significantly greater in patients in the unfavorable outcome group than those with favorable outcome (P < 0.01), as shown in Fig. 2. For RAPID software, the median (IQR) of ICV in favorable outcome group was 0 (0–10.25) mL, while 13 (1.5–37) mL in unfavorable outcome patients. The median (IQR) of PVs were 111 (60.25–149) mL in favorable group and 168 (114–214) mL in unfavorable group. The result of UGuard ICV measurement of favorable outcome group was 6 (2–14.75) mL and 19 (7.5–46.5) mL in unfavorable outcome group. The PV of favorable outcome group was 105 (76–115.75) mL compared with 145 (102.5–239) mL in unfavorable group.

Univariable logistic regression analysis was used to analyze the relationship between RAPID and UGuard CTP measurements and the favorable outcome group. All of the four measured values of ICV and PV were significantly associated with decreased chance of favorable outcome (Table 2, all P < 0.05). ROC analyses were performed and AUC, 95% CI, Youden index, cutoff value, sensitivity, and specificity are shown in Table 3. ROC curve of subjects is shown in Fig. 3.

Fig. 3
figure 3

Receiver operating characteristic curves for (A) ischemic core volume measured by RAPID and UGuard (B) penumbra volume measured by RAPID and UGuard.

Full size image

The AUC of ICV for favorable outcome as measured by RAPID was 0.70 (0.62–0.78), with a cut-off value of 8.5 ml, a Youden index of 0.34, a sensitivity of 73% and a specificity of 61%. The AUC of ICV measured by UGuard was 0.72 (0.65–0.80) with a cut-off value of 11.5 ml, a Youden index of 0.37, a sensitivity of 67% and a specificity of 70%; The P value of the Delong test was 0.43, showing that the ICVs measured by the two packages had no difference in predicting favorable outcome.

The AUC of the PV measured by RAPID was 0.70 (0.62–0.78), with a cut-off value of 146.0 ml, a Youden index of 0.34, a sensitivity of 76% and a specificity of 58%. The AUC of the PV measured by UGuard was 0.65 (0.56–0.73), with a cut-off value of 123.5 ml, a Youden index of 0.32, a sensitivity of 67% and a specificity of 65%. The P value of the Delong test was 0.03, indicating that the PVs measured by RAPID had a better predictive performance than that measured by UGuard.

Multivariable models to predict favorable clinical outcome

Lasso regression with an alpha value of 1 (L1 regularization) was used to select the variables and remove component multicollinearity between variables. The optimal lambda value, determined through cross-validation, was 0.453. After using the lambda corresponding to the minimum standard error distancing from mean square error, gender, age, NIHSS score on admission, atrial fibrillation and mTICI were independent predictors of favorable clinical outcomes. CTP measurements (ICV or PV) measured by RAPID and UGuard as well as the above independent predictors were all incorporated to establish 6 predictive models (Table 4). With the increase of age, the probability of good outcomes decreased significantly. Patients with lower NIHSS score at admission were more likely to have a favorable outcome. Patients without atrial fibrillation had a higher probability of favorable outcome than patients with AF. Patients with successful recanalization (mTICI 2b − 3) after EVT were more likely to have a favorable outcome. In addition, the measurement of two software(RAPID and UGuard)were all associated with reduction of the probability of favorable outcome.

The ROC curves are shown in Fig. 4. There was no significant difference in AUC between Model 1 and Model 2, Model 3 and Model 4, Model 5 and Model 6 respectively, specificity and sensitivity of Model 2 was 0.83 and 0.72. The results showed that all the six models had good predictive performance and were reliable in predicting favorable functional outcome of AIS patients after EVT, and the model incorporating the ICV and PV measured by UGuard software (Model 2) showed the best predictive ability.

Fig. 4
figure 4

Receiver operating characteristic curves of the 6 different multivariable models. Model 1 with RAPID core volume and penumbra volume, Model 2 with UGuard core volume and penumbra volume, Model 3 with RAPID core volume, Model 4 with UGuard core volume, Model 5 with RAPID penumbra volume, Model 6 with UGuard penumbra volume.

Full size image

Subgroup analysis was conducted in groups with a sample size larger than 30. All P for interactions were greater than 0.05 throughout stratified subgroups (Supp Fig. 3), demonstrating that there was no interaction between clinical factors and UGuard measurements in influencing clinical outcomes. Therefore, the results of our multivariable model were rigorous.

Discussion

This study investigated the agreement and correlation between the RAPID and UGuard softwares in evaluating ICV and PV by using CTP data and their performance in predicting favorable outcomes following EVT. RAPID is a reliable software that has been validated extensively in randomized controlled trials8,9,12,13. In the measurement of ICV and PV, we found excellent correlation and agreement between the UGuard and RAPID software. In terms of predicting favorable outcomes, we found that after combining clinical predictors, the performance of UGuard was comparable to that of RAPID. Our study suggests that the UGuard software package is an alternative tool for analyzing ischemic CTP data with reliable predictive ability of favorable prognosis in AIS patients receiving EVT.

According to a previous study, RAPID software may underestimate the ICV in patients who receive intravenous iodine contrast injections as opposed to contrast-naïve patients, particularly during early time window14. All patients in our study received intravenous iodine contrast during CTP examination, which may have amplified the underestimation of ICV by RAPID. Koopman et al.15 compared RAPID with syngo.via® software. The study showed syngo.via® software similarly yielded significantly larger ICV measurements when compared to RAPID. Taken together, these may help to account for the difference of ICV measured by RAPID vs. UGuard in this study.

Our study showed significant differences in the measurements of ICV and PV between favorable and unfavorable outcome groups after EVT, and univariable logistic regression showed that larger ICVs and PVs were significantly associated with decreased occurrence of favorable outcome after EVT. A number of studies suggested that functional outcomes after EVT are associated with ischemic core volume, which conversely does not modify the treatment benefit of EVT16,17,18. In this regard, penumbral imaging may have its benefits by predicting functional improvement and independence after EVT19,20. Our findings indicated that the ICV measured by UGuard had similar performance with RAPID in predicting favorable outcome, with UGuard having a higher specificity.

The phenomena of ghost infarct core (GIC) has been proposed in a number of studies21,22, implying that clinical decision should not be solely based on CTP-determined core volume, but also take into account clinical factors. As a result in this study, we developed multivariable logistic regression models in both software packages to investigate the association between CTP measurements and clinical prognosis. We employed penalized logistic regression (PLR) method, which uses L1 regularized LASSO regression to select variables before multivariable logistic regression, and to remove component covariance to generate a more accurate model. Our multivariable models that utilized the ICV measured by UGuard, and the model used the RAPID measurements had similar performance in predicting favorable clinical outcome, as did the models that used the PV output from two softwares. Also, the models incorporating the ICV and PV measurment showed similar performance and the UGuard software model showed the best predicting performance. These results imply that a combined model incorporating clinical factors and CTP measurements can be instrumental in predicting functional outcome and guide therapy selection23.

Our study has limitations. First, because this is a retrospective study, we missed to collect some certain variables for baseline comparisons, e.g., time metrics such as onset-to-door time and onset-to-recanalization time which may help to explain the discrepancy of ICV measurements between RAPID and UGuard. However, this variability may be an advantage for the extrapolation of the results. Secondly, we did not compare with DWI-quantified core volume, which has been deemed as “gold standard”. However, since CT-based technology including CTP, instead of MRI, have been widely used in acute ischemic stroke patient assessment24,25, the direct comparison between UGuard and RAPID may have more practical significance. In addition, DWI image after the procedure may not be a good reference for comparison, since it is substantially dependent on the status of recanalization. Thirdly, most patients in our study had small infarct core volume. As our study was conducted before publication of the large ischemic core trials4,26,27, it is not known how the comparison between UGuard and RAPID will perform in patients with larger ischemic core infarct who receives EVT. At last, our study focused on evaluating the consistency between UGuard and RAPID in the agreement and correlation of ICV and PV measurement, future studies should explore the potential advantages of UGuard in terms of diagnostic accuracy, processing time, and clinical relevance. Such investigations could provide a more comprehensive understanding of UGuard’s performance and its potential benefits over existing tools. Our study did not collect data on infarct-associated seizures, intravenous thrombolysis, anatomical variants, brain tumors, or procedural details (e.g., thrombectomy duration, stent retriever passes, intra-arterial thrombolysis, stent placement, or residual stenosis). Additionally, we did not correlate the measurement of two softwares (RAPID and UGuard) with the mTICI score. While these factors were beyond the scope of our investigation, future studies should incorporate them to provide a more comprehensive understanding of their impact on perfusion analysis, thrombectomy outcomes, and alignment with clinical standards.

In conclusion, the UGuard software demonstrated good correlation with the RAPID software in the evaluation of ischemic core volume and penumbra volume in acute ischemic stroke patients undergoing EVT. The results of our study suggest that UGuard may serve as an alternative in the automated CTP assessment of acute ischemic stroke patients who are considered for EVT.