Introduction

Gliomas are the most common and lethal malignant primary brain tumors with significant mortality and morbidity1. Glioma-associated epilepsy (GAE) is frequently observed in patients with gliomas2,3 and significantly affects their quality of life. The etiology of GAE is likely multifactorial and complex and remains poorly comprehended4. Previous study5 suggested that response to successive antiepileptic drugs (AED) regimens is poorer than in the non–tumor-associated epilepsy population. If glioma patients have preoperative seizures, lack of gross total resection and postoperative disabling seizures, timely AED up-titration is recommended. If seizures persist, then either re-resection or a comprehensive epilepsy program assessment to identify and remove the ictal onset zone should be performed5. A better understanding of the mechanisms and early identification of GAE is crucial to protect neurocognition, restrict disease progression, and improve the quality of life of patients3,6.

The isocitrate dehydrogenase (IDH) gene is mutated in more than 70% of grade II and III gliomas7. IDH mutation status was suggested by the World Health Organization (WHO)8 as an essential biomarker for glioma subtyping and is associated with survival rate. IDH1 and IDH2 mutations are genetic alterations that occur frequently in gliomas patients and IDH1 mutations are more commonly associated with glioma than IDH2 mutations9. The classification of IDH into IDH-wild type (IDH-W) and IDH-mutant type (IDH-M) has now become an integral part of the molecular diagnosis of gliomas10. The current procedure for the identification of IDH mutation status with tissue samples is invasive and expensive. Therefore, a non-invasive alternative is highly desirable. Both classical machine learning (ML) and deep learning (DL) have been used in the non-invasive prediction of IDH mutation in glioma patients. While classical ML combining radiomics and clinical features has become a powerful tool in identifying IDH mutations11, DL models have also achieved remarkable success due to its ability to automatically learn relevant features from magnetic resonance imaging (MRI) data12.

Many studies have shown that IDH mutations are frequently associated with preoperative seizures in patients with grade II13,14,15 and grade III–IV gliomas13,16,17. D-2-hydroxyglutarate (D2HG) product of mutant IDH is similar in structure to glutamate, a widespread excitatory neurotransmitter in the central nervous system. Therefore, D2HG may disturb excitatory and inhibitory regions in the brain, subsequently triggering epilepsy18. IDH-M gliomas are more prone to develop seizures than IDH-W gliomas, and tumor growth may stimulate seizures, which may promote tumor growth in return19. This has important implications for the personalized management of GAE, as inhibitors targeting IDH-M may also improve antiepileptic therapy in patients with IDH-M gliomas. Utilizing correlation between IDH mutation and epilepsy identification tasks could improve the clinical diagnosis and treatment of glioma patients.

MRI is an indispensable preoperative examination for patients with glioma due to its high-resolution multiplanar structural information. Currently, multiparametric MRI (mpMRI) is routinely employed for the diagnosis and delineation of tumor compartments. Radiomics extracts large number of quantitative features from medical images and uses them to build predictive models that relate image features to the target information in an objective, repeatable, and non-invasive manner20. Previous studies have used radiomics for both IDH mutation status identification21 and GAE occurrence identification22,23 based on MRI. However, these studies examined these two issues separately, failing to take advantage of their latent correlation. Since IDH mutation status and GAE are closely related, we hypothesized that models for these two targets may share some common radiomics features and building models with shared features may lead to improved performance and better understanding of the models.

Materials and methods

Data

This study was approved by the institutional review board and the requirement for written informed consent was waived for its retrospective nature. Our research was carried out in accordance with the Declaration of Helsinki. We enrolled 399 consecutive patients with glioma who underwent MRI scanning before surgery at the hospital between August 2016 and September 2019 as the main cohort. The inclusion criteria and exclusion criteria are shown in Fig. 1. Preoperative diagnosis of GAE was based on clinical manifestations, electroencephalography (EEG), and imaging findings. IDH mutation, including the IDH1 or IDH2 mutation in the resected tumor was identified using immunohistochemistry. Patients from the hospital were scanned using 3.0 T MRI scanners (Magnetom Trio TIM/Prisma, Verio or Skyra, Siemens Healthcare; Discovery 750, GE Medical Systems). The T1-weighted image (T1WI), T2-weighted image (T2WI), and post-Gadolinium T1 weighted image (T1Gd) were scanned with a turbo spin-echo sequence (TR/TE 2000–3800/90–120 ms), spin-echo sequence (TR/TE 200–220/2–3 ms), and spin-echo sequence (TR/TE 250/2.5 ms), respectively. T2 fluid-attenuated inversion recovery image (T2-FLAIR) images were scanned with TI = 2400–2500 ms, TE = 81–135 ms, and TR = 8000–8500 ms. For all scans, image matrix = 256 × 256, field of view (FOV) = 240 × 240 mm2, section thickness = 5 mm, and intersection gap = 1 mm.

Fig. 1
Fig. 1
Full size image

Flowchart to illustrate the selection of the main cohort of the study.

The internal dataset was randomly split into a training cohort (n = 279) and an internal test cohort (n = 120). Repeated random splitting was used to ensure primary clinical characteristics, such as age (p = 0.334), gender (p = 0.603), and tumor grade (p = 0.934) exhibited no significant difference (p < 0.05 was regarded as statistically significant) between their distributions in the training and internal test cohorts.

A multi-center dataset from The Cancer Imaging Archive (TCIA)24 was used as external test cohort for IDH mutation status identification. The clinical information is provided by The Cancer Genome Atlas (TCGA)25. The data is screened for the availability of gender, age, IDH mutation status, preoperative T1WI, T2WI, T1Gd and T2-FLAIR images. The final external test cohort includes 228 patients.

Tumor segmentation and image preprocessing

To assess the performance of automatic segmentation, region of interest (ROI) containing the entire tumor and peritumoral edema was manually drawn on the T2-FLAIR images in the main cohort using the ITK-SNAP (version 3.6.0) software by two experienced neuroradiologists. For each patient, T1WI, T2WI, and T1Gd images were aligned to T2-FLAIR images using Elastix26,27. Before feature extraction, all images were normalized to the range of [0,1] and processed with the Contrast Limited Adaptive Histogram Equalization (CLAHE) algorithm28 to standardize the intensity variation between different MRI systems.

For automatic tumor segmentation, brain extraction was performed using HD-BET29, then a pre-trained nnU-Net model30 was used to automatically segment tumor ROIs comprising the entire tumor and peritumoral edema.

Radiomics feature extraction

Feature extraction was performed using the PyRadiomics (version 3.0)31 package in Python (version 3.7.6). For each case, features were extracted from auto-segmented ROI, including 14 shape features extracted from ROI mask, 18 first-order statistical features, and 75 texture features extracted from each of the original T1WI, T2WI, T1Gd, and T2-FLAIR images and corresponding Laplacian of Gaussian (LoG) filtered images, by three kernel sizes (1.5, 3.0 and 5.0 mm). The extracted texture features included those based on the gray level co-occurrence matrix (GLCM), gray level run length matrix (GLRLM), gray level size zone matrix (GLSZM), neighboring gray tone difference matrix (NGTDM), and gray level dependence matrix (GLDM). Age and gender were also used in model construction. All the features were normalized using z-score. For feature understandability and stability, we deliberately avoided using wavelet derived features.

Feature selection and model construction

Study workflow is shown in Fig. 2. First, to reduce feature dimensions, Pearson correlation coefficient (PCC) values between all features were calculated, and if the PCC value between two features was higher than 0.99, we randomly removed one of them. For feature selection, Least absolute shrinkage and selection operator (LASSO) was applied to each task to select task-specific features, and multi-task LASSO32 was used to select features shared by two tasks. For both LASSO and multi-task LASSO, we used bootstrapping to improve the robustness of feature selection. That is, each experiment was replicated 100 times using random resampling with replacement over the training cohort. In each repetition, we randomly select 80% cases in the training cohort for model construction, and 20% cases for cross-validation. Models with the validation area under the receiver operating characteristic curve (AUC) ≥ 0.7 were retained and features that were selected more than 80 times in the retained models were kept for subsequent analysis. The penalty term coefficient \(\:\lambda\:\) used in LASSO was empirically set to 0.001.

Fig. 2
Fig. 2
Full size image

Workflow of the study. Radiomics features were extracted from ROIs segmented with a pretrained model. Single-task LASSO was utilized to select task specific features for each task. Multi-task LASSO was utilized to select features shared by two tasks. Finally, task-specific and shared features were combined to develop a radiomics model for identifying IDH mutations status and glioma-associated epilepsy (GAE).

Finally, the shared and task-specific features were combined to construct the final model with LASSO for each task. Hyper-parameter \(\:\lambda\:\) was iterated between 0.1 and 0.0001, and determined by 1-SE rule to reduce number of features used in the model. Since the final models for two tasks retained their specific feature while sharing some of the features, we call this approach collaborative multi-task (CMT) learning.

For comparison, two single-task LASSO models and a multi-task LASSO model were also built with the same feature set and hyperparameters to identify the IDH mutation status and epilepsy.

In the entire process of all model construction, only the training cohort was used. Our code is available at: https://github.com/wangyidada/CMTLasso.git.

Evaluation of models

Receiver operating characteristic (ROC) curve analysis, confusion matrix and waterfall plot were used to evaluate the performance of the models, with both the internal and external test cohorts. Accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were also calculated at a cutoff value that maximized the value of the Youden index in the training cohort. DeLong test was used to pairwise compare the ROCs between different models. To investigate the model characteristics, Pearson correlation coefficient values between all features selected by either IDH mutation status model or GAE identification model were calculated and visualized.

Statistical analysis

Age is reported as the mean and range, and the difference in its distribution between the GAE and non-GAE groups and between the IDH-M and IDH-W groups was assessed using the Mann-Whitney U test. Gender and tumor grade were reported as frequencies and proportions, and differences between groups were assessed using the Pearson chi-square test. Statistical analysis was performed using SciPy (version 1.5.2) and statistical significance was defined as p-value < 0.05. The Radiomics Quality Score (RQS)20 was used for evaluating the quality of our study.

Results

Demographics

The main clinical and radiological characteristics of patients in the three cohorts are summarized in Tables 1 and 2. The number of patients in the training, internal, and external test cohorts were 279 (IDH-M/IDH-W: 146/133; GAE/non-GAE: 173/106), 120 (IDH-M/IDH-W: 60/60; GAE/non-GAE: 75/45), and 228 (IDH-M/IDH-W: 82/146), respectively. The tables indicate that there were significant differences (p-value < 0.001) in age and tumor grade between the IDH-M with IDH-W groups, as well as in the GAE and non-GAE groups. Statistically, young patients and patients with low-grade glioma (LGG) had a higher chance of having IDH-M type and were more likely to develop epilepsy.

Table 1 Clinical characteristics of cohorts used for IDH mutation status identification.
Full size table
Table 2 Clinical characteristics of cohorts used GAE identification.
Full size table

Performance of tumor segmentation

The pre-trained nnU-Net achieved a mean Dice of 0.847 ± 0.153 in the internal dataset. Figure 3 shows the segmentation results and ground truth for three patients with WHO grades II-IV.

Fig. 3
Fig. 3
Full size image

Visual comparison of segmentation results and the ground truth of 3 random patients with grade II (patient1), III (patient2) and IV (patient3), respectively, on T2-FLAIR images.

Performance of single-task LASSO models

For IDH mutation status identification, the single-task LASSO model utilized 23 features and achieved AUC values of 0.932 [95% confidence interval (CI), 0.901–0.958], 0.940 [95% CI,0.901–0.976] and 0.882 [95% CI, 0.831–0.926] in the training, internal test and external test cohorts, respectively.

For epilepsy identification, the single-task LASSO model utilized 17 features and achieved AUC values of 0.902 [95% CI, 0.865–0.936] and 0.871 [95% CI, 0.798–0.933] in the training and internal test cohorts, respectively. Detailed performance metrics of the two single-task models are listed in Tables 3 and 4, respectively. The ROC curves are shown in Fig. 4. The features retained in the two models and their corresponding weights are shown in Supplementary Figure S1. No shared features were selected by both the IDH mutation status and epilepsy models, which means single-task models use a total of 40 features for two tasks.

Table 3 Performance of models for IDH status identification in different cohorts.
Full size table
Table 4 Performance of different models for identifying epilepsy.
Full size table
Fig. 4
Fig. 4
Full size image

IDH mutation status and epilepsy identification results by single task (ST), multi-task (MT) and collaborative multi-task (CMT) models in different cohorts. (a–c) ROC curves for IDH mutation status identification by CMT, MT and ST models; (d–f) ROC curves for epilepsy identification by CMT, MT and ST models; (g–i) waterfall plots for the distribution of epilepsy identification in the internal test cohort, IDH mutation status identification in the internal test cohort, and IDH mutation status identification in the external test cohort by CMT models, respectively. (j–l) confusion matrix of epilepsy identification in the internal test cohort, IDH mutation status identification in the internal test cohort, and IDH mutation status identification in the external test cohort by CMT models, respectively. ROC = receiver operating characteristic; AUC = area under curve; GAE = glioma-associated epilepsy.

Performance of multi-task LASSO model

The multi-task LASSO model used a shared set of 30 features and achieved training AUCs of 0.927 [95% CI, 0.897–0.953] and 0.902 [95% CI, 0.864–0.934], internal test AUCs of 0.938 [95% CI,0.892–0.974] and 0.870 [95% CI, 0.809–0.972] for IDH mutation status identification and epilepsy identification, respectively. The multi-task model achieved AUC of 0.867 [95% CI, 0.817–0.915] in the external test cohort for identification of IDH mutation status. Detailed performance metrics of the models are also listed in Tables 3 and 4. The ROC curves are shown in Fig. 4. The weights of all the features for both tasks in the multi-task model are shown in Supplementary Figure S2.

Performance of CMT LASSO models

For IDH mutation status identification, the CMT LASSO model contained 20 features and achieved AUC values of 0.948 [95% CI, 0.923–0.970], 0.946 [95% CI, 0.902–0.982] and 0.860 [95% CI, 0.806–0.910] in the training, internal test and external test cohorts, respectively. For epilepsy identification, the CMT LASSO model contained 16 features and achieved AUC values of 0.924 [95% CI, 0.893–0.952] and 0.880 [95% CI, 0.811–0.937] in the training and internal test cohorts, respectively. Detailed performance metrics of the two models are listed in Tables 3 and 4 for comparison. The ROC curves, confusion matrix, and waterfall plots are shown in Fig. 4. The features included in the two models and their corresponding weights are shown in Fig. 5. Eight features were shared by both the IDH mutation status and epilepsy models, which means a total of 28 features were used for two tasks.

Fig. 5
Fig. 5
Full size image

Features and their corresponding weights used in final CMT models for identifying epilepsy occurrence (a) and IDH mutation status (b). Green and purple bars stand for weights for shared and task-specific features, respectively.

In the internal test cohort, there was no significant difference between CMT and ST LASSO models for IDH mutation status (p = 0.672) and epilepsy (p = 0.507) identification, and there was significant differences between CMT and MT LASSO models (p < 1e-5) in both two tasks. Overall, the proposed model outperformed the both single task and multi-task LASSO models. PCC values between all features selected by two models were shown Fig. 6, for ST, MT, and CMT schemes separately. Our study derived an RQS score of 19 (36 in total) (Supplementary Table S1).

Fig. 6
Fig. 6
Full size image

Pearson correlation coefficient (PCC) values between features selected by IDH and GAE models constructed with different approaches: single task (ST) (a), multi-task (MT) (b), and collaborative multi-task (CMT) (c) models for IDH mutation status and glioma-associated epilepsy (GAE) identification. Feature names are represented as sequence name for brevity. The detailed feature names and PCC values are shown in enlarged version in Supplementary Figures S3, S4, and S5.

Discussion

Typical radiomics studies treat each problem independently, ignoring the correlations between associated problems. On the other hand, multi-task learning (MTL) can jointly learn multiple related tasks so that the knowledge contained in a task can be leveraged by other tasks33. However, MTL uses a common set of features for all tasks, ignoring the unique characteristics of each task, which may lead to a sub-optimal model performance. Thus, we proposed an approach to combine classic LASSO and multi-task LASSO to select both shared and task-specific features to build collaborative multitask models without performance compromise. To the best of our knowledge, this is the first attempt to construct two radiomics models for correlated targets using both task-specific and shared radiomics features.

Epilepsy is a common and disabling symptom of many types of brain tumors, associated with tumor location, patient age, and histopathological subtype14. D2HG product of IDH mutation may increase neuronal activity by mimicking the activity of glutamate on the NMDA receptor, and IDH mutation gliomas are more likely to cause epilepsy in patients13. IDH1/2 mutations were reported to be an independent predictor for epilepsy, as a presenting feature in low-grade gliomas patients15. Patients affected by IDH mutant gliomas present more frequently with epileptic activity than patients with IDH wild-type gliomas9. Therefore, identifying IDH-mutant status is helpful for identifying treatments for epilepsy.

Manual delineation of ROIs is inevitably influenced by radiologists’ experience and subjective judgment, leading to the variability in extracted radiomics features. Thus, auto-segmentation was used to alleviate the workload and subjectivity of manual delineation. The pre-trained nnU-Net achieved good performance in segmenting the whole tumor ROIs, as demonstrated not only by the quantitative metrics in the internal dataset, but also by the performance of downstream diagnosis models.

Many studies have reported that the radiomics features extracted from MRI in patients with gliomas identified IDH mutation status with an AUC between 0.80 and 0.9611,34,35, and epilepsy with an AUC between 0.82 and 0.87 22,23,36,37. The details of these studies are summarized in Supplementary Tables S2 and S3. While the performance achieved by the proposed CMT learning ranked among the top, the major difference between this study and previous ones is that the correlation between IDH mutations and epilepsy was exploited to find the shared feature set. This minimized the number of features used, simplifying radiomics models and lowering the risk of overfitting. Different from previous multi-task studies38,39, we also made use of task-specific features to improve the performance of model. In the internal dataset, the CMT model achieved the highest AUC values for both tasks. Over the external dataset, the CMT model achieved a performance comparable to ST model for IDH mutations identification with smaller number of features (20 vs. 23), which reduced the model complexity. Interestingly, incorporation of task-specific features not only increased the performance of the model, but also reduced the total number of features used from 30 of multi-task model to 28 of CMT models, implying the importance of combining shared features with task-specific features. The reduction makes the model easier to understand and increases the chance of finding useful, generalizable biomarkers. More importantly, the features shared by the two tasks can be used to perform correlations association analysis between different tasks. The weights of most features have the same sign in both tasks, suggesting that inherent connection might exist between IDH-M and GAE.

Our results suggested that features from preoperative routine mpMRI were important biomarkers for gliomas since they are associated with both IDH mutations and epilepsy. Figure 5 outlined that the feature extracted from T1Gd and T2-FLAIR sequences exhibited high importance scores. Previous study40 also found that T1Gd was the optimal MRI sequence for IDH mutation status identification, T2-FLAIR sequence was moderately suitable, and T1WI and T2WI sequences were identified as the least suitable options. Age was shared by two tasks and negatively correlated with IDH-M/epilepsy, which is consistent with the observation that LGG is more common in younger patients and is correlated with a higher chance of having IDH-M/epilepsy than HGG patients41,42. Although tumor grade is an important biomarker for both tasks, its confirmation requires invasive surgery or biopsy, which limits the clinical use of the model. Therefore, we did not include tumor grade as a clinical characteristic in the model building.

Figure 6 visualizes the correlation between features retained in the models, reflecting feature redundancy in the model. It can be seen from Fig. 6a, in models built with single task LASSO, features retained in IDH model and GAE model are heavily correlated though there are no feature overlapping between these two models. This demonstrated that a failure to reveal the connections between two tasks. Multi-task model in Fig. 6b makes full use of the associations of the two tasks, however, there is still heavy redundancy in the model, implying an over-complex model and sub-optimal performance. In comparison, in CMT models in Fig. 6c, feature redundancy is reduced greatly. This is in accordance with the fact that CMT models used fewer number of features in total and in each model.

Here we would like to appeal to researchers developing radiomics models to pay more attention to the intrinsic quality of the models, including the complexity of the model, and the redundancy of the features retained in model, rather than just focusing on the classic performance metrics. It was reported in a systematic review that underpowered studies, which included more features than the sample size could support, lead inevitably to false-positive results43. For example, it has been suggested that the “rule of thumb” that there should be minimum 10 samples in the smaller group for each feature included in the model, should be used as an initial step to determine the sample size44. Reducing the feature number to reduce the risk of overfitting was also included in the widely-used radiomics quality score20. Among models with no significantly different performances, a simpler one using fewer features should be preferred. This idea is reflected in the popular 1-SE rule for feature selection and explains why the CMT models are deemed better than single-task models. Since the redundancy in the features used in the model is positively associated with the number of features used, visualization of the redundancy as in Fig. 6 may help us to determine whether the model can be further simplified.

Our study also has some limitations. Firstly, external validation was used only for IDH status identification. So, further validation with a larger, multi-institutional dataset covering both tasks is warranted. Furthermore, the clinical relevance of simultaneous prediction of IDH status and epilepsy is not yet clear. While this problem can be explored further in the future, we also hope the proposed CMT scheme could be validated in more studies where the simultaneous predictions of multiple tasks are more clinically relevant. Secondly, due to the limited dataset size, we did not conduct stratified studies on grade II, III, and IV glioma patients, which we believe can further clarify the implications of this study to patients with different grade gliomas. Thirdly, we built all models with features extracted from the whole tumor. Therefore, further studies using features from sub-regions of the tumor, such as the enhanced components, necrosis, and edema, might improve the model performance and provide more insight into the disease. Finally, the proposed CMT scheme can be validated in more studies on different problems.

In conclusion, we developed a novel radiomics approach for the identification of IDH mutation status and GAE using age and radiomic features extracted from mpMRI. The proposed approach exploited the underlying connections between these two problems, reduced the risk of overfitting, improved the model performance, and can potentially been used to find valuable generalized biomarkers for multiple clinical problems. The proposed model may be used as a convenient and non-invasive tool for the diagnosis and treatment of patients with gliomas.