Introduction

Brain metastases (BM) have been detected more commonly in recent years due to advances in diagnostic imaging and systemic treatment. It has been reported that 1–2% of BM originate from colorectal cancer (CRC).1,2 Recursive partitioning analysis (RPA)3 based on prognostic factors including Karnofsky performance status (KPS) score, age, adequate control of primary tumor, presence of extracranial metastases, and the diagnosis-specific graded prognostic assessment (DS-GPA) which includes the number of brain metastases have been developed and validated for predicting the prognosis of BM regardless of the primary tumor site.4,5,6 These are recognized to be reliable prognostic factors in patients with BM.7,8,9.

The KPS score indicates the functional status of a patient and3,4 is a standard method, for measuring the ability of cancer patients to perform ordinary tasks especially in the field of neurosurgery.10 This score ranges from 0 (death) to 100 (no complaints, no evidence of disease). The KPS score is the only significant prognostic factor shared among gastrointestinal cancers, such as CRC, gastric cancer, and esophageal cancer.5,11,12 In 2019, we investigated specific prognostic factors in 68 patients with brain metastases from CRC using the KPS score as an essential covariate, and found that KPS, number of brain lesions, and previous history of chemotherapy were independent prognostic factors.13 In a subsequent study, we also found that survival improved since 2014, possibly attributed to advances in chemotherapy and the more widespread use of stereotactic radiotherapy.14.

Nomograms are graphical representations of statistical prediction models and are widely used as prognostic tools in oncology and medicine. Balachandran et al.15 reported a systematic approach to practical evaluation and interpretation of nomogram-derived prognoses, emphasizing the limitations of nomograms and correcting misconceptions about them. Recently, several nomograms have been developed to predict oncological outcomes in patients with CRC,16,17,18,19, including postoperative complications.20,21 However, brain metastases from CRC are so rare that only a few validated prognostic models have been proposed. Pietrantonio et al.22 evaluated the performance of prognostic classifications, such as RPA and DS-GPA, and developed a nomogram for estimating survival in patients with brain metastases from CRC using the covariates of age, KPS, and numbers and sites of brain metastases.

Given the limited applicability of general BM prognostic scores to CRC-BM and the lack of validated CRC-specific tools, a dedicated nomogram may help clinicians estimate individual survival more accurately and guide treatment selection, multidisciplinary planning, and patient counseling. The aim of this study was to develop and externally validate a nomogram for predicting OS in patients with brain metastases from CRC.

Materials and methods

Patient selection and data collection

The retrospective study consisted of two parts: training cohort and validation cohort. The training cohort included patients with synchronous or metachronous brain metastases after treatment for CRC at the National Cancer Center Hospital between January 1987 and December 2017, and the validation cohort included patients who had received initial treatment for primary CRC at other institutions between February 1987 and July 2017, referred to the Department of Neurosurgery and Neuro-Oncology at the National Cancer Center Hospital for treatment of brain metastases (Fig. 1). It should be noted that there was no overlap of patients between the training and validation cohorts; each patient was uniquely assigned to one cohort only.

Fig. 1
figure 1

Flow chart of patients in this study. The final study population consisted of 226 patients with brain metastasis (BM) from colorectal cancer (CRC) treated at the National Cancer Center Hospital (NCCH) from 1985 to 2017. The training cohort consisted of 112 consecutive patients, accounting for 1.0% of all patients with CRC. The validation cohort consisted of 114 patients with brain metastases who were treated for primary CRC at other institutions in 1987–2017 and referred to our hospital to treat BM from CRC.

Full size image

The primary outcome of this study was OS defined as the interval between the date of diagnosis of brain metastases and the date of death from any cause or censored at last follow-up. The following parameters at diagnosis of brain metastases were assessed as the candidates for prognostic factors: sex, age, KPS, number of brain metastases, maximum diameter of brain metastases, and the clinicopathological features of the primary tumor, the time interval between diagnosis of the primary tumor and diagnosis of brain metastases, presence of extracranial metastases, and control of the primary tumor.

The cut-off date was August 2017 in both the training and validation cohorts.

Ethics statement

This study was approved by the Institutional Review Board (IRB) of the National Cancer Center Hospital (IRB code: 2017–437). The research was conducted in accordance with the ethical standards of the Declaration of Helsinki. Given the retrospective nature of this study, the requirement for informed consent was formally waived by IRB of the National Cancer Center Hospital.

Statistical analysis

OS was estimated using the Kaplan–Meier method, and the log-rank test was used to compare survival. Independent prognostic factors in patients with CRC and brain metastases were identified using multivariable Cox proportional hazards regression models were fitted to evaluate the relationship between prognostic factors and OS to adjusting potential confounders. The proportional hazards assumption was verified by tests of correlations with time and examination of residual plots. To allow for non-linear relationships, continuous variables were modeled with restricted cubic splines23 and were transformed to an appropriate form for satisfying the proportional hazards and linearity assumptions.

Variables for constructing the nomogram were selected from prognostic factors identified in multivariable analysis of the training cohort and other known prognostic factors in brain metastases from non-specific primary tumors.3,4,7,8,13 A nomogram was constructed for predicting 3-year, and 5-year OS rates, in which survival probability was obtained by summing the scores identified on the points scale for each variable.

The nomogram was validated by analyzing discrimination and calibration in the validation set, as described in our previous report.17 The concordance index (C-index) gives the probability that one patient with worse prognosis predicted by the nomogram than the other in two randomly selected patients actually had death event earlier. Because this study included censored data, Harrell’s C-index was used.23 In general, a C-index value > 0.75 is taken to indicate relatively good discrimination. The nomogram was calibrated by comparing the means for predicted survival with those for actual survival based on Kaplan–Meier estimates24 after grouping nomogram-predicted survival by decile. We followed the TRIPOD guidelines25,26 for reporting of studies that develop, validate, or update a prediction model, either for diagnostic or prognostic purposes.

OS was estimated using the Kaplan–Meier method, and the log-rank test was used to assess differences in survival. Multivariable Cox proportional hazards regression models were fitted to evaluate the relationship between brain metastases from CRC and OS, while controlling for potential confounders. Independent prognostic factors in patients with CRC and brain metastases were determined using Cox proportional hazards models.

Data are described as number of patients, the proportion (percentage), or hazard ratio (HR) with 95% confidence interval (CI), as appropriate. All statistical analyses were performed using R version 3.5.2 (The R Foundation for Statistical Computing R, Vienna, Austria). A P-value < 0.05 was considered statistically significant.

Results

Characteristics of the training and validation cohorts (Table 1)

Table 1 Patient demographics and tumor characteristics.
Full size table

During the study period, 9388 patients with stage I/II/III CRC underwent tumor resection, and 1479 patients with stage IV CRC were treated at our institution (Fig. 1). Fifteen (1.0%) of patients with stage IV disease had synchronous brain metastases at the diagnosis of CRC. Furthermore, 97 patients had metachronous brain metastases. Accordingly, the training cohort consisted of 112 consecutive patients, accounting for 1.0% of all patients with CRC (Fig. 1). Sixty-seven of the 112 patients were male and 45 were female. Median age at diagnosis of brain metastases was 62 years (range, 32–84). The KPS was ≥ 70 in 54 patients (48%) and < 70 in 58 patients (52%). Ninety-three patients (83%) had extracranial disease; 76 patients (68%) had a limited number of brain lesions (1–3) and 36 (32%) had multiple lesions (≥ 4). The primary tumor was uncontrolled in 13 patients (12%) (Table 1).

The validation cohort consisted of 114 patients. Sixty-five patients in this cohort were male and 49 were female. Median age at diagnosis of brain metastases was 61 years (range, 43–68). The KPS score was ≥ 70 in 73 patients (64%) and < 70 in 41 (36%). One hundred and five patients (92%) had extracranial disease. Ninety-one patients (80%) had a limited number of brain lesions (1–3) whereas 23 patients (20%) had multiple brain lesions (≥ 4). The primary tumor was uncontrolled in 12 patients (11%).

Compared with the training cohort, the validation cohort had significantly higher proportions of patients who had ≥ 4 brain metastases (P < 0.03), were chemotherapy-naive (P = 0.0164), had a KPS score ≥ 70 (P = 0.0165), and had extracranial lesions (P = 0.04).

Long-term outcomes after diagnosis of brain metastases and causes of death

Median survival time was 6.6 months in the training set and 5.8 months in the validation set. The 1-year OS rate was 29.0% in the training set and 27.0% in the validation set, and the 3-year OS rates were 7.2% and 8.4%. There was no significant difference in OS between these two cohorts. Of note, one patient in the training cohort and 3 in the validation cohort survived for more than 5 years (Fig. 2).

Fig. 2
figure 2

Overall survival curve of training the training cohort (n = 112) and the validation cohort (n = 114). Median survival time was 6.8 months in the training set and 5.8 months in the validation set. The 1-year OS rate was 28.0% in the training set and 27.0% in the validation set; the respective 3-year OS rates were 7.4% and 8.4%. MST, median survival time; OS, overall survival.

Full size image

HRs with 95% CIs for selected variables in Cox proportional hazards models are shown in Table 2. The KPS score (HR 1.64, 95% CI 1.05–2.6; p = 0.043, for a score < 70 vs ≥ 70), number of brain metastases (HR 2.13, 95% CI 1.18–3.91; p = 0.02, for extensive vs solitary), and history of chemotherapy (HR 1.39, 95% CI 1.16–1.67; p < 0.01, per unit increase, respectively) were identified to be statistically significantly associated with prognosis.

Table 2 Variables selected for the Cox proportional hazards regression model for overall survival.
Full size table

After KPS score and number of brain metastases were grouped into categories before modelling due to their skewed distribution, nomograms which reflects the impacts of prognostic factors were constructed to predict the 1-year, 3-year, and 5-year probability of survival in patients with brain metastases from CRC (Fig. 3) based on independent variables in the multivariable Cox regression model. The KPS score and number of brain metastases had a skewed distribution and were grouped into categories before modelling.

Fig. 3
figure 3

Nomograms to evaluate OS in patients with brain metastases from CRC. Using the nomogram, the 1-year, 3-year, and 5-year probability of survival can be estimated, which reflects the effects of prognostic factors. The KPS score and number of brain metastases had a skewed distribution and were grouped into categories before modeling. BM, brain metastases; KPS, Karnofsky performance status; prob., probability.

Full size image

In the internal validation, the Harrell’s C-index for the nomogram predicting survival was 0.70. Calibration curves for the predicted and actual 3-, 6-, and 12-month overall survival (OS) rates are shown in Fig. 4; actual survival was estimated by Kaplan–Meier analysis with 95% confidence intervals. The calibration curves showed good agreement between predicted and observed survival across all three time points.

Fig. 4
figure 4

Calibration plot for validation cohort (n = 112) and Harrell’s C-index value for the nomogram predicting survival. The calibration curves for the nomogram are shown and depict the predicted and actual 3-, 6- and 12-months OS rates. Actual survival rates with 95% CIs were calculated by Kaplan–Meier analysis. Calibration estimates how close the risk estimated by the nomogram is to the observed risk and is depicted by a calibration plot. The nomogram makes an optimistic prediction for the training cohort (the solid line is below the gray line). CI, confidence interval; OS, overall survival.

Full size image

In the external validation set, the nomogram yielded a Harrell’s C-index of 0.63, and the calibration plots (Fig. 5) indicated that the predictive performance was more modest than in the internal validation. Interestingly, the 3-month calibration curve showed that the observed survival rate was substantially higher than the predicted rate, suggesting that the model may underestimate early survival in external cohorts. Calibration at 6 and 12 months demonstrated more acceptable agreement between predicted and observed outcomes.

Fig. 5
figure 5

Calibration plot for validation cohort (n = 114). The nomogram makes a pessimistic prediction for the external validation cohort (the solid line is generally above the dotted line). Furthermore, as time passes, the predictive ability decreases (i.e., the calibration deviates from the x = y line), which may be due to the poor prognosis (median survival of approximately 6 months). CI, confidence interval; OS, overall survival.

Full size image

To further explore the potential performance of a unified prognostic model, we constructed an exploratory nomogram using the entire cohort (n = 226) by refitting the model. The C-index of the refitted model was 0.68, slightly lower than that of the training cohort model (0.70), but the calibration plots showed good agreement between predicted and observed survival at 3, 6, and 12 months. These results are shown in Supplementary Figs. 1 and 2.

Discussion

The prognostic nomogram developed in this study has three key advantages. First, the predictive model was based on the largest population ever analyzed. Second, we included CRC-specific factors based on previous studies as well as factors for primary tumors from any site in the development of the predictive model. Third, we used a dataset of consecutive patients referred from other institutions to check for external validity. Therefore, this nomogram can be considered to have high clinical utility.

Selecting covariates based on a priori clinical hypotheses avoids excluding covariates based on incomplete data and selection purely based on statistical significance; treatment per se should not be added to covariates to prevent selection bias unless there are validated data from a randomized clinical trial.15 Inclusion of covariates in a multivariable analysis should follow Harrell’s guideline (number of events should exceed the number of covariates by at least tenfold). According to these principles, we selected potential predictors based on the previous literature, including age, KPS,5,6 number of brain lesions,27 and history of chemotherapy.13 Furthermore, volume-related factors, such as maximum diameter of the largest tumor and cumulative tumor volume, are important in determining the treatment strategy28,29 (surgery, stereotactic radiosurgery, whole-brain radiation therapy, or systemic therapy). To further assess the relationship between tumor size and lesion burden, we conducted an exploratory analysis examining whether maximum diameter (≥ 3 cm vs < 3 cm) was associated with the number of brain metastases. However, neither Pearson’s chi-squared test (p = 0.2376) nor likelihood ratio test (p = 0.1933) indicated a statistically significant association. This supports the interpretation that tumor size and number may reflect distinct biological characteristics, justifying their inclusion as independent variables. According to the 2021 edition of the National Comprehensive Cancer Network Clinical Practice Guidelines in Oncology of Central Nervous System Cancers, the definition of extensive or multiple BM, which until the 2017 edition used the number of BM as an essential prognostic factor when selecting treatment for BM, was updated to recognize the concept of total intracranial tumor volume in the 2021 edition. As background for this update, it was noted that there may be a group of patients for whom stereotactic radiotherapy is as effective as whole-brain radiotherapy and can significantly protect cognitive function.29 Thus, after selecting covariates that followed previous findings and performing multivariate analysis, the KPS, number of brain lesions, and history of chemotherapy were also independent prognostic factors in our training cohort.

The general idea of validating a prediction model is to establish that it performs well for new patients. External validation, namely, validating the model using a new dataset collected from a relevant patient population at a different center, investigates whether a prediction model is transportable or generalizable to new patients.30,31 Discrimination quantifies the ability of the model to distinguish between low-risk and high-risk patients, and a concordance index23 is widely used as a measure of predictive discrimination. Concordance measures usually take values between 0.5 and 1, where a value of 0.5 indicates a random chance of correct prediction (no discrimination) and a value of 1.0 indicates perfect discrimination.

Calibration estimates how close the risk estimated by the nomogram is to the observed risk and is depicted by a calibration plot (Figs. 4 and 5). It is not an intrinsic property of a nomogram but rather an evaluation of how it performs in a particular cohort. This may be the result of selection bias in the patients referred to our hospital from other institutions. The nomogram makes an optimistic prediction for the training cohort (the solid line is below the gray line) and a pessimistic prediction for the external validation cohort (the solid line is generally above the dotted line). Furthermore, as time passes, the predictive ability decreases (i.e., the calibration deviates from the x = y line), which may be due to the poor prognosis (median survival of approximately 6 months).

As a result of discrimination and calibration, the evaluation of the nomogram created in this study was modest, suggesting that more cases need to be accumulated to improve the usefulness of the nomogram in clinical practice.

This study had some potential limitations. First, the retrospective design may have introduced bias. Second, the sample size was relatively small, though to our knowledge this was one of the largest studies yet reported in patients with brain metastases from CRC and sufficient background data were available, including KPS. Third, the study period was long and there have been dramatic changes in the detectability of brain metastases and their treatment strategies during that time. Although our previous study demonstrated an improvement in prognosis after 2014, likely due to advances in systemic chemotherapy and focal radiotherapy14, we did not include year of diagnosis as a covariate in the present model. This is because key clinical variables that directly influence treatment indications such as number and size of brain metastases, KPS score, and history of chemotherapy—were already incorporated and may partially account for temporal changes. In addition, although systemic chemotherapy was included in our model, the use of molecularly targeted agents, which were introduced after 2017, was not separately accounted for and was considered indirectly reflected by the systemic chemotherapy variable. Moreover, the use of immune checkpoint inhibitors for dMMR/MSI-high CRC was only introduced after 2021, which is beyond the study period. This should be acknowledged as a limitation.

Nonetheless, residual temporal bias may still exist and should be considered when applying this nomogram to future patients. Despite these limitations, our findings warrant confirmation in a larger series of patients with CRC and brain metastases.

Conclusions

In addition to KPS and number of brain metastases, which are known prognostic factors for brain metastases regardless of the primary site, we identified history of chemotherapy before diagnosis of brain metastasis to be an independent prognostic factor for brain metastases from CRC. We developed a prognostic nomogram using this factor in addition to known prognostic factors and evaluated its predictive ability both internally and externally.

The results of the external validation were more modest than those of the internal evaluation, suggesting that further accumulation of cases is necessary to improve the usefulness of this nomogram in clinical practice.