Introduction

There is concern that clinical risk prediction models may reinforce existing disparities or introduce new biases in the allocation of healthcare resources1,2,3,4. Regulatory bodies seek to understand the impact of risk prediction models that include race and ethnicity to inform the implementation of risk-targeted care, such as decision-making criteria, risk communication, and issues around decision thresholds. Race and ethnicity are often included in breast cancer risk prediction models used for prevention because breast cancer incidence and mortality vary by race and ethnicity5,6,7,8,9. Assessment of algorithmic bias, attributes intrinsic to a model that may result in differential model performance in different groups1, and risk model equity has primarily focused on algorithms that impact treatment, such as lung function, cardiovascular disease, and stroke, with fewer evaluating risk models for cancer prevention10. One study that evaluated a lung cancer risk model found removing race and ethnicity increased screening eligibility for Hispanic and Asian individuals, but decreased eligibility for Black individuals11, while a prostate cancer risk model found adding a race coefficient provides a more accurate prediction of prostate cancer risk among Black men12.

Advanced breast cancer is a surrogate for breast cancer mortality13, making advanced breast cancer risk a clinically relevant measure to assess when discussing screening strategies with women. The Breast Cancer Surveillance Consortium (BCSC) developed the first actionable advanced breast cancer risk model to guide decisions on screening interval and the need for supplemental imaging in individuals with high advanced breast cancer risk (https://tools.bcsc-scc.ucdavis.edu/AdvBC6yearRisk/#/)5. The model includes race and ethnicity, and demonstrated that advanced breast cancer rates are twofold higher in Black compared to White women. Overall, the model has shown that in a screening population, most women (69%) are at low or average advanced breast cancer risk with annual or biennial screening, suggesting these women can undergo biennial screening to minimize the likelihood of screening harms associated with annual screening. Twelve percent of women are at intermediate advanced breast cancer risk with biennial screening and average risk with annual screening, suggesting they might benefit from receiving annual screening, and 17% are at intermediate or high advanced cancer risk with annual or biennial screening, suggesting consideration of supplemental imaging5. Given the 2024 USPSTF recommendation for biennial screening mammography for all women aged 40–74 years14, and the federal breast density law enacted in September 2024 requiring radiology practices to inform the 47% of women with dense breasts that they should consider of supplemental imaging15, the actionable advanced breast cancer risk calculator is uniquely positioned to guide clinical decisions on screening interval and supplemental imaging.

To assess the impact of including race and ethnicity in the BCSC advanced breast cancer risk model on algorithmic bias and model equity, we evaluated the effect of removing race and ethnicity from the model. We hypothesize that removing race and ethnicity from the advanced breast cancer risk model will reduce calibration and discrimination across race and ethnicity groups.

Results

The study cohort included 931,186 women aged 40–74 years undergoing 2,542,382 annual and 752,049 biennial screening mammograms who developed 1110 and 760 advanced breast cancers and 7297 and 4237 non-advanced breast cancers, respectively. Compared with women screened biennially, women screened annually tended to be older, have a family history of breast, and history of breast biopsy (Table 1).

Table 1 Characteristics of women undergoing annual and biennial screening5
Full size table

Calibration

In annual screeners, excluding race and ethnicity from the model resulted in overestimation of advanced breast cancer risk in Asian women compared to the original model [expected/observed = 1.28 (95%CI = 1.05–1.66) vs. 1.00 (95%CI = 0.82–1.29)] and an underestimation of advanced breast cancer risk in Black women [expected/observed=0.61 (95%CI = 0.53–0.70) vs. 1.00 (95%CI = 0.88–1.15)] and Other/Multiple race women [expected/observed=0.82 (95%CI = 0.70–0.99) vs. 1.00 (95%CI = 0.85–1.21)] (Table 2; Fig. S1). Biennial screeners had similar results except excluding race and ethnicity from the model resulted in overestimation of advanced breast cancer risk in Hispanic women compared to the original model [expected/observed=1.14 (95%CI = 0.92–1.52) vs. 1.00 (95%CI = 0.80–1.33)] while the model excluding race and ethnicity and the original model were both well calibrated for Other/Multiple race and White women undergoing biennial screening (Table 1; Fig. S2).

Table 2 Calibration for annual and biennial screeners, including and excluding race and ethnicity from the advanced breast cancer risk model
Full size table

Discrimination

Overall discrimination was slightly higher for the model including race and ethnicity than the model excluding race and ethnicity [AUC = 0.682 (95%CI = 0.670–0.694) vs. 0.677 (95%CI = 0.665–0.689)] (Table 3). Discrimination was similar across racial and ethnic groups for the two models, with discrimination slightly higher for Asian, Black and Other/Multiple race women in the model excluding vs. including race and ethnicity and slightly lower for Hispanic and White women.

Table 3 Area under the receiver operating characteristic curve for the advanced breast cancer risk model with and without race and ethnicity
Full size table

Absolute risk

Based on predicted advanced breast cancer risk under annual screening, the percentage of Asian women estimated at intermediate/high risk when excluding race and ethnicity from the advanced breast cancer risk model increased from 3.4% to 10.2% (Table 4). By contrast, the percentage of Black women estimated at intermediate/high advanced breast cancer risk when excluding race and ethnicity from the advanced breast cancer risk model decreased from 58.5% to 24.1% as did the percentage of Other/Multiple race women 27.9% to 17.0% (Table 4). White and Hispanic women had similar percentages of intermediate/high advanced breast cancer risk, whether including or excluding race and ethnicity from the advanced breast cancer risk model.

Table 4 Prevalence of high, intermediate, average, low and very low cumulative risk of advanced breast cancer after 6 years of annual screening in models including and excluding race and ethnicity
Full size table

Among women diagnosed with advanced breast cancer undergoing annual screening, a higher percentage of Black, Hispanic and Other/Multiple race women were identified as intermediate/high advanced breast cancer risk with the model including vs. excluding race and ethnicity (Fig. 1). By contrast, a higher percentage of Asian and White women are identified as intermediate/high advanced breast cancer risk with the model excluding vs. including race and ethnicity. For example, the percentage of advanced breast cancer classified as intermediate/high risk among Black women decreased from 75.3% to 47.5% and in Asian women increased from 6.1% to 16.7%. Black, Hispanic, and Other/Multiple race women diagnosed with advanced breast cancer have a lower estimated median absolute advanced breast cancer risk with the advanced breast cancer risk model excluding race and ethnicity vs. a model including race and ethnicity, while Asian and White women have higher estimated advanced breast cancer risk (Fig. 1, Table 5).

Fig. 1
figure 1

Violin plots showing the distribution of predicted cumulative risk of advanced breast cancer after 6 years of annual screening using models that include and exclude race and ethnicity, with prevalence of high, intermediate, average, low and very low cumulative risk, stratified by races and women diagnosed and not diagnosed with advanced breast cancer. Risk threshold based on distribution of risk in the combined sample of annual and biennial screeners of all races for each model: High risk; >95th percentile; intermediate risk,>75th and ≤95th percentile; average risk; >25th and ≤75th percentile; low risk; >5th and ≤25th percentile; very low risk; ≤5th percentile. Risk and prevalence adjusted by US population weights and standardized to the same population for annual and biennial.

Full size image
Table 5 Median (IQR) values for cumulative 6-year predicted advanced breast cancer risk using models including or excluding race, stratified by race and ethnicity and advanced breast cancer status
Full size table

Among women not diagnosed with advanced breast cancer undergoing annual screening, a higher percentage of Asian and White women were incorrectly categorized as intermediate/high advanced breast cancer risk for the model excluding vs. including race and ethnicity. A lower percentage of Black, Hispanic and Other/Multiple race women not diagnosed with advanced breast cancer were categorized as intermediate/high advanced breast cancer risk when race and ethnicity were excluded from the model. Black and Other/Multiple race women who did not develop advanced breast cancer have a lower estimated median absolute advanced breast cancer risk with the advanced breast cancer risk model excluding race and ethnicity, while Asian and White women have a higher estimated advanced breast cancer risk (Fig. 1, Table 5).

Biennial screeners have similar results to annual screeners, except Hispanic and other/multiple race women have a slightly greater percentage of women identified as intermediate/high risk of advanced breast cancer with the model excluding race and ethnicity (Table 5, Table S1, and Fig. S3).

Discussion

This study examined whether removing race and ethnicity from the BCSC advanced breast cancer risk model, used for decision-making on screening frequency and supplemental imaging, impacted the accurate categorization of women as intermediate/high advanced breast cancer risk. Calibration was optimal for all racial and ethnic groups when race and ethnicity was included in the advanced breast cancer risk model, whereas excluding race and ethnicity from the model resulted in overestimation of risk in some racial and ethnic groups and underestimation in others. We found that including race and ethnicity in the model categorized a higher proportion of Black, Hispanic and Other/Multiple race women who would develop advanced breast cancer as intermediate/high advanced breast cancer risk compared to a model excluding race and ethnicity. By contrast, including race and ethnicity in the advanced breast cancer risk model categorized a lower proportion of Asian and White women who develop advanced breast cancer as intermediate/high advanced breast cancer risk compared to the model excluding race and ethnicity. Given Black, Hispanic and Other/Multiple race women have a higher proportion of advanced breast cancer than Asian women, a model including race and ethnicity will identify more of these women who may benefit from annual screening and/or supplemental imaging.

Black women have the largest proportion of women diagnosed with advanced breast cancer who would not be identified as having intermediate/high advanced breast cancer risk if race and ethnicity are removed from the advanced breast cancer risk model. Including race and ethnicity in the model would identify the largest number of Black women diagnosed with advanced breast cancer as having intermediate/high advanced breast cancer risk for consideration of annual screening and/or supplemental imaging; however, it would also identify the largest number of Black women not diagnosed with advanced breast cancer who might be similarly considered for annual screening and/or supplemental imaging. By contrast, Asian women have the lowest rate of advanced breast cancer, so a small number of women diagnosed with advanced breast cancer would not be identified as intermediate/high advanced breast cancer risk if race and ethnicity are included in the advanced breast cancer risk model. Excluding race and ethnicity from the model would identify more Asian women who will develop advanced breast cancer, who could consider annual screening and/or supplemental imaging, as well as more women who will not develop advanced breast cancer, but who could be recommended for annual screening and/or supplemental imaging. Thus, there is a trade-off between appropriately offering more frequent screening and/or supplemental imaging to women with the highest rate of developing advanced breast cancer and inappropriately offering more frequent screening and/or supplemental imaging to women who will not develop advanced breast cancer, leading to false-positive tests and benign biopsies16. By including race and ethnicity in the advanced breast cancer risk model, a greater number of Black, Hispanic and Other/Multiple race women at intermediate/high advanced breast cancer risk who will develop advanced breast cancer could be offered annual screening and/or supplemental imaging at the detriment of a smaller number of Asian women being identified as intermediate/high advanced breast cancer risk who will develop advanced breast cancer.

A study of colon cancer patients examined whether including race and ethnicity as a predictor in a colorectal cancer recurrence risk algorithm results in racial and ethnic differences in model accuracy that could potentially lead to unequal treatment17. The authors found calibration was worse when excluding race and ethnicity, and this could lead to inappropriate care recommendations for some racial and ethnic groups. A study of a colon cancer risk prediction model found that adjusting for race and ethnicity compensated for data quality differences for key predictors and increased the fraction of Black participants among the predicted high-risk group eligible for screening18. Another study evaluating a lung cancer risk model found that removing race and ethnicity increased screening eligibility for Hispanic and Asian individuals but decreased eligibility for Black individuals11. A cross-sectional study found removing race and ethnicity from the Breast Cancer Risk Assessment Tool resulted in substantial miscalibration, overestimating invasive cancer risk in Asian women19. As with our advanced breast cancer risk model, excluding vs. including race and ethnicity from the model resulted in reduced calibration and had differential effects by race and ethnicity, with the magnitude of the effect greatest for Black women.

We studied a large, diverse, population-based sample of women undergoing annual or biennial screening, which makes our results most applicable to regular screeners. Some estimated confidence intervals are wide due to small sample sizes, resulting in imprecise risk estimates. Our study does not address optimal advanced breast cancer risk thresholds or individual preferences for advanced breast cancer risk thresholds. Harms were not assessed, but published studies have reported the cumulative risk of false-positive mammography and biopsy by screening interval and reported almost 2-fold greater harms with annual vs. biennial screening and supplemental imaging16,20,21. We were unable to separate Pacific Islander women from other racial and ethnic groups, which is an important modification for future model versions, given that Pacific Islander women are at increased risk of advanced breast cancer22.

Determining advanced breast cancer risk in women with risk factors for advanced breast cancer (Black race, obesity, and heterogeneously or extremely dense breasts) and offering annual screening and/or supplemental imaging to those at intermediate/high advanced breast cancer risk could lead to a decrease in advanced breast cancer diagnoses and breast cancer mortality. Supplemental imaging and more frequent screening among those at elevated breast cancer risk have been associated with reductions in advanced cancer rates and breast cancer mortality23,24. Mutation carriers who undergo supplemental MRI screening have been shown to have reduced advanced breast cancer rates compared to those who undergo mammography alone23. Modeling studies have shown that more deaths are averted from breast cancer among women at high breast cancer risk screened annually vs. biennially24. Recently, supplemental ultrasound among women with heterogeneously or extremely dense breasts has been shown to have higher cancer detection rates for women with high advanced breast cancer risk as opposed to high invasive cancer risk25. Taken together, these studies suggest that assessing advanced breast cancer risk and tailoring screening frequency and supplemental imaging use could reduce the risk of an advanced breast cancer. However, studies are needed and underway to directly evaluate the association of advanced breast cancer risk assessment and clinical management strategies with the development of advanced breast cancer and breast cancer mortality.

We found removing race and ethnicity from the advanced breast cancer risk model worsened calibration for Black and Asian women and had a small impact on discrimination. Removing race and ethnicity from the model resulted in identifying fewer Black, Hispanic and Other/Multiple race women who would develop advanced breast cancer as having intermediate/high advanced breast cancer risk—women who may have benefited from annual screening and/or supplemental imaging. Including race and ethnicity in the advanced breast cancer risk model identifies a high proportion of women who develop advanced breast cancer as having high advanced breast cancer risk, supporting retaining race and ethnicity in the model. Thus, given our model is well calibrated, which is a necessary criterion for model fairness26, risk models with modest discriminatory accuracy (AUCs of 0.6–0.7) such as ours still provide a useful perspective for individual counseling and for weighing the harms and benefits of preventive clinical interventions27. For example, a 59-year-old Black postmenopausal obese woman with heterogeneously dense breasts undergoing annual or biennial mammography would be at high risk of advanced breast cancer and could consider supplemental imaging. Whereas the same woman who was of normal weight with scattered fibroglandular density would be at average risk of advanced cancer with annual or biennial screening.

Incorporating new risk factors associated with advanced breast cancer risk, such as mammographic calcifications28, social vulnerability index29, and/or artificial intelligence-derived measures30 into the advanced breast cancer risk model with the goal of improving discriminatory accuracy will require re-evaluation of whether model calibration is maintained for all women.

Methods

Study setting and data sources

Data were from the BCSC mammography registries (http://www.bcsc-research.org/index.html), whose demographics are comparable to the US population31,32,33. We prospectively collected data on women’s characteristics and mammography from radiology facilities. Breast cancer diagnoses and tumor characteristics were obtained by linking women to pathology databases; regional Surveillance, Epidemiology, and End Results programs; and state tumor registries. Deaths were obtained by linking to state death records. Registries and the Statistical Coordinating Center (SCC) received Institutional Review Board approval for active or passive consenting processes or a waiver of consent to enroll participants, link data, and perform analyses. All procedures were Health Insurance Portability and Accountability Act compliant, and registries and the SCC received a Federal Certificate of Confidentiality and other protections for the identities of women, physicians, and facilities.

Participants

We used the data used to create the BCSC advanced breast cancer risk model as previously described5. In brief, screening mammograms were defined based on radiologist’s reported clinical indication. To reflect women routinely screened, we identified 3,507,682 screening mammograms performed from January 2005 through December 2017 among women aged 40–74 years with a mammogram 11–30 months earlier. Annual screening was defined as having a prior mammogram within 11–18 (mean = 13.8) months, and biennial screening was defined as having a prior mammogram within 19–30 (mean = 23.7) months. We excluded screens from women with either a breast cancer history, mastectomy or lobular carcinoma in situ, screening mammograms that were either unilateral, preceded by mammography within 9 months, performed with a screening ultrasound on the same day, or screening MRI occurred 12 months before or after leaving 3,294,431 annual or biennial screens. The model is available on the web (https://tools.bcsc-scc.ucdavis.edu/AdvBC6yearRisk/#/) and can be downloaded via the iPhone or Google Play app stores. In the last two years, the web page has been visited over 4200 times, the free iOS App has been downloaded >2000 times, and the free Android App has been downloaded >300 times.

Measures, definitions, outcomes

We collected demographic and breast health history from self-administered surveys at screening and/or extracted from electronic health records. Women self-reported race and ethnicity separately which we classify into the following categories with Hispanic as a separate group and all race groups are non-Hispanic: Hispanic and non-Hispanic African American/Black, Asian, White, Other/Multiple race (non-Hispanic Native American/Alaskan Native, Native Hawaiian or Pacific Islander), or with two or more reported races, or other). Radiologists categorized breast density during clinical interpretation using Breast Imaging Reporting and Data System (BI-RADS)34 density categories: almost entirely fat, scattered fibroglandular densities, heterogeneously dense, and extremely dense. Postmenopausal women were those with both ovaries removed, whose periods had stopped naturally, who were current postmenopausal hormone therapy users, or who were age 60 or older. Premenopausal women reported a period within the last 180 days or birth control hormone use. Perimenopausal women were not sure if their periods had stopped, or their last menstrual period was 180-364 days prior35,36,37. Body mass index (BMI) was categorized as <18.5 kg/m2 = underweight, 18.5–24.9 kg/m2 = normal weight, 25.0–29.9 kg/m2 = overweight, 30.0–34.9 kg/m2 = obese I, and ≥35.0 kg/m2 = obese II/III38.

We grouped prior benign diagnoses abstracted from clinical pathology reports based on the highest grade as proliferative with atypia > proliferative without atypia > non-proliferative using published taxonomy39,40,41 or as unknown if a woman reported a prior biopsy with no available BCSC pathology result.

Mammograms were linked to invasive breast cancer or ductal carcinoma in situ diagnoses within 12 months after annual and 24 months after biennial mammography. We calculated the American Joint Committee on Cancer, 8th edition prognostic, pathologic stage42 using anatomic staging elements, tumor grade, and estrogen, progesterone, and human epidermal growth factor receptor status as in the original BCSC advanced breast cancer risk model5.

Statistical approach

The screening mammogram was the unit of analysis unless otherwise specified. We characterized mammograms associated with no advanced breast cancer or advanced breast cancer for annual and biennial screens according to risk factors. We estimated absolute advanced breast cancer risk and competing events (death or early-stage cancer) using logistic regression. For the original BCSC advanced breast cancer model5, separate models were fit by menopausal status (premenopausal versus postmenopausal) and screening interval and included age (linear and quadratic), race and ethnicity, first-degree breast cancer family history, history of benign biopsy, BMI, and breast density. To evaluate the impact of excluding race and ethnicity, we further fitted similar models but excluded race and ethnicity and compared the performance of the two models. Before model fitting, 15 imputed values for each missing variable were generated using MICE (multiple imputation chained equations)43. For each covariate combination, risk scores from a single screening round were estimated by averaging over the 15 risk scores estimated from each imputed dataset. Full details of MICE methods used in developing the model have been previously published5. Model calibration, which tests if predicted risks are numerically accurate, was estimated by the ratio of expected to observed number of advanced breast cancers after one screening round overall and for race/ethnic groups. Model discriminatory accuracy, which tests if the model can tell who is at high versus low risk, was summarized overall and for racial/ethnic groups using the area under the receiver operating characteristic curve (AUC) estimated with 5-fold cross-validation to compare predicted to observed risk after one screening round (Fig. S4).

Cumulative 6-year advanced breast cancer risks after six annual and three biennial screens were estimated using a discrete time survival model based on the fitted logistic regression models for each screening round while considering competing risks of death or early-stage cancer44. Advanced breast cancer could occur after any annual or biennial screen during the six-year follow-up period. Mean six-year cumulative risks and interquartile ranges for annual and biennial screening were standardized to the U.S. population of women based on age, race and ethnicity, and breast cancer family history by weighting the overall study cohort45,46. According to the six-year cumulative advanced breast cancer risk, women were categorized into five risk levels for each model: high, >95th percentile; intermediate, >75th to ≤95th percentile; average, >25th and ≤75th percentile; low, >5th and ≤25th percentile; very low, ≤5th percentile. The percentages of women in each risk level were compared between the two models for the overall cohort and by race/ethnic groups. Violin plots including boxes representing the middle 50% of cumulative advanced breast cancer risk were used to compare the two models, stratified by race/ethnic groups and advanced breast cancer status.

Data were analyzed using R version 4.3.3 (R Foundation for Statistical Computing, Vienna, Austria) and SAS version 9.4 (SAS Institute, Cary, NC).