Fig. 5: Applying the models to neoplastic and non-neoplastic regions.

a Pathologist segmentation of neoplastic regions (orange) and non-neoplastic regions (blue) on top of 200 H&E slides from the Carmel-Test dataset. Selected tiles are magnified for detailed observation. The segmentation was carefully done to avoid cross-contamination, with the pathologist marking only the clearly identifiable regions. b The model’s AUC performance was evaluated in three scenarios: applying it exclusively to neoplastic regions (orange) regions, to non-neoplastic regions (blue), and to both. This evaluation demonstrated that non-neoplastic regions contribute valuable information regarding the ER status of the tumor. The 95% confidence intervals are indicated, where the number of independent samples is n = 164. c Tiles from non-neoplastic regions with high and low ER prediction scores are illustrated for a better understanding of the system’s decision-making. d The heatmaps overlaid on top of the H&E images highlight the areas that the system relied on for making the prediction for ER status. The system did not highlight the lymphocyte area in the neoplastic region. In the non-neoplastic areas, the system highlighted columnar cell changes but did not highlight benign epithelial tissues, including the sclerosing adenosis. The color legend of the heatmap is shown. A stronger yellow color indicates a higher score for ER+ prediction, signifying that the system places greater reliance on this region for its final prediction. H&E Hematoxylin and Eosin, ER Estrogen Receptor, PR Progesterone Receptor, AUC Area Under the Curve.