Fig. 1: GNN representation of primary breast tumor and related prognostic models.

a Development of breast cancer prognostic biomarkers. b Main strategies of biomarker development for disease prognosis. c Co-registered multi-biomarker spatial heterogeneity (IGNN) in primary breast tumor with biomarker-biomarker interactions unavailable from corresponding single-marker heterogeneity (TACS1-8). d Personalized TACS1-8 reginal distributions in co-registered images (H&E, MPM of second harmonic generation SHG, and MPM of two-photon excited fluorescence TPEF) from one exemplary patient (9 regions/nodes, each of which encoded with an 8-bit vector) that result in one graph structure input for IGNN model and another non-graph input for TACS1-8 model. In the IGNN model that ends with an IGNN score, GRU based attention blocks execute node information propagation and aggregation along the graph structure and keep or remove node information related to prognostic prediction, whereas graph convolution and full connection extract prognostic graph representation and high-dimensional features sequentially. H&E hematoxylin and eosin, IHC immunohistochemistry, MP molecular profiling, ER estrogen receptor, PR progesterone receptor, HER2 human epidermal growth factor receptor 2, Ki67 specific nonhistone nuclear protein, MPM multiphoton microscopy, TPEF two-photon excited (intrinsic) fluorescence, SHG second harmonic generation, TACS (TACS1-8) tumor-associated collagen signatures, IGNN intratumor graph neural network, GRU gated recurrent units, SELU scaled exponential linear units.