Fig. 3: Flowchart and network architecture of the proposed method.

a Flowchart of the proposed DL method. (i) WSIs in multi-resolution pyramid tile-based structure \({\left\{{{{{\rm{Q}}}}}^{l}\right\}}_{l = 1}^{L}\) are fed into (ii) the proposed foreground patch selection (FPS) model to rapidly locate high-resolution foreground patches \(\left\{{{{{\bf{u}}}}}_{i}^{d,L}\right\}\) without marker regions annotated by medical experts. Then, (iii) the weakly supervised tumor-like tissue segmentation model Ψ_tumor applies to the selected foreground patches \(\left\{{{{{\bf{u}}}}}_{i}^{d,L}\right\}\) to further generate the tumor-like patch attention score \({{{{\boldsymbol{\xi }}}}}_{{q}_{j}^{{\prime} L}}\). Next, (iv) the proposed iterative patch sampling (IPS) method samples representative patches \(\left\{{{{{\bf{q}}}}}_{j}^{{\prime} L}\right\}\) with high attention score \({{{{\boldsymbol{\xi }}}}}_{{q}_{j}^{{\prime} L}}\). Afterwards, (v) the individual patch probability \({{{{\boldsymbol{\gamma }}}}}_{j}^{d,L}\) of the representative patch \({{{{\bf{q}}}}}_{j}^{{\prime} L}\) is obtained using InceptionV3 classifier, while (vi) the individual patch decision weight \({{{{\boldsymbol{\omega }}}}}_{{q}_{j}^{{\prime} L}}\) of the representative patch \({{{{\bf{q}}}}}_{j}^{{\prime} L}\) is computed. Subsequently, (vii) the proposed weighted softmax integrated decision (WSID) model produces a reliable and robust slide level probability \({\gamma }^{{\prime} d}\). (viii) Finally, the MSI status prediction \({D}_{MSI}^{d}\) of the d-th patient is generated. b The detailed networks of the proposed DL method.