Fig. 1: The overall procedure of this study.

a Preprocessing and feature extraction. The original whole slide image of each patient was tiled into image patches of size 2048 by 2048 pixels. This was followed by automated nuclei detection using a watershed-based algorithm and graph- and shape-based feature extraction from the detected nuclei. Targeted tile selection was then performed to select the most representative tiles determined by a dimensionality reduction-based algorithm from each case. TILs were then detected using a Support Vector Machine model followed by extraction of multiple TIL-based features. b Construction of prognostic models using TIL-based features extracted from H&E images using Cox proportional hazards regression model with the least shrinkage and selection operator (LASSO) method as a feature selection tool. c Determination of the molecular composition of prognostic signals identified from H&E images by first co-registering the QIF images and their corresponding tissue microarrays (TMA) and then interrogating the molecular composition of the TIL patterns identified as prognostic from H&E images. d Features-Pathway association. The association between prognostic features from H&E images and (1) biological pathways implicated in immune recognition, response, and evasion, and (2) ISs was studied. e Prediction of response status using TIL density feature-predicted risk categories identified from b in advanced NSCLC patients treated with chemotherapy.