Extended Data Fig. 5: Comparison of the PETTOP, MRITOP, PET/MRITOP and PET/MRI classifiers on the clinical test data.
Column-wise, left to right: H&E histology of four CRLM each resected from a different patient, the corresponding phenotypic maps predicted by the PETTOP, MRITOP, PET/MRITOP and PET/MRI classifiers, and the factored probability maps of each phenotype for the PET/MRITOP model. The PET/MRITOP model was trained using the combined training data of the eight most relevant PET/MRI features, shown in Fig. 4g. Whereas the PETTOP and MRITOP models were trained using the relevant features only from the respective modalities. The solid lines partition the presented examples patient-wise and the blue contours in the histology isolate the tumour from the liver tissue. The metastasis in the third row was too large to be processed on a single histology slide, and therefore had to be sectioned into three separate pieces. The dashed black lines in the stitched H&E image show the borders of the separated tumour units. The phenotypic maps are joint probability maps that were colour-coded based on the tumour tissue class colour map shown in the key. Thus, the trained classifiers probabilistically assigned each voxel in the phenotypic maps to either one of the three phenotypic classes. The colour-coded arrows in the histology indicate the respective phenotypic class, whereas the white arrow points towards the tumour region that was falsely classified by all models. All four insets show viable tumour regions within the largely necrotic CRLM. The factored probability maps of the PET/MRI classifier are shown in Fig. 6.
