Extended Data Fig. 2: Image and transcriptomic representations analysis.

a, Clustering performance on all ST-bank data. Top: clustering performance using transcriptomic embeddings generated from OmiCLIP model before and after training. Bottom: clustering performance usings image embeddings from OmiCLIP model before and after training, and image embeddings generated from UNI and Pro-GigaPath, respectively. The Calinski-Harabasz scores were calculated on the embeddings using the pre-trained OmiCLIP transcriptomic (top) and image (bottom) encoders, evaluated for each organ type. Higher Calinski-Harabasz scores indicate better separation capability between clusters of the embeddings. In the box plots, the middle line represents the median, the box boundaries indicate the interquartile range, and the whiskers extend to data points within 1.5× the interquartile range. Sample sizes are skin: 163, brain: 119, breast: 97, heart: 73, kidney: 73, embryo: 73, others: 64, liver: 57, prostate: 49, spinal cord: 44, ovary: 32, colon: 29, pancreas: 25, lung: 22, tonsil: 18, uterus: 17, adipose: 15, small intestine: 14, and stomach: 12. b, Image and transcriptomic embeddings of the spinal cord, liver cancer, brain cancer, kidney cancer and skin cancer samples. Each row corresponds to a WSI and showcases information from two modalities. The first column are H&E images showing tissue morphology; the second column are the heatmaps of ST data with the colors indicating the ST data clustering using Leiden algorithm (Methods); the third column are the UMAP of image embeddings colored by ST Leiden clusters before and after contrastive learning; the fourth column are the UMAP of transcriptomics embeddings colored by ST Leiden clusters before and after contrastive learning.