Fig. 1: Translation-integration framework.
From: Combining spatial transcriptomics with tissue morphology
a Feature extraction from spatial transcriptomics and morphology on 10X Visium spatial transcriptomics and H&E57. We use the nth spot in Visium \({x}_{G}^{n}\) as the center to extract the nth image patch \({x}_{M}^{n}\). The modality-specific encoders \({e}_{{\theta }_{G}}\) and \({e}_{{\theta }_{M}}\) will output the gene expression feature vector \({h}_{G}^{n}\) and the morphological feature vector \({h}_{M}^{n}\) for position (in, jn). b Intuition of the framework. By assuming the amount of relevant spatial gene expression information is fixed (solid blue area), we get four different scenarios depending on the morphological features. We express these four scenarios by the quadrants formed when presenting the relevance of the morphological features as the y-axis and their shared information with the gene expression/spatial transcriptomics features as the x-axis.
