Fig. 1: Schematic of ELLA and simulation results.

a ELLA is a method for modeling the subcellular localization of mRNAs and detecting genes that display spatial variation within cells in high-resolution spatial transcriptomics. ELLA takes as inputs the spatial gene expression data along with the nuclear center and cell segmentation information. It first performs data pre-processing to create a unified cellular coordinate system to anchor diverse cell morphologies. It then fits a nonhomogeneous Poisson process model for each gene to capture its spatial distribution within cells, computes a P value to capture any subcellular expression pattern observed along the cellular radius, and estimates such a pattern in the form of estimated pattern expression intensity and pattern score. ELLA is capable of borrowing information across cells through a joint likelihood framework to substantially improve detection power, while taking advantage of multiple intensity kernel functions to capture the distinct subcellular expression patterns that may be encountered in various biological settings to ensure robust performance. b Quantile-quantile plots of the expected and observed −log10 P values in the baseline null simulation, where gene expression is randomly distributed spatially within cells. ELLA was compared to SPRAWL, Bento, and Wilcox. c Radar plots show the powers in the alternative simulations with multiple cells across eleven symmetric subcellular expression patterns, where gene expression is enriched in specific subcellular regions. ELLA was compared to SPRAWL and Wilcox, and power was evaluated based on 5% FDR. d Radar plots of the power of different methods in the alternative simulations with multiple cells across three asymmetric subcellular expression patterns, where gene expressions exhibit distinct asymmetric patterns. ELLA was compared to SPRAWL and Wilcox, and power was evaluated based on 5% FDR. ELLA’s power for radial punctate setting with piecewise constant kernels is marked with “*”. e Radar plots show the power of different methods in the additional alternative simulations with one cell across five symmetric subcellular expression patterns, where gene expression is enriched in specific subcellular regions within cells. ELLA was compared to Bento, and power was evaluated based on 5% FDR. Source data are provided as a Source data file.