SOFisher: reinforcement learning-guided experiment designs for spatial omics

Abstract

Spatial omics technologies enable the precise detection of proteins and RNAs at high spatial resolution. Designing spatial omics experiments requires careful consideration of “what” targets to measure and “where” to position the field of views (FOVs). Current FOV sampling strategies often involve acquiring densely sampled FOVs and stitching them together, which is time-consuming, resource-intensive, and sometimes impossible. To optimize FOV sampling strategies, we propose SOFisher, a reinforcement learning-based framework that harnesses the knowledge gained from the sequence of previously sampled FOVs to guide the selection of the next FOV position, to improve the efficiency of capturing more regions of interest. We rigorously evaluated SOFisher’s performance using comprehensive simulations based on real spatial datasets, and our results clearly demonstrated that SOFisher consistently outperformed the conventional approach across various metrics. SOFisher’s robustness and generalizability were further validated through cross-domain generalization tests and its adaptability to varying FOV sizes. On a real Alzheimer’s Disease (AD) dataset, SOFisher successfully guided the selection of FOVs containing neurofibrillary tangles and amyloid-β plaques in both single and dual target tissue landmark scenarios. Remarkably, with the trained SOFisher policy, the guided experiment design of spatial single-omics on small number of FOVs yielded insights into AD-related cell states, subtypes, and gene programs previously obtained through spatial multi-omics experiments on large tissue slices. We further showcased SOFisher’s applications on a colorectal cancer dataset with complex tissue structures and high heterogeneity. Beyond cell type based targeting, we extended SOFisher’s reward function to maximize gene expression levels across diverse spatial patterns and enhanced its exploration capacity through SOFisherWR (SOFisher With Restart) to comprehensively capture discontinuous target enriched regions. SOFisher has the potential to revolutionize the experiment design of spatial biology.