Fig. 4

Heuristic-based approach to evaluate and reassign poorly fitted transcript groups. (A) High-level workflow for transcript reassignment, involving the identification of direct neighboring cells or groups in space, assessing total transcript tLLR scores under the most probable cell types of both the cell/group itself and its neighbors, and determining refinement actions such as creating new cells, trimming, or merging. (B) Decision tree and detailed criteria for deciding refinement actions based on the analysis of physical and expression contexts with respect to baseline distribution of transcript number and total transcript score of relevant reference cell types observed in the query dataset under the original cell segmentations. The inset plot at lower left corner in (B) shows the XY scatter plot of transcripts within an example pair of merging candidates, where Leiden clustering is performed on transcripts’ spatial coordinates to segregate them into different spatial clusters shown in different colors. The merging interface, as defined by the fraction of query transcripts sharing the same spatial clusters as the transcripts in its merging candidate partner, is used to evaluate whether sufficient physical contact exists between the pair of merging candidates, emphasizing the necessity of spatial constraint on a valid merging event.