Fig. 3 | Scientific Reports

Fig. 3

From: Using transcripts to refine image based cell segmentation with FastReseg

Fig. 3

Detection and segregation of misassigned transcript groups. (A) XYZ scatter plots of transcript tLLR scores for a flagged example cell with spatial dependency score of 50.54, visualized both in 3D (left) and across multiple 2D Z-slices (right). Each point represents a transcript, colored by transcriptional score under the most probable cell type given the cell’s overall gene expression profile. This visualization highlights the spatial distribution of potentially misassigned transcripts within the cell. (B) XYZ scatter plots of the decision values produced by a Support Vector Machine (SVM) model trained to predict whether a given transcript would have transcript score below the − 2 cutoff given its spatial coordinate within the same cell shown in (A). Negative decision values correspond to below-cutoff poor-fit prediction by the SVM model. In both (A) and (B), the shape of the points corresponds to the classification predicted by the SVM-based spatial modeling of the given cell. Transcripts predicted to have transcript score below cutoff are treated as flagged misassigned transcripts and depicted as hollow diamond points in the XY scatter plots for multiple Z-slices on the right of (A) and (B). The difference in the spatial pattern of transcript tLLR scores and SVM decision values of same cell provides insights into the spatial constraints on predicting misassigned transcripts. (C) Additional examples of cells with varying degrees of spatial dependency on their transcript scores (5.06, 9.01, 12.11, 28.68). For each cell, transcripts predicted as poor-fit with negative decision values by the SVM are further segregated into spatially distinct groups, shown in different colors at bottom panel. These groups indicate the likely origins of those flagged misassigned transcripts from different neighboring source cells, providing a basis for further targeted reassignment and refinement of cell segmentation.

Back to article page