Fig. 2: Existing shape space methods are sensitive to orientation, which is not resolved by image prealignment.
a We train a VAE on a synthetic two-class dataset of ellipses (left panel). The classes separate in the embedding space forming nested circles (coloured by class, middle panel). Sampled images in the embedding space (right panel) show that orientation is encoded along the circumference of the nested circles, which means that objects with similar shape but different orientation are highly separated (right panel). b Clustering error rates from a synthetic cell dataset on O2-VAE vs VAE: an error means the cell and its rotated version are assigned to a different cluster. Absolute angles far from 0∘ and 180∘ have higher error, suggesting that orientation sensitivity is the issue (histogram height is mean value, and error bars are 95% confidence intervals, CI from nonparametric bootstrap). The remaining panels are about VAE with prealignment. c The result of a preprocessing algorithm to `prealign' 2d segmented hiPSCs. Given a dataset (left), rotate and flip images (middle) so that objects with similar shape are aligned with each other (right). This helps ensure they have similar learned representations. d We design a test to check whether pairs of images with similar shapes have similar embeddings. We identify similar-shape pairs (Methods), then we identify `embedding errors', which is when a similar-shaped pair is separated in the embedding space. e Example image pairs that are `embedding errors' for prealignment-based methods but not for O2-VAE. They have bad pairwise alignment after prealignment. We show UMAPs for prealign-VAE and O2-VAE and draw lines between the embeddings of the example pairs that are visualised on the left. f Quantitative comparisons of errors with O2-VAE and prealign-VAE for two datasets: hIPSCs23 and HPA3; (centre point is the mean value, and error bars are 95% confidence intervals, CI from nonparametric bootstrap). (Left) if similar-shaped pairs are far in embedding space, they are an `embedding error'. (Right) if similar-shaped pairs are not grouped by clustering, it is a `clustering consistency error'. Prealign-VAE has higher errors than O2-VAE. Source data for (b) and (f) are provided as a Source Data file.
