Fig. 1: Overview of MiLoPYP modules for self-supervised cellular content exploration and semi-supervised particle localization.
From: MiLoPYP: self-supervised molecular pattern mining and particle localization in situ
a, The cellular content exploration module learns an embedding for each subtomogram and corresponding patches from the aligned tilt series. The embeddings can be visualized in three different ways: (1) the 2D grid visualization displays patches in a 0–1 grid based on the learned embedding coordinates using a rainbow color scheme (top, middle); (2) the 3D tomogram visualization maps patches back to the 3D tomogram coloring the corresponding center coordinates based on the embedding coordinates (top, right); and (3) an interactive session displays embeddings in 3D space allowing the user to perform interactive selection, visualization and export of training coordinates for specific protein species (bottom). b, MiLoPYP's refinement module performs protein localization using few-shot learning followed by a protein-specific post-processing step that determines particle locations based on their shape. Training is done using partially annotated tomograms produced by the cellular content exploration module. The final output consists of 3D tomogram coordinates for each particle species that can be used for downstream SPT.
