Fig. 1: Volumetric localization microscopy (VLM) with deep learning.

a Imaging and processing pipeline of VLM. Single-molecule image sequences are captured with a high-resolution Fourier light-field microscope. Raw frames are preprocessed, cropped, and fed into the volumetric reconstruction network (V-Net) for 3D reconstruction. Output volumes are partitioned into sub-regions for processing with the localization network (L-Net) and then concatenated into final super-resolved volumes. Optional drift correction ensures spatial fidelity. MLA, microlens array; CAM, camera sensor. b Schematic of V-Net, trained to decode 3D emitter positions from 2D light-field projections. Inputs (light-field projections of 3D-distributed simulated beads) are transformed into reconstructed 3D volumes, with reconstruction loss calculated against ground-truth wide-field point-spread functions (lower left block, Simulated beads convolve with WF PSF). c Schematic of L-Net, trained to predict super-resolved 3D emitter locations. Inputs are formed by convolving ground-truth simulated emitter positions with 3D Gaussian ellipsoids (lower left block, Simulated locations convolve with Gauss) that match V-Net output resolution, with localization loss computed between predicted super-resolution and ground-truth volumes.