Fig. 1: AOViFT workflow. | Nature Methods

Fig. 1: AOViFT workflow.

From: Fourier-based three-dimensional multistage transformer for aberration correction in multicellular specimens

Fig. 1

a, AOViFT correction. An aberrated 3D volume is preprocessed and cast into a Fourier embedding, which is passed to a 3D vision transformer model to predict the detection wavefront. A DM compensates for this aberration, enabling acquisition of a corrected volume (D, depth; H, height; W, width). b, The Fourier embedding, \({\mathcal{E}}\). The Fourier transform of the 3D volume is embedded into a lower space (\({\mathcal{E}}\in {{\mathbb{R}}}^{\ell \times d\times d}\)), consisting of three amplitude planes (α1, α2, α3) and three phase planes (φ1, φ2, φ3), each of size d × d where d is the Fourier embedding size. c, AOViFT model. The Fourier embedding is input to a dual-stage 3D vision transformer model. At each stage, the Fourier planes are tiled into k patches (Patchify), applying a radially encoded positional embedding to each patch. These patches are passed through n Transformer layers. At the end of each stage, a residual connection is added, and the patches are merged back to the shape matching the stage input (Merge patches). After all stages, the resulting patches are pooled (GlobalAvgPool) and connected with a dense layer to output the z Zernike coefficients.

Back to article page