Fig. 1: Design of Alpha-LFM.

The principle and architecture of Alpha-LFM and its training strategy. a The stepwise network is designed based on the physical model of the light-field imaging process. The light-field imaging process encodes multidimensional degradation mainly including: (i) the frequency aliasing caused by the limited NA of objectives and sub-apertures of microlens, (ii) dimension compression during light field encoding, and (iii) the noise addition by the camera recording. b Three light-field-aware sub-networks with view-attention denoising, spatial-angular de-aliasing, and VCD 3D reconstruction restoration tasks are designed to disentangle and sequentially solve the complex light-field inversion problem. Multi-stage training data, including De-aliased LF, Clean LF, and Noisy LF, are synthesized from the same 3D SR data using a “physics-embedded hierarchical data synthesis” to guide the training of the sub-networks. The decomposed-progressive optimization strategy ensures the collaboration of the sub-networks when training the multi-stage data. c Direct Alpha-Net reconstruction when the training and testing datasets are of the same type of structure. d The schematic illustrates the adaptive inference strategy of Alpha-LFM. When inferring previously unseen types of structures, experimentally-obtained 2D WF and corresponding LF images of the new structure are used to instantly tune the pre-trained model, making it adaptive to the unseen structures.