Fig. 1: Optimization and real-time schedule of DeepCAD-RT.

a, Model simplification by feature pruning. The total number of model parameters was reduced from ~16.3 million (16,315,585) to ~1.0 million (1,020,337) for higher processing speed and less memory consumption. b, Performance comparison between DeepCAD and DeepCAD-RT. Deployment optimization refers to hardware acceleration by further optimizing the deployment of deep neural networks on GPU cards. An example image sequence of 490 × 490 × 300 (x-y-t) pixels was partitioned into 75 patches (150 × 150 × 150 pixels with 40% overlap) to obtain these performance measurements on the same GPU (GeForce RTX 3090, Nvidia) with one batch size. In total, ~2.53 × 10⁸ pixels flowed through the network. All hyperparameters remained the same except the method. The red dashed line on the right indicates the imaging time (~9.6 s) of the example data; GB, gigabytes. c, Real-time schedule of DeepCAD-RT. The continuous data stream acquired from the microscope acquisition software was packaged into 3D (x-y-t) minibatches and fed into DeepCAD-RT. To maximize the processing speed, three parallel threads were programmed for image acquisition, data processing and display, respectively. For each batch, half of the overlap was discarded to avoid marginal artifacts. Overlapping frames between two consecutive batches are rendered with overlapping colors. d, Schematic of real-time denoising implemented with DeepCAD-RT on a two-photon microscope. Raw noisy data and the corresponding denoised data are displayed synchronously, which will be saved as separated files automatically at the end of the imaging session.