Fig. 1: Vision-based multichannel seismocardiography (SCG) acquired by an ordinary smartphone camera.

a Chest videos of 14 subjects were recorded during breath-hold maneuvers at end-inhalation and end-exhalation while a grid of patterned stickers was attached to their chest. b An object detection model was developed to determine the location of the patterned stickers in the first video frame. Consent was obtained for publication of the subject’s chest photograph. c Target tracking and sub-pixel refinement were used to determine the displacement of the sticker grid frame by frame. SCG signals in the right-to-left and head-to-foot directions were then calculated from these displacement signals. d Examples of m × n (=6 × 6) SCG grids. The right-to-left and head-to-foot SCG signals were plotted in blue and green colors, respectively. The magnified inset shows the SCG segments corresponding to cardiac cycles and their ensemble average (the darker segment). e To enhance the resolution of the multichannel SCG, two deep learning models were developed: one to interpolate the signals between two adjacent horizontal SCG signals (an example of the training sample for this model is shown in the red box), and the second one to interpolate the signals between two adjacent vertical SCG signals (a training sample is shown in the green box). f The architecture of the model for SCG grid resolution enhancement. g Examples of enhanced-resolution (2m−1) × (2n−1) SCG grids. The signals in the green boxes were predicted from the adjacent red boxes (either horizontal or vertical). The signals in the yellow boxes were predicted from the two adjacent horizontal green boxes.