Fig. 1: The architecture of SBeA.
a, Video acquisition for the free-social behaviour test. The camera array is used for behavioural capturing and it is calibrated by the chessboard images. There are two phases for behavioural video capturing including social behaviour test and animal digital identity. Phase 1 captures the videos of free-social interactions of two mice. Phase 2 captures the identities of each mouse in phase 1. b, Data annotation for AI training. The SBeA needs the annotations of multi-animal contour and single-animal pose. c, The multistage artificial neural networks for 3D pose tracking. d, The outputs of 3D pose tracking. The left shows the outputs of AI including video instances, multi-animal poses and multi-animal identities. The centre shows the combination of video instances, multi-animal poses and multi-animal identities with camera calibration parameters for 3D reconstruction with identities. The right shows the visualization of 3D poses with identities. e, Parallel dynamic decomposition of body trajectories. Raw 3D trajectories of two animals can be decomposed into locomotion, non-locomotor movement and body distance. After dynamical temporal decomposition, these three parts are merged as social behaviour motifs for behavioural mapping. f, Social behaviour metric. Social behaviour motifs are clustered and phenotyped according to the distribution in social behaviour space. M1, mouse 1. M2, mouse 2. Mp, mouse with index p. Mq, mouse with index q. Mn, mouse with index n.
