Fig. 3: Bidirectional transfer learning-based animal identification.
a, Concept diagram of bidirectional transfer learning-based animal identification. The well-trained segmentation model on multi-animals can be transferred to the single-animal videos, and the well-trained identity recognition model on the single animal can also be transferred to multi-animals. The transfer learning of two models reduces unnecessary manual annotations of animal identities. b, Segmentation model reuse. The left shows an animal being put in the transparent circular open field and the video streams are captured by a camera array. The centre shows the well-trained VisTR is reused for the single animal. The right shows the output of well-trained VisTR on the single animal. c, Single-animal identification model training. The left shows the single-animal instances of multiview are cropped, cascaded and resized to an image. The centre shows the use of EfficientNet as the backbone to train the multi-animal identification classifier. The right shows the identity recognition pattern visualization by LayerCAM. d, Multi-animal segmentation with 3D reprojection. The left shows mask reprojection of each camera view. The right shows the crop, cascade and resize of two animal instances from matched camera view angles. e, Identification model reuse. The well-trained identification model on the single animal can be reused in multi-animal identification. f, Confusion matrix of single-animal identification. g, Feature representation of single-animal identification using t-SNE. h, The sorted validation precision of f. i, The sorted silhouette coefficient of g (mean ± s.d., one-way ANOVA with Dunn’s multiple comparisons test, n = 60, adjusted P values from bottom (M2) to top (M7) are 0.0684, 0.0415, >0.9999, <0.0001, >0.9999, <0.0001, <0.0001, <0.0001 and <0.0001). j, The manual validation precision of multi-animal identification. *P < 0.05, **P < 0.01, ***P < 0.001, ****P < 0.0001.
