Fig. 3: Even low fidelity 3D models can be used to train performant networks for low-magnification applications such as animal detection. | Nature Communications

Fig. 3: Even low fidelity 3D models can be used to train performant networks for low-magnification applications such as animal detection.

From: replicAnt: a pipeline for generating annotated images of animals in complex environments using Unreal Engine

Fig. 3

a 3D models of a worker and a soldier desert termite (Gnathamitermes sp.) were sculpted and textured from reference images in Blender v3.1. b A digital population comprising 80 workers and 20 soldiers, each with randomised scale, hue, contrast, and saturation, was generated from these models (see Supplementary Table 4), and used within replicAnt to generate a synthetic dataset with 10,000 annotated images. c Examples of image render passes (top row), and bounding box annotations (bottom row). d Test data was recorded in the field, using a Nikon D850 and a Nikkor 18–105 mm lens. e Example frames demonstrate the high precision of a YOLOv4 detector trained exclusively on synthetic data; only a small number of occluded or blurry individuals were missed, and few false positives were produced (confidence threshold of 0.65, and non-maximum suppression of 0.45). f The YOLOv44 network converged after about 20000 iterations, and achieved an Average Precision (AP) of 0.956 ± 0.001, retrieved from returned detections on 1000 hand-annotated frames of 49 freely moving termites (see “Methods”). Error bars (f) indicate the standard deviation of the respective mean AP, computed every 1000 training iterations on the unseen real data with fivefold cross-validation using different withheld synthetic data splits during training. Source data are provided as a Source data file.

Back to article page