Fig. 3: Keypoint-MoSeq captures the temporal structure of behavior.

a, Output from four methods applied to the same 2D keypoint dataset. b, Distribution of state durations for each method in a. c, Left: average keypoint change scores (z-scored) around transitions identified by each method. Right: distribution of change scores at the transition point (‘MMper’ refers to MotionMapper). d, Distribution of mouse heights (measured by depth camera) for each unsupervised behavior state. States are classified as rear specific (and given a non-gray color in the plot) if they have median height > 6 cm. e, Accuracy of models trained to predict mouse height from behavior labels showing the distribution of accuracies across N = 10 recordings. f, Bottom: state sequences from keypoint-MoSeq and B-SOiD during a pair of example rears. States are colored as in d. Top: mouse height over time with rears shaded gray. Callouts show depth and IR views of the mouse during two example frames. g, Mouse height aligned to the onsets (solid lines) or offsets (dashed lines) of rear-specific states defined in d, showing mean and 95% confidence of the mean. h, Signals captured from a head-mounted IMU, including absolute 3D head orientation (top) and relative linear acceleration (bottom). Each signal and its rate of change, including angular velocity (ang. vel.) and jerk (the derivative of acceleration), are plotted during a 5-s interval. Figure created with SciDraw under a CC BY 4.0 license. i, IMU signals aligned to the onsets of each behavioral state. Each heat map row represents a state. Line plots show the median across states for angular velocity and jerk (average and standard across N = 10 model fits). Keypoint-MoSeq peaks at a higher value for both signals (P < 0.0005, N = 10, Mann–Whitney U test).