Extended Data Fig. 2: CNN train/test separation, model independence, and flickering.

A, The three arousal states, REM, NREM, and wake, were not evenly distributed in the recorded data (dataset distribution). Training on equal amounts of data from each state, that is, class balancing (training distribution), prevents CNN models from simply learning to predict the most frequent class whenstate information is not clear. B, 24 h of data spanning a complete 12-12 light/dark cycles was included from each animal. Cartoon is a schematic illustrating 24 h of light/dark (top row), and two examples of train/test segmentation of data (bottom rows). CNNs were trained on 18 h of data. Test data comprised six contiguous hours of data spanning a light/dark transition. C, Models were trained using consensus human labels (top row). Independent models were trained and tested in each brain region within each animal (bottom five rows show the predictions of five models, each trained to recapitulate the human labels from data recorded within the brain region indicated on the left). CNNs are well suited to overcome label error, such as when a human score lags or leads a transition. The provided example demonstrates disagreement and model independence surrounding a global state transition (wake into NREM). The y-axis of CNN models indicates instantaneous confidence [0-1] in each state via the proportion of each of three colors at each point in time. D, In contrast to assessment of model accuracy, flickers are extracted from both train and test output. Flickers detected in the training component of data represent CNNs directly disagreeing with training labels (red line indicates example of wake-to-REM flicker). Human labeled data (top row) and corresponding CNN predictions (bottom row) are computed on an interval set by the experimenter. Flickers were extracted and cross validated in both 2.6 s and 1 s interval models.