Detection of human drowsiness by the vestibulo-ocular reflex compensating for heartbeat-induced head perturbations

Abstract

The vestibulo-ocular reflex (VOR) stabilizes vision during head movements by counter-rotating the eyes in the orbits. Although considered one of the simplest reflexes due to its minimal neuronal circuity comprising a 3-neuron arc, previous studies have shown that VOR performance deteriorates in both monkeys and humans when they are drowsy. Given constant head perturbations under dynamic environments, the VOR has been proposed as a viable biomarker for detecting human drowsiness in automobiles and other moving vehicles. However, under stationary environments where exogenous head movements are absent, its applicability has been questioned. In this study, we demonstrate that each heartbeat generates small yet distinctive head movements, and the VOR compensates for these minor head perturbations. Furthermore, we show that the effectiveness of VOR responses varies with the degree of drowsiness, indicating that the VOR can serve as an indicator of drowsiness, even in stationary contexts such as in classrooms and offices.

Introduction

Drowsiness can manifest unexpectedly during activities such as studying, working, or driving, where it may lead to serious consequences, underscoring the importance of early detection of decreased alertness. Drowsiness is associated with changes in autonomic nervous system (ANS) activities, characterized by a shift towards parasympathetic dominance over the sympathetic nervous system. This shift influences the activity of various ANS-regulated organs, prompting the development of methods to assess arousal levels through biological signals from these organs. Such measures include heart rate¹, pulse wave² and pupil diameter^3,4, alongside facial expressions and ocular features like eyelid opening^5,6, blink speed^6,7,8,9, and eye movements^10,11, which have been identified as effective indicators of reduced alertness. Among the indicators of drowsiness, pupil fluctuation, vergence angle fluctuation, and the vestibulo-ocular reflex (VOR) have been identified as early predictors, as they exhibit changes prior to the subjective perception of drowsiness^12,13,14,15. The VOR, in particular, has attracted attention for its potential in detecting and predicting decreased alertness. This oculomotor reflex counter-rotates the eyes within their orbits in response to head movements, thereby stabilizing our vision during active and passive motions. The neuronal circuitry subserving the VOR has been well identified, with its most direct pathway comprising only three neurons, known as the 3-neuron arc¹⁶. This configuration allows for one of the fastest reflexive responses in the human body, achieving a latency of approximately 10 ms^17,18. The VOR is distinguished not only by its ultra-short latency but also by its high efficacy in compensating for head motion. Specifically, the VOR gain defined as the ratio of eye angular velocity to head angular velocity is closely unity in light conditions when humans and non-human primates are in awake states^19,20,21. However, this precise oculomotor reflex has been shown to degrade in monkeys^20,22 and humans^12,14 during states of drowsiness. Interestingly, changes in the VOR gain and its variability precede the conscious awareness of sleepiness in human participants¹², offering a potential predictive measure for drowsiness¹². Given that the VOR is continuously activated by head movements induced by road bumps, as well as by actions such as accelerating, braking, and steering when riding in moving vehicles, it presents a viable method for detecting and predicting drowsiness under such dynamic conditions¹². However, in stationary situations, such as studying in a classroom or working in an office, where external head motions are minimal, the utility of the VOR for drowsiness detection has been questioned. Currently, we address this issue by examining how the VOR responds to minimal head perturbations endogenously induced by heartbeats, a phenomenon previously reported to impair reading abilities in individuals with vestibular disorders²³. We demonstrate that these subtle head movements are compensated for by the VOR in healthy individuals, and that VOR efficacy degrades with increased levels of drowsiness.

Methods

Experiments were conducted with the approval of the Ethical Review Committee of the author’s institution (approval no. 290069, 20220017). Also, all experimental procedures were designed and conducted following the Declaration of Helsinki.

Participants

Fifteen participants (male 12, female 3), all in their twenties (ave. 22±1.8 yo), participated in the experiment. They provided informed participatory consent after the aims and procedures of the experiment were explained. Individuals with myopia or hyperopia used proper soft contact lenses to correct their refractive errors. Some of them were asked to raise their eyelashes with an eyelash curler as needed to obtain stable clear eye images.

Experimental design

Environment

Figure 1 illustrates the experimental setup and the equipment utilized. Participants were positioned in a chair, oriented towards a uniform white wall located 1.0 m in front of them. A compact LED array, serving as a synchronization signal for data collection, was situated on a small table directly before the participant. Additionally, a speaker, positioned behind the participants, emitted cues signaling them to report their level of drowsiness. The experiments were carried out in a dimly lit room, a condition chosen to facilitate the onset of drowsiness while simultaneously ensuring that the participants’ pupil sizes remained within the optimal range for precise detection by the eye-tracking device.

Fig. 1

Schematic diagram of the experimental setup.

Measured signals

We measured electrocardiogram (ECG), respiration (RSP), binocular eye positions and head angular velocity. The ECG was recorded using disposable electrodes (Kendall Medi-Trace 100, Cardinal Health, USA), and the RSP was monitored with a chest band (TR-111 A, Nihon Kohden, Japan). Both signals were amplified and filtered using a biological amplifier (BA1008, Miyuki Giken, Japan), following impedance adjustment via an associated head amplifier. The ECG data were captured at a sampling rate of 1,000 Hz using the Spike2 application (Cambridge Electronics Design, Inc., England) on a PC (dynabook, Toshiba, Japan) through an AD converter (Power1401, Cambridge Electronics Design, Inc., England).

Eye and head movements were quantified using one of two eye tracker models (EyeSeeCam sci, EyeSeeCam sci2, EyeSeeTech, Germany) and recorded on a Mac computer (EyeSeeCam sci: MacBook Pro, EyeSeeCam sci2: MacBook Air, Apple, USA) at sampling frequencies of 256 Hz (EyeSeeCam sci) or 250 Hz (EyeSeeCam sci2), respectively. The EyeSeeCam sci model, in addition to capturing head and eye data at 256 Hz, features a scene camera that records video at 220 frames per second (fps).

To ensure synchronization between EyeSeeCam sci and Spike2 datasets, a white LED light array was activated once. In the EyeSeeCam sci system, this LED light was captured by the scene camera, while the corresponding voltage change was logged by Spike2. For the EyeSeeCam sci2, synchronization between head and eye data and Spike2 recordings was achieved through voltage variations during serial communication facilitated by an Arduino board (Arduino Uno R3, Arduino cc, USA). Data from the EyeSeeCam sci2 was relayed via UDP communication to the PC, which then issued commands to the Arduino. Spike2 documented the voltage changes during Arduino’s serial communication, allowing for synchronization with the EyeSeeCam sci2 data by matching the initial voltage rise recorded in the Spike2 data with the command timing saved in the EyeSeeCam sci2 dataset.

Preparation

First, participants had disposal ECG electrodes affixed in a 3-electrode (Lead II) configuration, facilitating the measurement of the electrocardiogram. Then, a chest band was positioned around the participant’s diaphragm to monitor respiration rates. The EyeSeeCam system, incorporating cameras (EyeSeeCam sci: two for binocular eye tracking and one for scene capture, EyeSeeCam sci2: two for binocular eye tracking), was meticulously adjusted to optimize image quality for each participant. These preparatory procedures were executed under the same dim lighting conditions as the main experiment, ensuring that participants’ visual systems were adequately acclimated to the environment before the study began. Furthermore, we confirmed that the participants were not drowsy at all before the start of the experiment.

Paradigm

Participants were instructed to maintain their gaze on a specific point on the wall directly in front of them for a duration of 12 min. To assess their level of drowsiness, they were required to self-report their current state of alertness every 2 min. These reporting intervals were demarcated by a discrete, low-volume sound cue. The criteria for evaluating drowsiness levels were outlined in Table 1¹², providing a standardized scale for participants to reference when making their assessments.

Table 1 Definition of subjective drowsiness level.

Data analysis

The data acquired from Spike2 and EyeSeeCam were exported to MATLAB (Mathworks) for offline analysis. We focused on the horizontal (yaw-axis) and vertical (pitch-axis) angular velocity data of the head and eyes to assess the VOR in relation to ECG and RSP data. To derive horizontal and vertical eye angular velocities, we differentiated the eye position data and applied a moving average filter, adjusting its order to eliminate measurement noise for individual participant.

EyeSeeCam sci analysis

Initially, video footage from the scene camera was processed to extract individual frames. The first frame displaying the active synchronization LED was pinpointed as a key reference point. This allowed for the alignment of EyeSeeCam and Spike2 data by correlating this visual cue with the corresponding voltage spike detected in the Spike2 dataset. Following this synchronization step, all data were uniformly resampled to a frequency of 250 Hz to facilitate integrated analysis.

EyeSeeCam sci2 analysis

For the EyeSeeCam sci2 setup, the synchronization process involved matching the trigger signal’s timestamp, recorded alongside the EyeSeeCam data, with the onset of the voltage increase noted in the Spike2 data. Given the original recording frequency of 1000 Hz for the Spike2 data, a resampling procedure was applied to standardize the analysis frequency at 250 Hz across all datasets.

Identification of factors inducing head movement

Without exogenous head oscillation, the possible causes of head motion could be respiration as well as heartbeat. Therefore, frequency analysis of ECG, RSP, horizontal head angular velocity, and vertical head angular velocity was performed to determine the cause of head oscillation at rest. First, each data was divided into 2-minute segments. Then, each segment was Fourier transformed (using the fft function in MATLAB) to obtain the amplitude spectrum, and an additive average was calculated.

To confirm the presence of head and eye movements associated with the heartbeat, we calculated the average head and eye angular velocity waveforms elicited by each heartbeat throughout the entire experimental duration (20 min). This process involved calculating the average waveforms of head angular velocity and eye angular velocity, aligned at the peak of each ECG R-wave, over a 1 s interval (spanning from 0.3 s before to 0.7 s after R-wave peaks). Saccades and blinks, which were not the focus of the current study, were omitted from the calculation of the average eye angular velocity waveform. In addition, to visualize changes in head and eye movements over time, the 20-minute data set was divided into 60 s intervals, overlapping by 50 s each. The average head and eye angular velocity waveforms for each segment were then calculated using the same method based on the R-wave peak triggering.

VOR parameters

Head angular velocity and eye angular velocity, after excluding periods of blinks and saccades, were utilized to estimate VOR parameters, specifically VOR gain (${G}_{VOR}$) and standard deviation of residuals (${SD}_{VOR}$). ${G}_{VOR}$ and ${SD}_{VOR}$ were calculate using Eqs. (1) and (2), respectively by means of the least squares method^12,13,24,25.

$${V}_{eye}\left(t\right)={G}_{VOR}\cdot{V}_{head}(t-\tau)+dc+\epsilon\left(t\right)$$

(1)

$${SD}_{VOR}=\sqrt{\frac{1}{N-1}{\sum}_{t=1}^{N}{\epsilon\left(t\right)}^{2}}$$

(2)

In these equations, ${V}_{eye}\left(t\right)$ and ${V}_{head}\left(t\right)$ represent the eye angular velocity and the head angular velocity at time $t$, respectively, τ denotes the delay time of the VOR system (i.e., the latency from head movement to compensatory eye movement), dc is a constant, ε is the residual of the regression model, and N is the number of data points. Normally, the delay time τ is about 10 ms or shorter¹⁸. In our study, conducted at a sampling frequency of 250 Hz, a delay of two to three points should be observable. However, preliminary analysis did not reveal the expected 10 ms delay, nor did it enhance the accuracy of VOR gain estimation, possibly due to noise within the recorded eye angular velocity data. Consequently, τ was set to 0 for the current analyses.

In the estimation process using the least-squares method, data points lying beyond 3σ (three times the standard deviation) were excluded. The estimation was iteratively repeated until the ${G}_{VOR}$value converged within a specified range of plus or minus 10%. If, at the end of this process, the remaining data points constituted less than 30% of the original dataset, the estimation results were considered invalid due to insufficient data for accurate VOR parameter estimation. Additionally, when blinks and saccades were excluded, if over 60% of the data were discarded, the VOR parameters were not determined. This was attributed to suboptimal measurement conditions that precluded precise calculations. Each parameter (${G}_{VOR}$, ${SD}_{VOR}$, dc) was calculated by segmenting the 20 min recording into intervals of either 120 s.

To account for inter-subject variability, VOR parameters were evaluated by calculating the following rates of change:

$${G}_{VOR}^{Norm}\left(t\right)=\frac{{G}_{VOR}\left(t\right)-\overline{{G}_{VOR}}}{\overline{{G}_{VOR}}}\times100$$

(3)

$${SD}_{VOR}^{Norm}\left(t\right)=\frac{{SD}_{VOR}\left(t\right)-\overline{{SD}_{VOR}}}{\overline{{SD}_{VOR}}}\times100$$

(4)

where ${G}_{VOR}$ and ${SD}_{VOR}$ are instantaneous VOR gain and SD calculated using the data from a time window t-120 to t s. $\overline{{G}_{VOR}}$ and $\overline{{SD}_{VOR}}$ are mean of VOR gain and SD, respectively, from the first 30 s. All participants confirmed that they were completely free of drowsiness before the start of the experiment. Therefore, it was natural to assume that they were fully awake for at least the first 30 s of the experiment. Thus, ${G}_{VOR}^{Norm}$ and ${SD}_{VOR}^{Norm}$ are VOR gain and SD calculated for the first 30 s and normalized using the baseline “awake” VOR performance values.

Classification of drowsiness by using VOR parameters

To classify the participants’ drowsiness state (awake and drowsy) by the VOR, a support vector machine (SVM) was used. The SVM model were trained using fitsvm (MATLAB. kernel function: radial basis function) with all participants’ ${G}_{VOR}^{Norm}$ and ${SD}_{VOR}^{Norm}$ data grouped as awake ( ≦ Lv.1) and drowsy ( ≧ Lv.3). The trained SVM model was then used to predict all data (included Lv.2 data) for each participant. Subsequently, the classification success rate for each of awake ( ≦ Lv.1) and drowsy ( ≧ Lv.2) was calculated.

Results

Head motion under stationary environment

Figure 2 A illustrates a segment (10 s) of the entire 20-minute experiment for a representative participant (Participant E), showing from top to bottom the ECG, RSP, horizontal head angular velocity, eye angular position, eye angular velocity, vertical head angular velocity, eye angular position and eye angular velocity. In the horizontal eye position and velocity, the left and right eyes are depicted in orange and magenta, respectively while in the vertical eye position and velocity, they are shown in green and purple, respectively. Additionally, in these eye traces pale colors indicate the raw data, whereas dark colors are those after the removal of saccades and blinks. In the ECG trace, sharp, peaky potions are identified as R waves, which indicate the periods of ventricular excitation. In the RSP trace, the positive and negative directions correspond to inhalation and exhalation, respectively. In this participant the heart rate slightly exceeded 1 beat per second (bps), or 60 beats per minute (bpm) while respiration occurred approximately once every 4 s. In the head angular velocity traces, small oscillations within ± 2 deg/s are observed in both the horizontal (Yaw-axis) and vertical (Pitch-axis) directions.

Fig. 2

Measured data and their frequency spectra. (A) from top to bottom, ECG, respiration (RSP), horizontal head angular velocity, eye angular position, eye angular velocity, vertical head angular velocity, vertical eye angular position, and eye angular velocity for a representative participant (Participant E). In horizontal traces, positive and negative values indicate rightward (CW) and leftward (CCW) angular motion, respectively. In vertical traces, positive and negative values indicate upward and downward angular motion. In the eye position and angular velocity traces, data from the left and right eyes are superimposed, with original unprocessed data shown in pale color and data filtered to eliminate saccades and blinks presented in a darker color. (B) The averaged frequency spectra of ECG and RSP (top panel), and horizontal head angular velocity in brown and vertical velocity in blue (bottom panel) for Participant E. The colored areas—pale red for horizontal head angular velocity and pale blue for vertical head angular velocity—highlight the 60% attenuation intervals from the peak values of the ECG and RSP spectra. (C) Comparison of ECG and RSP frequency components in head angular velocity for all participants. A pair of dots connected by a line indicates data from each participant. Significant differences were observed between the ECG and RSP components in head angular velocity for horizontal (**: $p$< 0.01) and vertical (*: $p$< 0.05) respectively.

Figure 2B shows the amplitude spectra of ECG (black), RSP (gray), horizontal (brown) and vertical head angular velocity (blue) of the same participant. The peak frequency of the RSP is 0.28 Hz, while that of the ECG is 1.37 Hz. In the vertical head angular velocity spectrum, two distinctive peaks corresponding to the RSP and ECG peaks were observed while in the horizontal head angular velocity spectrum, an apparent peak appeared only at the frequency corresponding to the ECG peak. This result indicates that the participant’s head oscillations were mainly due to heartbeats in the horizontal direction and both heartbeats and respiration in the vertical direction.

Figure 2C summarizes magnitudes of respiratory and heartbeat frequency components in the horizontal (brown, left) and vertical (blue, right) head angular velocity spectra of all the 15 participants. The values for these frequency components were calculated as the areas within the upper and lower frequencies determined by 60% of their maximum values (ECG: pale red area, RSP: pale blue area in Fig. 2B). Fourteen out of 15 participants showed a greater magnitude for the heartbeat component than for the respiratory component in horizontal head movement. Similarly, 13 out of 15 participants showed the same trend in vertical head movement. A two-sample t-test (ttest2 function in MATLAB) revealed significant differences between the magnitudes of ECG and RSP components in both horizontal ($p~=~8.2~ \times ~{10^{ - 3}}$) and vertical ($p~=~3.29~ \times ~{10^{ - 2}}$) head angular velocity spectra.

These results indicate that the origins of head motion under the stationary environment in both horizontal and vertical directions can be attributed to heartbeats and respiration, with a predominant contribution from the former in most participants. Additionally, it has been demonstrated that the VOR exhibits high-pass filter characteristics, presenting greater gains or sensitivities at higher frequency head motions²⁰. Given the relatively small magnitude of these head motions under the stationary condition, the current research primarily concentrated on VOR induced by heartbeats.

VOR caused by heartbeats

To confirm that the VOR is elicited in synchrony with heartbeats in the time domain, we calculated the average eye angular velocity waveforms by aligning them at the peak of each R-wave in both horizontal and vertical directions for all participants as shown in Fig. 3A, B, respectively. Each panel represents the result for an individual participant, with the vertical gray dotted line at time 0 indicating the occurrence of R-wave peaks. In Fig. 3A, the averaged horizontal head angular velocity, and left and right eye angular velocities are depicted in brown, orange, and magenta, respectively. Correspondingly, in Fig. 3B, these averaged vertical angular velocities are shown in blue (head), green (left eye), and purple (right eye). Despite variations in individual head motion waveform, the averaged traces consistently show that significant, synchronous changes in head angular velocity occur in conjunction with heartbeats for all participants, in both horizontal and vertical directions. Notably, the eye angular velocity traces of both eyes closely match each other as has been demonstrated previously¹⁹, and mirror (opposite sign) the head angular velocity trace in each participant. This observation indicates that each heartbeat induces a small but distinctive head perturbation which is compensated for by the VOR, thus maintaining visual stability.

Fig. 3

Head and eye movements after heartbeats. (A) Horizontal head (brown) and eye angular velocities (left eye: orange, right eye: magenta) averaged over heartbeats for all participants. The vertical gray dotted line at 0s indicates the peak timing of the R wave in the ECG, which was used for additive averaging. (B) Averaged vertical head (blue) and eye angular velocities (left eye: green, right eye: purple) for all participants in the same format as (A).

Changes in VOR parameters with drowsiness

A illustrates the change in the drowsiness level (Lv., as defined in Table 1¹²) reported by each participant at 2-minute intervals throughout the 20-minute experiment. Grey and black bars denote periods when participants were not drowsy ( ≦ Lv.1) and drowsy ( ≧ Lv.2), respectively. Of the 15 participants, all but Participant A (14 in total) experienced drowsiness at some point during the experiment.

Fig. 4

Changes in drowsiness level, averaged ECG, head and eye angular velocities over the course of the experiment. (A) Drowsiness levels reported by each participant. Gray and black bars indicate before and after perceiving drowsiness, respectively. (B) and (D) Example for a representative participant (Participant E). From top to bottom, 3D (left column) plots and their 2D top views (right column) of averaged ECG, horizontal (B) and vertical (D) head angular velocity, left and right eye angular velocities. (C) Changes in drowsiness levels of this participant.

Figure 4B left panels illustrate the changes in ECG, horizontal head and eye angular velocities averaged over 1-minute interval by aligning at R-wave peaks (designated as time 0) across the 20-minute experimental duration for a representative participant (Participant G). The right panels are their 2D projections with color gradients representing the intensity of the data. Figure 4D shows those for the vertical direction. ECG data in Fig. 4B, D are identical. In Fig. 4C, the change in the drowsiness level of this participant is redisplayed from Fig. 4A along the time axes of Fig. 4B, D.

This participant’s drowsiness level progressively increased from Lv.1 to Lv.6 throughout the experimental session. The averaged ECG and head angular velocity waveforms in both horizontal and vertical directions remained unchanged over the experimental duration, suggesting that heartbeat induced head motion is not affected by drowsiness levels. The eye angular velocity 2D maps in both directions initially resembled mirror images of the head angular velocity 2D maps with brighter periods of the former corresponding to darker periods of the latter, and vice versa. These mirror imaged patterns of head and eye angular velocities confirm the presence of VOR triggered by heartbeats in both horizontal and vertical directions. However, as the experiment progressed and the drowsiness level increased, this inverse relationship became less clear. In the later stages of the experiment the appearance of small, patchy areas in the eye angular velocity 2D maps became increasingly evident, particularly in the vertical direction. They are attributable to a reduction of the number of available data points for calculating the averaged traces. The primary cause of this reduction is the frequent or prolonged eye closure observed as participants became drowsier.

To evaluate the changes in the VOR evoked by heartbeat-induced head motions relative to drowsiness level we characterized it with its gain (${G}_{VOR}$) and variability (${SD}_{VOR}$) as VOR parameters (see Method). Figure 5 illustrates changes in the VOR parameters in the horizontal (B) and vertical directions (C) alongside reported drowsiness levels (A) for two representative participants (Right: Participant E, Left: Participant G). In Fig. 5A, in the same format as Fig. 4A, C, Participant E became aware of drowsiness in 6 min (= Lv.2) with the drowsiness intensifying to Lv.7 by the end of the experiment. Similarly, Participant G perceived drowsiness at 8 min, progressively reaching a peak drowsiness of Lv.6. In Fig. 5B, in both participants, ${G}_{VOR}$ and ${SD}_{VOR}$ derived from their horizontal VOR tend to decrease and increase, respectively as their levels of drowsiness increase over time. In Fig. 5C, similar trends are observed for the vertical VOR. The scatter diagrams in the upper portion of the ${G}_{VOR}$ panels plot eye angular velocity against head angular velocity to exemplify the calculation of VOR parameters in alert and drowsy states indicated by the vertical dotted lines at time 2 and 12 min, respectively. The solid black lines represent regression lines, whose slopes correspond to ${G}_{VOR}$ values, and the dispersion around these regression lines quantifies ${SD}_{VOR}$. The data for vertical VOR parameters for Participant E at 20 min were not included due to an insufficient number of data points for reliable VOR parameter estimation (see Method).

Fig. 5

Changes in VOR parameters over the course of the experiment in two representative participants (Participants E and G). (A) Change in drowsiness levels. (B) and (C): Changes in horizontal (B) and vertical (C) VOR parameters ${G}_{VOR}^{Norm}$and ${SD}_{VOR}^{Norm}$), and exemplified scatter plots of head vs. eye angular velocity to visualize estimated VOR parameters where the slope of the black regression line indicates the estimated ${G}_{VOR}$ and variability around the line indicates ${SD}_{VOR}$.

Figure 6A, B illustrate the relationships between VOR parameters and drowsiness levels for all participants (A: horizontal VOR, B: vertical VOR). The upper and lower panels illustrate those of ${G}_{VOR}$ and ${SD}_{VOR}$, respectively. In the scatter diagrams, solid black regression lines reveal that ${G}_{VOR}$ and ${SD}_{VOR}$ exhibit negative and positive slopes, respectively. These slopes (fitlm function in MATLAB. horizontal ${G_{VOR}}$: −3.16, horizontal $S{D_{VOR}}$: 16.01, vertical ${G_{VOR}}$: −3.12, vertical $S{D_{VOR}}$: 13.96) are significantly different from zero (F-test. horizontal ${G_{VOR}}$: $p~=~6.37~ \times ~{10^{ - 4}}$, horizontal $S{D_{VOR}}$: $p~=~1.4~ \times ~{10^{ - 7}}$, vertical $S{D_{VOR}}$: $p~=~2~ \times ~{10^{ - 8}}$) except for vertical ${G_{VOR}}$ ($p~=~5.79~ \times ~{10^{ - 2}}$), indicating a trend that ${G_{VOR}}$ decreases and $S{D_{VOR}}$ increases as drowsiness levels rise. Note that data points, represented by smaller dots at Lv.5 or higher were excluded from the regression line estimation because of an insufficient number of data points for reliable VOR parameter calculation at these drowsiness levels (see Fig. 4B, D). These findings are consistent with results from a previous study¹², where VOR was evaluated using much larger and faster head movements to simulate experiences in a moving vehicle. Notably, in the previous study, only the vertical VOR was evaluated as it is the predominant direction of head motion caused by road bumps and breaking/accelerating behaviors in vehicles. Furthermore, in the right columns of Fig. 6A, B, the data were divided into instances when participants were not consciously aware of drowsiness (pale red: awake, ≦ Lv.1) and when they were (pale blue: drowsy, Lv.2 to Lv.4), and presented by box plots. In the horizontal VOR (Fig. 6A), compared to when awake, drowsy states exhibited a reduction in ${G_{VOR}}$ (upper panel) and an increase in $S{D_{VOR}}$ (lower panel), both of which were statistically significant (Wilcoxon rank sum test, ranksum function in MATLAB. ${G_{VOR}}$: $p~=~8.75~ \times ~{10^{ - 5}}$, $S{D_{VOR}}$: $p~=~3.19~ \times ~{10^{ - 3}}$). The same was true in the vertical VOR (Fig. 6B) in $S{D_{VOR}}$ ($p~=~1.41~ \times ~{10^{ - 7}}$) while the decrease in ${G_{VOR}}$ ($p~=~3.18~ \times ~{10^{ - 1}}$) was not statistically significant.

Fig. 6

Relationships between drowsiness levels and VOR parameters. (A) and (B) Scatter plots (left column) of drowsiness levels (awake: ≦ Lv.1 in pale red, drowsy: ≧ Lv.2 in pale blue) versus VOR parameters (upper row: ${G}_{VOR}^{Norm}$, lower row: ${SD}_{VOR}^{Norm}$) in horizontal (A) and vertical (B) directions, and box plots (right column) of VOR parameters when awake ( ≦ Lv.1 in pale red) and drowsy (Lv.2 to Lv.4 in pale blue) states. Data from all participants are used. In the scatter plots, black lines indicate regression lines. Data at drowsiness levels Lv.5 or higher (indicated by smaller pale blue dots) were not used to calculate the regression lines. ** indicates $p$< 0.01; for horizontal ${G}_{VOR}^{Norm}$, ${SD}_{VOR}^{Norm}$ and vertical ${SD}_{VOR}^{Norm}$, the regression lines were significantly different from 0, and there were also significant differences between awake ( ≦ Lv.1) and drowsy( ≧ Lv.2) in the mean values of these parameters. (C) and (D): Scatter plots of ${G}_{VOR}^{Norm}$ versus ${SD}_{VOR}^{Norm}$ for horizontal (C) and vertical (D) VOR, and awake (white area) - drowsy (darker area) classification using SVM. Data from awake ( ≦ Lv.1) and drowsy ( ≧ Lv.3) states are indicated in pale red and pale blue, respectively. Data from Lv.2 (light purple dots) were not used to train the SVM model, although they were used to calculate the success rate of classification.

These observations indicate that, while the correlation is less pronounced compared to the relationship identified in the previous study¹² which assessed VOR induced by relatively large head motions, a noticeable trend persists where ${G}_{VOR}$ decreases and ${SD}_{VOR}$ increases in association with drowsiness, even when evaluating the VOR triggered by subtle head movements due to heartbeats. More precisely, it might be possible to determine a resting human’s drowsiness using either horizontal ${G}_{VOR}$, horizontal ${SD}_{VOR}$, vertical ${SD}_{VOR}$, or their combinations.

Awake and drowsy state classification

Finally, we attempted to classify awake and drowsy states using VOR parameters. As an example, we employed a support vector machine (SVM) to classify our data into two classes: awake and drowsy.

Figure 6C is a scatter diagram of ${G}_{VOR}^{Norm}$ and ${SD}_{VOR}^{Norm}$ estimated every 2 min at different drowsiness levels for all participants. Pale red dots represent instances when participants were awake ( ≦ Lv.1) and pale blue dots indicate drowsy states ( ≧ Lv.3). Pale purple dots represent the period when the participants were sure they were drowsy (= Lv.2). Figure 6D illustrates the same diagram for vertical VOR parameters. When awake, ${G}_{VOR}^{Norm}$ and ${SD}_{VOR}^{Norm}$ are not much different from the original baseline awake condition, thus the red dots are distributed near the origin (${G}_{VOR}^{Norm}$, ${SD}_{VOR}^{Norm}$) = (0, 0). In contrast, when drowsy, ${G}_{VOR}^{Norm}$ tends to decrease and ${SD}_{VOR}^{Norm}$ to increase, resulting in the dispersion of blue dots toward the second quadrant. In each diagram, the boundary of the white and gray areas delineates the threshold separating awake states (white area) from drowsy states (darker area) determined by SVM using data from all participants. To facilitate the classification, data at Lv.2 were not used. Table 2 summarizes the accuracy of the classification using this boundary for each participant. Among the 15 participants, the awake states were accurately classified as awake in more than 75% of instances for 11 participants using horizontal VOR parameters, and for 9 participants using vertical VOR parameters. Of the 14 participants who experienced drowsiness during the experiment, the drowsy states were correctly identified as drowsy in more than 75% of instances for 10 participants with horizontal VOR and 8 participants with vertical VOR. Across all participants, the average classification accuracies were: for horizontal VOR, 85.9% for awake states and 80.8% for drowsy states; for vertical VOR, 81.2% for awake states and 71.2% for drowsy states. Thus, the accuracy of discrimination was higher when using horizontal VOR parameters compared to vertical VOR parameters.

Table 2 Classification rate by SVM for each participant when awake and drowsy.

Discussion

Currently, a broad spectrum of physiological signals is being studied as potential markers of alertness, among which vestibulo-ocular reflex (VOR) performance has been identified as notably effective in detecting and predicting drowsiness¹². The method of evaluating VOR is distinguished by its simplicity compared to EEG or ECG techniques, as it does not need electrode placement and only requires the measurement of head and eye movements which can be done with recent portable devices such as a smartphone^26,27. This simplicity is further highlighted by its reliance on merely two parameters: gain (${G}_{VOR}$) and variability (${SD}_{VOR}$). Despite its advantages, the utility of the VOR has been considered limited in environments where external physical stimuli cause head perturbations, given that VOR activation is contingent upon head movement. This implies that while VOR assessment is advantageous in scenarios involving external motion, such as driving, its effectiveness in stationary settings, like desk work, has been questioned. Nevertheless, it has been observed that monkeys exhibit eye movements at frequencies similar to their heartbeat²⁸, suggesting a natural, endogenous subtle head movement. In humans, the body and head movements induced by heartbeats and respiration are detected as ballistocardiogram (BCG)^29,30. The BCG has been utilized to estimate heart rate and respiratory interval, and it has also been shown to be an indicator of sleepiness level^31,32. Additionally, in cases where patients exhibited spontaneous ocular nystagmus during fundus examinations, it was inferred that the lack of VOR contributes to oscillatory vision and retinal oscillations³³. Therefore, our study concentrated on intrinsic head movements stimulated by heartbeats and respiration.

Our results demonstrated that head is moved in both horizontal and vertical directions under stationary condition and the head motion contains frequency components in both respiration (around 0.25 Hz) and heartbeats (around 1 Hz) frequency bands. Notably, the frequency component of heartbeat is greater than that of respiration in most participants. Considering the small head motion at rest and the high-pass-filter characteristics of the VOR system^20,34, our evaluation focused on VOR responses evoked by heartbeat-induced head motions. Our results confirmed that the VOR is activated in both horizontal and vertical directions, and effectively compensates for distinctive head perturbations induced by each heartbeat (Fig. 3). We also verified that the VOR is not an artifact, but it is clearly induced by head movement (see Supplementary Material; Figure S1). The images shown in Supplementary Figure S1 was taken from human participants who gave informed consent for this image to be published in an online open-access publication. This is the first direct demonstration that the VOR maintains our clear vision which can be disrupted every time our heart beats.

Evaluation of the VOR in relation to drowsiness revealed that VOR parameters (${G}_{VOR}$ and ${SD}_{VOR}$) varied with levels of drowsiness (Figs. 4, 5, and 6). This observation aligns with findings from a previous study¹² which focused solely on the vertical VOR under a dynamic environment, such as in moving vehicles. The current study is the first demonstrating that horizontal VOR parameters also reflect human drowsiness under a stationary environment. This distinction was not trivial given that the neuronal circuitries subserving horizontal and vertical VOR are largely segregated in their pathways³⁵. Generally, in conventional video-oculography, horizontal eye movements are measured with less noise compared to the vertical component which is prone to be contaminated by eyelashes and eyelids (see Fig. 4D). Additionally, the current results revealed that both ${G}_{VOR}$ and ${SD}_{VOR}$ of the horizontal VOR exhibited significant differences between awake and drowsy states, while for vertical VOR, only ${SD}_{VOR}$ showed a significant change (Fig. 6). Therefore, under stationary conditions where robust head motions are not present, monitoring horizontal VOR may be preferable for drowsiness detection.

Interestingly, the VOR is regarded as a simple reflex with its shortest pathways for both horizontal and vertical directions involving a 3-neuron arc¹⁶. Yet, the mechanisms by which drowsiness degrades this simple and fundamental oculomotor reflex remain unknow. A possible mechanism involves decreased activity of the Raphe nucleus, a serotonergic brainstem nucleus related to sleep regulation, during states of drowsiness. This reduction in Raphe activity subsequently modulates its target brain regions, including the cerebellum. Notably, the cerebellar flocculus significantly influence the VOR by inhibiting vestibular neurons primarily located in the medial and superior vestibular nuclei in the horizontal and vertical VOR systems, respectively^36,37.

Conclusion

The present study demonstrated that every heartbeat causes distinctive head perturbations in both horizontal and vertical directions, and the VOR is evoked to compensate for this head motion. The VOR parameters (${G}_{VOR}$ and ${SD}_{VOR}$), especially those of horizontal VOR changed significantly with drowsiness levels. These results suggest that the VOR can be a useful biomarker for detecting drowsiness, even in a stationary environment where exogenous head motion does not occur constantly.