Table 3 Overview of validation studies using other than five- or four-stage systems

From: Evaluating the performance of wearable EEG sleep monitoring devices: a meta-analysis approach

Study

Device

# of el.a

El. Position

El. Type

# of part.

Age [years]

Part.

Env.

Nights

Ref.

Epochs

Device scoringb

 

ACC

κ

SE

SP

PPV

NPV

F1

MCC

Forehead

Headband

Zhang et al. (2024)65

LANMAO sleep recorder (prototype)65

2

Fp1, Fp2

Wet

34 (M: 16, F: 18)

4.28 ± 1.61 (1-7) days

Healthy

Controlled

1

PSG scored by 2 experts according to AASM

N/R

LANMAO proprietary algorithm

OA (mi.)

0.80c

0.65c

N/R

N/R

N/R

N/R

0.77c

N/R

Wake

0.87c

0.64c

N/R

N/R

N/R

N/R

0.74c

N/R

NREM

0.86c

0.74c

N/R

N/R

N/R

N/R

0.80c

N/R

REM

0.81c

0.63c

N/R

N/R

N/R

N/R

0.74c

N/R

Lucey et al. (2016)67

X4 Sleep Profiler70

2

AF7, AF8 (Fpz as ref)

Dry

29 (M: 17, F: 12)

54 ± 15.7 (25–80)

Sleep disorders

Controlled

1

PSG scored by 2 experts according to AASM

19326

Manual scoring

OA (mi.)

N/R

0.67c

N/R

N/R

N/R

N/R

N/R

N/R

OA (ma.)

0.90d

N/R

0.68d

0.93d

0.66d

0.92d

0.67d

N/R

Wake

0.89c

N/R

0.21c

0.96c

0.29c

0.93c

0.72d

N/R

N1

0.90c

N/R

0.21c

0.96c

0.29c

0.93c

0.24d

N/R

N2&N3

0.88c

N/R

0.87c

0.89c

0.91c

0.84c

0.89d

N/R

REM

0.94c

N/R

0.86c

0.95c

0.76c

0.98c

0.81d

N/R

Ear

                     

In-ear

                     

Nakamura et al. (2017)68

Prototype developed in Goverdovsky et al. (2016)74, (2017)75

4

2 per ear

Wet

4 (M:4, F:0)

25–36

Healthy

Controlled

1 nap

Scalp-EEG scored by an expert according to AASM

293

Machine learning (SVM)

OA (mi.)

0.77

0.65

0.77

0.92

0.77

0.92

0.77

0.65

OA (ma.)

0.88

0.63

0.70

0.91

0.74

0.92

0.72

0.63

Wake

0.90

0.72

0.79

0.93

0.78

0.94

0.79

0.72

N1

0.88

0.50

0.50

0.95

0.66

0.91

0.57

0.51

N2

0.84

0.67

0.89

0.78

0.79

0.89

0.84

0.68

N3

0.92

0.62

0.60

0.97

0.75

0.94

0.67

0.63

Palo et al. (2024)66

IDUN Guardian Development Kit89

2

1 per ear

Dry

10

18–60

Healthy

Controlled

1

PSG scored by 3 experts according to AASM

4568

Manual scoring

Wake

N/R

N/R

0.80c

N/R

0.84c

N/R

0.79c

N/R

NREM

N/R

N/R

0.95c

N/R

0.88c

N/R

0.91c

N/R

REM

N/R

N/R

0.47c

N/R

0.65c

N/R

0.53c

N/R

  1. All evaluation metric values were calculated from confusion matrices provided in the original publications unless indicated otherwise.
  2. ACC accuracy, AASM American Academy of Sleep Medicine, EEG electroencephalogram, El. electrode, Env. environment, F females, F1 F1 score, M males, MCC Matthews correlation coefficient, NPV negative predictive value, NREM non-rapid eye movement, OA (mi.) overall micro-averaged metrics, OA (ma.) overall macro-averaged metrics, Part. participants, PPV positive predictive value, PSG polysomnography, REM rapid eye movement, SE sensitivity, SP specificity, SVM support vector machine, κ Cohen’s kappa, N/R not reported.
  3. aThe number of recording electrodes.
  4. bIn studies reporting manual scoring of wEEG data, scoring was conducted on raw signals from the device. As most wEEG devices lacked EOG and EMG channels, REM sleep was identified based on EEG characteristics alone, such as low-amplitude mixed-frequency activity.
  5. cReported in original study.
  6. dCalculated from reported metrics.
Back to article page