Enhanced EEG signal classification in brain computer interfaces using hybrid deep learning models

Abstract

Brain-computer interfaces (BCIs) establish a communication pathway between the human brain and external devices by decoding neural signals. This study focuses on enhancing the classification of Motor Imagery (MI) within BCI systems by leveraging advanced machine learning and deep learning techniques. The accurate classification of electroencephalogram (EEG) data is crucial for enhancing BCI performance. The BCI architecture processes electroencephalography signals through three critical stages: data pre-processing, feature extraction, and classification. The research evaluates the performance of five traditional machine learning classifiers- K-Nearest Neighbors (KNN), Support Vector Classifier (SVC), Logistic Regression (LR), Random Forest (RF), and Naive Bayes (NB)-using the “PhysioNet EEG Motor Movement/Imagery Dataset”. This dataset encompasses EEG data from various motor tasks, including both actual and imagined movements. Among the traditional classifiers, Random Forest achieved the highest accuracy of 91%, underscoring its efficacy in motor imagery classification within BCI systems. In addition to conventional approaches, the study also explores deep learning techniques, with Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) networks yielding accuracies of 88.18% and 16.13%, respectively. However, the proposed hybrid model, which synergistically combines CNN and LSTM, significantly surpasses both traditional machine learning and individual deep learning methods, achieving an exceptional accuracy of 96.06%. This substantial improvement highlights the potential of hybrid deep learning models to advance the state of the art in BCI systems, offering a more robust and precise approach to motor imagery classification.

Introduction

Brain-computer interfaces offer a promising avenue to address the challenges faced by individuals with motor impairments. BCIs provide a direct communication pathway between the human brain and external devices, offering individuals with motor disabilities the potential to control devices and regain independence, thus enhancing their quality of life¹.Disabilities affect a substantial portion of the global population, with an estimated 16% experiencing some form of impairment. A disproportionate number (80%) of these individuals reside in developing countries or from global south, rendering them particularly vulnerable to the impacts of natural disasters, climate change, and public health crises². Motor disabilities constitute a significant subset of this population, affecting millions worldwide. Conditions such as Amyotrophic Lateral Sclerosis (ALS) progressively deteriorate motor neurons, leading to severe physical impairments³. In countries like Mexico, where 8 million individuals live with motor disabilities and 2 million face significant limitations, the need for innovative solutions is acute. Stroke is a leading global health crisis, claiming approximately 5 million lives annually and contributing significantly to disability-adjusted life years (DALYs). As the second leading cause of death worldwide, stroke survivors often face the debilitating challenge of upper limb hemiparesis, or partial/complete paralysis of one arm. Despite rehabilitation efforts, a substantial proportion (60–70%) of stroke patients continue to experience persistent arm movement impairments months after the event. This highlights the critical need for innovative rehabilitation approaches to address the ongoing needs of stroke survivors.

Individuals with motor impairments, such as those resulting from paralysis or spinal cord injuries, face significant challenges in performing daily activities. While able-bodied individuals rely on peripherals like keyboards and mice to interact with.

computers, those with motor disabilities require alternative communication and control methods. To address the needs of this population, BCIs have emerged as promising technology. For instance, individuals with severe motor impairments can utilize BCI systems to operate wheelchairs, communicate, or interact with their environment. Table 1 provides an overview of potential BCI user groups, their primary conditions, and the potential benefits of BCI technology.

Among the various applications of Brain-Computer Interfaces (BCI)⁴the classification of Motor Imagery (MI) signals holds particular significance due to its potential to enable individuals with motor disabilities to control external devices using their thoughts alone. This capability not only enhances the quality of life for such individuals but also opens up avenues for novel human-computer interaction paradigms. MI, the mental simulation of movement without actual physical execution, forms the cornerstone of many BCI systems⁵.

Table 1 Patients, diseases, and potential benefits of BCI.

By decoding neural signals associated with motor imagery tasks, BCI systems can translate these intentions into actionable commands, thereby facilitating real-time interaction with computers, prosthetic limbs, or other assistive devices¹⁶. However, the accurate and reliable classification of motor imagery signals remains a formidable challenge due to the inherent complexity of neural data and the variability across individuals.

MI Classification involves identifying and categorizing different types of imagined movements, such as imagining moving the left hand, right hand, or both feet. This process requires sophisticated signal processing and machine learning techniques to interpret the subtle and often noisy signals recorded from the brain, typically through EEG. Accurate classification is crucial for the effectiveness of BCI applications, as it directly impacts the system’s ability to understand and respond to user intentions as.

shown in Fig. 1.

Fig. 1

General BCI architecture.

Motivation

The motivation behind this research stems from the critical need to enhance the quality of life for individuals with motor disabilities through the development of more effective BCI systems. The ability to control external devices using thought alone has profound implications for assistive technology, enabling users to interact with their environment in ways that were previously impossible. Motor imagery-based BCIs offer a promising solution, but the challenge lies in accurately classifying the complex and variable EEG signals associated with imagined movements.

Current gap

• 1. The effectiveness of EEG-based BCIs heavily relies on the availability and quality of data. Many existing approaches struggle with small sample sizes and insufficient data diversity, leading to overfitting and poor generalization. Although data augmentation techniques have been explored, there is a gap in leveraging sophisticated methods like GANs for creating realistic synthetic EEG data that can address these limitations¹⁷.

• 2. Current machine learning methods often rely on basic feature extraction techniques that may not fully capture the complexity of EEG signals. While deep learning approaches have shown promise, they require extensive computational resources and large annotated datasets, which are not always available. There is a need for more efficient algorithms that can enhance feature extraction and classification without demanding excessive computational power¹⁸.

• 3. Deep learning models, although powerful, often operate as black boxes, making it difficult to interpret their decisions and ensure their reliability in clinical settings. There is a need for more interpretable models that provide insights into how they process EEG data and make classification decisions, which is crucial for gaining trust from healthcare practitioners and users¹⁹.

Contributions

• 1. This research presents a novel approach for enhancing the accuracy of EEG-based motor imagery classification by developing and implementing a hybrid deep learning model. The key contributions of this work are outlined as follows:

• 2. Our proposed hybrid model that leverages the strengths of both CNN and LSTM networks. The CNN component excels in extracting spatial features from EEG data, while the LSTM component captures temporal dependencies, making the model highly effective for time-series data like EEG signals.

• 3. The use of GANs is introduced as a mechanism to enhance classification performance. GANs are utilized to generate synthetic data that helps to balance the dataset and improve the generalization ability of the classifiers. This approach contributes to more robust model training and higher accuracy.

• 4. Our paper employs advanced preprocessing and feature extraction techniques, including Wavelet Transform, Riemannian Geometry, PCA, and t-distributed t-SNE. These techniques help in extracting critical features that are essential for improving the classification accuracy of the model.

• 5. The proposed approach achieves a classification accuracy of 96.06%, significantly outperforming traditional machine learning models, which achieved an accuracy of 91%. This substantial improvement demonstrates the effectiveness of the hybrid CNN-LSTM model in handling the complexities of EEG data.

• 6. The training process is optimized by limiting each epoch to 5 s, with the model reaching peak accuracy within just 30–50 epochs. This efficiency in training is crucial for practical applications where computational resources and time are limited.

• 7. The enhanced accuracy of the proposed model has significant implications for real-world applications, particularly in the field of healthcare. Higher accuracy in EEG classification can lead to improved patient outcomes in neurorehabilitation and BCI systems, enabling more precise control and interpretation of motor imagery.

• 8. The paper is structured as follows: Sect. 2 presents the related work, Sect. 3 outlines the methodology, Sect. 4 discusses the experimental results, and Sect. 5 concludes the paper with a summary of findings and potential future directions.

Related work

The field of BCI has seen significant advancements in recent years, particularly in the classification of motor imagery signals²⁰. Motor imagery involves the mental simulation of movement without actual physical execution, and it plays a crucial role in enabling BCI systems to translate neural signals into actionable commands.

Data pre-processing and feature extraction

Effective data pre-processing is vital for accurate motor imagery classification. Pre-processing steps typically include noise reduction, signal normalization, and artifact removal to enhance the quality of EEG signals. Common techniques involve filtering methods such as band-pass filters, which isolate the frequency bands of interest, and Independent Component Analysis (ICA) for artifact removal. Previous studies have highlighted the importance of these steps in improving the signal-to-noise ratio and subsequently the classification performance^21,22. Feature extraction is a critical step that involves transforming raw EEG signals into meaningful features that can be used for classification. Time-domain features such as mean, variance, and higher-order statistical moments have been widely used due to their simplicity and effectiveness. Frequency-domain features, obtained through methods like the Fast Fourier Transform (FFT) and Power Spectral Density (PSD), provide insights into the frequency components of the EEG signals associated with motor imagery tasks^23,24. Recent studies have also explored time-frequency domain features using wavelet transform and Short-Time Fourier Transform (STFT) to capture both temporal and spectral information. These features are essential for distinguishing between different motor imagery states.

Machine learning and deep learning approaches

This research investigates the application of machine learning algorithms for motor imagery classification in BCIs. The study compares the performance of traditional classifiers, including k-Nearest Neighbors (KNN), Support Vector Machines (SVM), Logistic Regression (LR), Random Forest (RF), and Naive Bayes (NB), in decoding neural patterns associated with different motor imagery tasks²⁵. While these algorithms have been successfully employed in previous BCI research, their effectiveness can vary depending on dataset characteristics and specific application requirements. To address potential limitations of traditional machine learning approaches, this study explores the use of deep learning techniques, specifically Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks²⁶. However, initial findings indicate that these models did not surpass the performance of the best traditional classifier. To further enhance classification accuracy, a hybrid approach combining CNN and LSTM is proposed. Traditional EEG data pre-processing and feature extraction commonly employed techniques such as auto-regressive modeling, STFT, Wavelet Transform (WT), and Discrete Wavelet Transform (DWT)²⁷.

Advanced techniques: riemannian geometry and data augmentation

In contrast, our approach employs a comprehensive pre-processing pipeline, including normalization, band-pass filtering, spatial filtering, and artifact removal to ensure clean and standardized EEG signals²⁸. As shown in Fig. 2, the EEG signal demonstrates clear changes before and after preprocessing. For advanced feature extraction, this study utilizes Wavelet Transform and Riemannian Geometry to capture both time-frequency characteristics and the intrinsic geometric structure of the EEG data²⁹. Additionally, dimensionality reduction techniques like Principal Component Analysis (PCA) and t-distributed Stochastic Neighbor Embedding (t-SNE) are applied to reduce computational complexity and enhance feature discrimination³⁰. A key innovation in our approach is the incorporation of Generative Adversarial Networks (GANs)³¹which are used to generate synthetic data, thereby augmenting the training set and improving the robustness and accuracy of the classification model. By integrating these advanced methods into the pre-processing, feature extraction, and data augmentation stages, our approach lays a robust foundation for accurate and efficient EEG signal classification. In Table 2 summary of state-of-the-art models in BCI highlighting contributions, limitations are shown.

Fig. 2

EEG signal difference before and after data pre-processing.

Table 2 Summary of state-of-the-art models in BCI highlighting contributions, limitations, and references.

Methodology

This section delves into the methodological underpinnings of the study, encompassing a detailed exploration of the dataset, Pre-processing, the implementation of feature extraction techniques, and the application of classification algorithms. A comprehensive overview of the experimental design and procedures is provided, facilitating a thorough understanding of the research methodology and its execution.

Proposed algorithm

In this paper, the authors have developed an advanced architecture within a BCI system designed to improve the classification of motor imagery tasks by effectively interpreting the brain signals associated with imagined movements. As illustrated in Fig. 3, the proposed system processes EEG signals by first feeding them into a sophisticated hybrid deep learning model that combines CNN with LSTM networks. This combination is specifically chosen to capitalize on CNNs’ exceptional ability to extract spatial features from multi-channel EEG data and LSTMs’ strength in capturing temporal dependencies over time.

To execute the experiment, the present research initiated the development of this hybrid CNN-LSTM model due to the unique advantages it offers in analyzing complex, non-linear EEG signals. Traditional machine learning algorithms like RF and SVC have been extensively used in previous BCI research and provide a solid benchmark; however, they are often limited by their inability to fully exploit the intricate temporal and spatial patterns inherent in EEG data. In contrast, our CNN-LSTM model was developed to overcome these limitations, leveraging the CNN’s capability to detect spatial patterns across the EEG electrode array and the LSTM’s ability to model the temporal sequence of brain activity during motor imagery.

For model selection, this study thoroughly evaluated various deep learning architectures, our evaluation revealed that while standalone CNNs and LSTMs provide good accuracy, their performance is significantly enhanced when combined into a hybrid model. The hybrid CNN-LSTM architecture outperformed these individual models by effectively capturing the full scope of EEG signal characteristics – from spatial feature maps generated by the CNN to temporal sequences processed by the LSTM. This comprehensive feature extraction and sequence modelling have led to a notable improvement in classification accuracy, making the hybrid model particularly well-suited for BCI applications.

Furthermore, recognizing the importance of data diversity in model training, the research incorporated GANs to augment our dataset with synthetic EEG signals. This augmentation strategy addresses the challenge of limited training data, which is common in EEG studies, by providing additional examples that mimic the real EEG signal distribution. The inclusion of GANs ensures that our CNN-LSTM model is trained on a more robust dataset, leading to better generalization and improved performance in real-world scenarios.

The newly developed hybrid CNN-LSTM network architecture consists of two main components: the first is the CNN module, responsible for multi-domain feature extraction, and the second is the LSTM module, which captures essential temporal dynamics for accurate classification. The use of GANs for data augmentation further enhances the robustness and reliability of the proposed model, resulting in superior classification performance compared to both traditional machine learning models and individual deep learning architectures.

Fig. 3

Workflow of proposed methodology.

Dataset description

In this study, the present work utilized the EEG Motor Movement/Imagery Dataset (version 1.0.0), a publicly accessible dataset hosted on PhysioNet⁴⁴. The data were recorded using the BCI2000 system, which captured 64-channel EEG signals following the international 10–10 system, at a sampling rate of 160 Hz. The dataset initially included 109 participants; however, due to annotation errors, data from six participants (S038, S088, S089, S092, S100, and S104) were excluded, leaving a final sample of 103 participants.

The experimental protocol consisted of 14 runs per participant, with the first two runs serving as baseline measurements—one with eyes open and the other with eyes closed—each lasting one minute. The remaining 12 runs involved four different tasks, each repeated three times. These tasks were as follows:

• Task 1: The participant opened and closed the corresponding fist when a target appeared on the left or right side of the screen, then relaxed.

• Task 2: The participant imagined performing the actions described in Task 1.

• Task 3: The participant opened and closed both fists if the target appeared at the top of the screen, or moved both feet if the target appeared at the bottom, then relaxed.

• Task 4: The participant imagined performing the actions described in Task 3.

Each task was repeated across three runs, resulting in a comprehensive dataset that captured both real and imagined motor movements. Each of these runs was annotated with markers indicating the onset of movement (T1 for left or both hands, T2 for right hand or both feet) and rest periods (T0). The dataset provided a robust foundation for analyzing motor imagery and movement in EEG signals across a large and diverse participant pool.

The research focused on the imagined movement tasks within the EEG Motor Movement/Imagery Dataset, specifically utilizing task combinations (A0) and (B0) across experimental runs 4, 6, 8, 10, 12, and 14. These runs involved participants imagining specific motor actions, with each run comprising three tasks, each lasting 5 s. The original dataset encodes these tasks as follows: T0 denotes the baseline period, T1 corresponds to the imagined movement of the left fist in task combination (A0) and both fists in task combination (B0), and T2 corresponds to the imagined movement of the right fist in task combination (A0) and both feet in task combination (B0). Each experimental run follows a consistent timeline: 5 s of baseline (T0), 5 s of T1, another 5 s of baseline, and 5 s of T2, resulting in a total of 15 baseline segments, 8 T1 segments, and 7 T2 segments per run.

Fig. 4

Illustrates six regions of interest (ROIs) located in close proximity to the sensorimotor cortex. These ROIs, designated as 1 to 6, provide a visual representation of the 64 EEG electrode positions on the scalp. The electrode locations are marked by small circles and labelled according to standard nomenclature. The channels forming each ROI are highlighted in the table (see Table 3 for details).

To facilitate effective classification for our study, the authors restructured the original task annotations into five distinct classes. Specifically, the authors assigned the label ‘E’ to T1 events related to the imagined left fist movement (task combination b), ‘F’ to T1 events related to the imagined movement of both fists (task combination d), ‘G’ to T2 events corresponding to the imagined right fist movement (task combination b), and ‘H’ to T2 events corresponding to the imagined movement of both feet (task combination d). Additionally, this study introduced a fifth class, ‘I,’ representing the baseline (T0) segments.

In Fig. 4. given our study’s focus on classifying imagined motor movements using advanced ML and DL techniques, this study further concentrated on analyzing EEG data from regions over the sensorimotor cortex, informed by previous research indicating the involvement of frontoparietal and central areas during motor imagery⁴⁵. Building on this, we extracted six regions of interest (ROIs) from the dataset, creating six separate subsets of data. Each data segment (a 5-second epoch) was then labelled with one of the five classes (E, F, G, G, I), resulting in a structured input matrix of size 640 × 2, where the two columns represent paired contralateral channels.

Table 3 Specification of channel pairs within each region of interest.

Our main objective was to leverage these datasets in training and evaluating various ML and DL models, with a particular emphasis on our proposed hybrid model that integrates CNNs and LSTM networks. This CNN + LSTM model is designed to enhance classification performance by exploiting CNNs’ capability to extract spatial features from EEG data and LSTMs’ ability to capture temporal dependencies. This hybrid approach provides a robust architecture for accurately classifying complex EEG signals associated with imagined motor tasks, advancing the field of BCIs and neuroimaging analysis.

Pre-processing

The quality of the raw EEG data significantly influences the performance of any BCI system. EEG signals are typically noisy and contain a variety of artifacts, such as eye blinks and muscle movements, which can obscure the neural signals relevant to motor imagery tasks. Therefore, the pre-processing stage in the algorithm is essential to ensure that the subsequent stages operate on clean and reliable data. The traditional methods for EEG data pre-processing and feature extraction have predominantly utilized techniques such as autoregressive (AR) modeling, STFT, WT, and DWT. In contrast, our approach employs a comprehensive pre-processing pipeline, including normalization, bandpass filtering, spatial filtering, and artifact removal to ensure clean and standardized EEG signals. For advanced feature extraction, the research utilizes Wavelet Transform and Riemannian Geometry to capture both time-frequency characteristics and the intrinsic geometric structure of the EEG data. Additionally, dimensionality reduction techniques like PCA and t-distributed Stochastic Neighbor Embedding (t-SNE) are applied to reduce computational complexity and enhance feature discrimination.

A key innovation in our approach is the incorporation of GANs, which are used to generate synthetic data, thereby augmenting the training set and improving the robustness and accuracy of the classification model. By integrating these advanced methods into the preprocessing, feature extraction, and data augmentation stages, our approach lays a robust foundation for accurate and efficient EEG signal classification^46,47,48.

• Normalization: By normalizing the EEG data, the algorithm ensures that each feature contributes equally to the learning process. This step is crucial in preventing features with larger scales from dominating the model’s decision-making process, which could lead to biased and inaccurate classifications.

  $$Xnormalized = \frac{{X - \mu }}{\sigma }$$

  (1)

  where X is the raw data, µ is the mean, and σ is the standard deviation.

• Bandpass filtering: EEG signals contain a wide range of frequencies, but not all of these frequencies are relevant for motor imagery tasks. By applying a bandpass filter (typically within the 0.5–50 Hz range), The algorithm isolates the frequency bands that are most informative for distinguishing between different motor imagery states. This step reduces the noise and irrelevant data, improving the signal-to-noise ratio, which is critical for accurate classification.

  $$\:H\left(s\right)=\:\frac{1}{\sqrt{1+{\left(\frac{s}{\omega\:c}\right)}^{2n}}}\:\:\:$$

  (2)

  where s is the Laplace transform variable, ω_c is the cutoff frequency, and n is the filter order.

• Spatial filtering: The application of spatial filtering, such as CSP, enhances the discriminability of EEG signals by focusing on spatial patterns that differentiate between motor imagery tasks. This technique maximizes the variance between classes, making it easier for classifiers to distinguish between different motor states.

  $$W = \arg \max \frac{{{W^\tau }{X_1}X_1^\tau W}}{{{W^\tau }{X_2}X_2^\tau W}}$$

  (3)

  where X₁ and X₂ and are the covariance matrices of two different classes.

• Artifact removal: EEG data often contain non-neural artifacts, such as those caused by eye blinks or muscle contractions. Techniques like ICA are employed to separate and remove these artifacts, ensuring that the classifiers focus on the neural signals that are truly indicative of motor imagery. This step is vital for improving the robustness and accuracy of the classification process.

  $$X\,=\,AS$$

  (4)

  where X is the observed data, A is the mixing matrix, and S is the source signal matrix (artifacts).

Feature extraction

Feature extraction is a critical step that transforms the pre-processed EEG data into a more informative representation, which captures the underlying neural activity associated with motor imagery. The features extracted during this stage provide the classifiers with the information needed to differentiate between different motor imagery states effectively.

• Wavelet transform: EEG signals are non-stationary, meaning that their statistical properties can change over time. The Wavelet Transform is particularly useful for analyzing such signals because it provides both time and frequency information, allowing the algorithm to capture transient events that are crucial for distinguishing between different motor imagery tasks. By decomposing the EEG signals into various frequency bands, the Wavelet Transform helps capture subtle temporal dynamics that may be missed by traditional Fourier analysis⁴⁶.

  $$x\left( t \right) = \sum\limits_{i = 1}^J {\sum\limits_{k = 1}^k {{c_j},{}_k{\psi _j},{k^{\left( t \right)}}} }$$

  (5)

  where c_j, k are wavelet coefficients, _kψ_j,k(t) are the wavelets, j is the scale, and k is the position.

• Riemannian geometry: The use of Riemannian geometry in feature extraction leverages the intrinsic geometric properties of EEG data. By representing covariance matrices on a Riemannian manifold, the algorithm can better capture the relationships between different EEG channels. This approach is particularly effective in capturing the spatial structure of EEG signals, which is critical for distinguishing between different motor imagery tasks.

  $$\:C=\frac{1}{T}\sum\:_{t=1}^{T}x\left(t\right){x\left(t\right)}^{T}$$

  (6)

• Principal component analysis (PCA): PCA is employed to reduce the dimensionality of the EEG data while retaining the most important features. This step simplifies the data, making it easier for classifiers to process without losing significant information. By focusing on the principal components that account for the most variance in the data, PCA helps the algorithm to emphasize the most informative aspects of the EEG signals.

  Table 4 Extracted features from time and frequency domains.

  $$Z\,=\,XW$$

  (7)

  where Z is the projected data, X is the mean-cantered data, and W contains the eigenvectors of the covariance matrix of X.

• t-SNE (t-distributed stochastic neighbor Embedding): While not typically used for direct classification, t-SNE is valuable for visualizing the high-dimensional EEG data in lower dimensions. This technique can reveal underlying structures in the data, such as clusters corresponding to different motor imagery states, providing insights that can guide the development and tuning of classifiers.

  Given a set of high-dimensional points X1, X2,. . ,Xn, t-SNE constructs a probability distribution Pij that represents the pairwise similarities between points:

  $${P_{ij}} = \frac{{\exp - \frac{{{{\left\| {{x_i} - {x_j}} \right\|}^2}}}{{2\sigma _i^2}}}}{{\sum\nolimits_{k = i} {\exp - \frac{{{{\left\| {{x_j} - {x_k}} \right\|}^2}}}{{2\sigma _i^2}}} }}$$

  (8)

  t-SNE then seeks a lower-dimensional representation y1,y2,. . ,yn that minimizes the divergence between P_{i j} and a similar distribution Q_{i j} defined in the low-dimensional space:

  $$KL(P\left\| Q \right.) = \sum\limits_{i = 1} {\sum\limits_{j = 1} {{P_{ij}}\log \frac{{{P_{ij}}}}{{{Q_{ij}}}}} }$$

  (9)

  Where:

  $${Q_{ij}} = \frac{{{{\left( {1 + {{\left\| {{y_i} - {y_j}} \right\|}^2}} \right)}^{ - 1}}}}{{\sum\nolimits_{k = 1} {{{\left( {1 + {{\left\| {{y_k} - {y_j}} \right\|}^2}} \right)}^{ - 1}}} }}$$

  (10)

Time domain features are statistical measures computed directly from the amplitude values of the EEG signal within predefined time windows as shown in Fig. 5. These features offer valuable insights into the central tendency and dispersion of the neural activity recorded by EEG electrodes. Figure 5 illustrates the extracted features in both the time and frequency domains. Subfigure (a) represents the time-domain features, which include statistical measures such as Mean, Median, Variance, and Standard Deviation (Std-dev). These features are derived directly from EEG signal amplitudes over predefined time windows. The Mean represents the average neural activity, while the Median provides a more robust measure against outliers. Variance quantifies the dispersion of EEG amplitudes, and Standard Deviation indicates the overall spread of values. Subfigure (b) visualizes the frequency-domain features, where power spectral density variations are analyzed to characterize signal behavior in the frequency spectrum. Key extracted features include Mean Frequency, Median Frequency, Variance of Frequency, and Standard Deviation of Frequency, which provide insights into how neural activity is distributed across different frequency bands. These frequency-domain features play a crucial role in understanding rhythmic neural patterns associated with motor imagery tasks. In Table 4. Among the time domain features, statistical measures such as Mean, Median Variance, and Standard Deviation (Std-dev) play a pivotal role in characterizing the temporal dynamics of EEG signals⁴⁷. Here’s a detailed exploration of these features:

Mean: The Mean represents the average amplitude of the EEG signal within a specified epoch or time window. It provides a measure of the central tendency of the signal, indicating the typical magnitude of neural activity during the recording period. A higher mean value suggests overall elevated levels of neural activation, while a lower mean value indicates reduced activity.

$$\:Mean ({\bar{x}})=\frac{1}{N}\sum\:_{i=1}^{N}{x}_{i}$$

(11)

Where N is the number of observations, and x_i represents each individual observation.

Median: The Median is the middle value of the EEG signal when all amplitude values are arranged in ascending order. Unlike the mean, which is influenced by extreme values (outliers), the median is robust to outliers and provides a measure of the central tendency that is less affected by extreme fluctuations in the signal. It offers insights into the typical or representative amplitude value within the data, particularly in the presence of non-Gaussian or skewed distributions.

Variance: Variance quantifies the dispersion or spread of amplitude values around the mean of the EEG signal. It measures the average squared deviation of each amplitude value from the mean, providing information about the variability or scatter of the data points. A higher variance indicates greater variability in neural activity, reflecting fluctuations or changes in the underlying cognitive processes.

Fig. 5

Visualization of frequency and time domain features.

$$\:{s}^{2}=\frac{1}{N-1}\sum\:_{I=1}^{N}{({x}_{i}-\bar{x}})^{2}$$

(12)

Where N is the number of data points in the population, n is the sample size, xi is each individual observation, µ is the population mean, and $\bar{x}$ is the sample mean.

These time domain features are fundamental in EEG analysis, offering quantitative descriptors of the temporal structure of brain activity during motor imagery tasks. By computing and analyzing these statistical measures, imagery, paving the way for the development of effective brain-computer interface systems and rehabilitation protocols.

Why GANs??

In the context of EEG signal processing, GANs can be leveraged to address challenges such as data scarcity and to enhance feature learning. In this study, GANs are employed primarily for data augmentation and feature generation. The EEG dataset used in our research is limited in size, which poses challenges for training deep learning models effectively. By generating synthetic EEG signals, GANs help in creating a more diverse and representative training set, leading to improved generalization of the models²⁷. Additionally, the complex features generated by GANs can capture intricate patterns in EEG data, which are often missed by conventional feature extraction methods. The GAN architecture used in this research consists of a deep convolutional generator and a discriminator network. The generator is designed to learn the distribution of the EEG signals and generate realistic synthetic samples, while the discriminator aims to distinguish between real and generated EEG signals. The GAN was trained for 100 epochs with a batch size of 64, utilizing the Wasserstein GAN architecture with gradient penalty to ensure stable training and convergence. The quality of the synthetic EEG signals generated by the GAN was evaluated using the Fréchet Inception Distance (FID) score, which measures the distance between the real and synthetic data distributions. A lower FID score indicates that the generated EEG signals closely resemble the real signals, validating the effectiveness of the GAN in data augmentation. The incorporation of GAN-generated EEG signals in the training process led to a notable improvement in model performance. Specifically, the hybrid CNN-LSTM model demonstrated a 10% increase in accuracy and a significant reduction in overfitting when trained with the augmented dataset. This highlights the effectiveness of GANs in enhancing the generalization capabilities of deep learning models for EEG classification.

Classification

Classification is the process of categorizing data points into predefined classes. In this paper, a classifier is employed to distinguish between different types of movements: left hand, and right hand. Five ML classification algorithms and two Deep Learning algorithms and one proposed hybrid model were utilized for this purpose:

K-Nearest neighbors (KNN)

KNN is a simple, yet effective, non-parametric algorithm widely used in BCI research. It classifies a sample based on the majority label of its nearest neighbors. Despite its simplicity, KNN often suffers from high computational cost and sensitivity to noisy data.

$$y(x) = \frac{1}{{{K_{i \in N\left( x \right)}}}}\sum\limits_{yi} {}$$

(13)

Support vector classifier (SVC)

SVC is a supervised learning model used for classification. It works by finding the hyperplane that best divides a dataset into classes. The model seeks to maximize the margin between the classes while minimizing classification errors.

Random forest (RF)

RF is an ensemble learning method that constructs multiple decision trees during training and outputs the mode of the classes for classification. It is highly effective in handling large datasets and provides high accuracy. In Fig. 10 Confusion metric is illustrated.

Logistic regression (LR)

Logistic Regression is a linear model used for binary classification. It estimates the probability that a given input belongs to a particular class (e.g., 0 or 1) by fitting a logistic function to the data. The model outputs probabilities between 0 and 1, which can be thresholded to assign class labels.

$$P\left( {y = 1\left| X \right.} \right) = \frac{1}{{1 + {e^{ - {w^T}X}}}}$$

(14)

Naïve Bayes (NB)

Naive Bayes is a probabilistic classifier based on Bayes’ theorem, assuming independence between predictors. Despite its simplicity and the unrealistic independence assumption, it performs well in various classification tasks, especially in text classification.

$$P\left( {C\left| X \right.} \right) = \frac{{P\left( {X\left| C \right.} \right)P\left( C \right)}}{{P\left( X \right)}}$$

(15)

Convolutional neural networks (CNN)

CNNs are deep learning models designed for processing grid-like data such as images. They are particularly effective at extracting spatial features through convolutional layers, which apply filters to input data to capture patterns like edges, textures, and shapes.

$$Y\,=\,f(W \cdot X\,+\,b)$$

(16)

The function f represents the activation function (like ReLU), W represents the filter weights, and b is the bias.

Long short-term memory (LSTM)

LSTM networks are a type of recurrent neural network (RNN) designed to learn long-term dependencies in sequential data. LSTMs overcome the vanishing gradient problem present in standard RNNs by using memory cells that can maintain information over time. The activation function σ controls what information is retained or discarded from the cell state.

$${h_t}\,=\sigma ({W_{hh}}{h_{t - 1}}+{W_{xh}}{x_i}+{b_h})$$

(17)

Proposed hybrid model (CNN + LSTM)

In this paper, this study proposed a novel approach for enhancing the classification accuracy of motor imagery tasks in BCI systems by integrating CNNs and LSTM networks into a hybrid model. This hybrid model is designed to leverage the strengths of both CNNs and LSTMs, making it highly effective in processing and analyzing complex EEG signals associated with imagined motor movements.

Fig. 6

Architecture of proposed hybrid model (CNN + LSTM).

Table 5 Detailed parameters used in the proposed hybrid CNN + LSTM architecture.

As depicted in Fig. 6, the EEG signals are first pre-processed and segmented into time-series data, which are then fed into the CNN module of our hybrid architecture.

The CNN component is responsible for extracting spatial features from the EEG data. Given the ability of CNNs to identify patterns and structures in grid-like data, the present work designed the CNN module with multiple convolutional layers, each followed by a pooling layer to reduce the dimensionality and computational load while retaining the most important features. The filters in the convolutional layers are specifically tuned to capture the distinct patterns present in the EEG signals, which vary in both amplitude and frequency. The detailed architecture of the CNN-LSTM hybrid model is described in Table 5.

To further enhance the model’s ability to classify these features accurately, the extracted spatial features are passed to the LSTM module, which is adept at capturing temporal dependencies. The LSTM network processes the sequential data, learning the temporal relationships between the features extracted by the CNN. This is particularly important for EEG data, where the temporal dynamics play a crucial role in distinguishing between different motor imagery tasks.

In our architecture, the authors experimented with various configurations of convolutional filters and LSTM units to identify the optimal combination for EEG signal analysis. Inspired by previous work on deep learning models, we incorporated both small and large convolutional filters within the CNN to ensure that the model can effectively capture both fine-grained and broad patterns in the data. This dual-filter approach, as shown in Fig. 6, allows the model to achieve a balance between capturing localized details and understanding broader patterns in the EEG signals.

The output of the LSTM module is then passed through a fully connected layer, which serves as the classifier in our hybrid model. This layer aggregates the learned features and outputs the final classification decision, predicting the specific motor imagery task being performed. To optimize the classification performance, this study also experimented with different activation functions and regularization techniques, which helped in reducing overfitting and improving the model’s generalization capabilities. Furthermore, to validate the effectiveness of the proposed hybrid model, the current investigation compared its performance against other classical machine learning algorithms such as SVM, Neural Networks (NN), and RF. The results showed that our CNN + LSTM model consistently outperformed these traditional approaches, underscoring the advantage of combining spatial and temporal feature extraction in a single architecture. This hybrid approach provides a powerful tool for advancing BCI applications, enabling more accurate and reliable control of external devices based on motor imagery tasks.

Comparison experiments

To assess the efficacy of our proposed methods, additional experiments were conducted. The performance of the models was evaluated using several metrics: accuracy, precision, recall, F1-score, Matthew Correlation Coefficient (MCC), and response time are calculated as follows⁴⁸:

$$Accuracy = \frac{{TP + TN}}{{TP + TN + FP + FN}}$$

$$Precision = \frac{{TP}}{{TP + FP}}$$

$$Recall = \frac{{TP}}{{TP + FN}}$$

$$F1{\text{-}}Score = \frac{{2 \times Precision \times Recall}}{{Precision + Recall}}$$

$$\:MCC=\:\sqrt{\frac{TP\:\times\:\:TN-FP\:\times\:\:FN}{\left(TP+FP\right)\times\:\left(TP+FN\right)\times\:\left(TN+FP\right)\times\:(TN+FN)}}$$

Experimental results

In this study, the authors focused on the classification of imaginary movements, specifically distinguishing between two classes: right hand vs. left hand. The objective was to evaluate the performance of various machine learning and deep learning classifiers in correctly identifying these imaginary movements based on EEG signals. Experiments were conducted on a personal computer equipped with a 8GB MacBook silicon based M1 chip processor using Python 3.7 on Google Colab. The classification accuracy, along with other performance metrics such as precision, recall, F1 score, and MCC, was calculated for each classifier to assess their effectiveness. Furthermore, this study proposed a new hybrid model to get a better accuracy for the classification.

Proposed Hybrid model overall classification performance in this study, the authors evaluated the performance of several traditional ML algorithms for classifying EEG signals related to imaginary hand and foot movements. Among the ML algorithms, RF achieved the highest classification accuracy of 91.04%, significantly outperforming the other methods. NB and LR also performed relatively well, with accuracies of 84.23% and 72.54%, respectively. However, KNN and SVM exhibited lower accuracies, with KNN achieving 46.03% and SVM 54.93%. These results highlight the varying effectiveness of traditional ML algorithms in handling EEG signal classification, with ensemble methods like RF showing superior performance.

To further enhance the classification accuracy, this study explored the use of DL algorithms, specifically CNN and LSTM networks. The CNN model demonstrated a strong performance with an accuracy of 88.18%, indicating its effectiveness in capturing spatial patterns within the EEG data. However, the LSTM model, which is typically effective for sequential data, did not perform as well, achieving a low accuracy of 16.13%. This discrepancy suggests that while LSTM networks are powerful for time-series data, the specific nature of the EEG signals in this study may have posed challenges for this model.

Fig. 7

ROC curves for different classifier in ROI.

Given the strengths of CNN in spatial feature extraction and the potential of LSTM in capturing temporal dependencies, this study proposed a hybrid model combining both CNN and LSTM for the classification task. This hybrid model was designed to leverage the complementary strengths of both architectures, aiming to improve the overall classification performance.

The accuracy of the proposed CNN + LSTM hybrid model was significantly higher than all the other models tested, achieving an impressive accuracy of 96.06%. This result demonstrates the effectiveness of integrating CNN and LSTM architectures, with the hybrid model outperforming not only traditional ML algorithms but also standalone deep learning models.

Fig. 8

(a) Visualize the raw EEG data of left- and right-hand vs. (b) visualize the left- and right-hand movement using proposed model.

Proposed hybrid model overall classification performance

Table 6 presents the classification results for the six considered channel combinations (ROIs). The table includes the number of epochs, and average performance metrics computed on the test set across all classes. Lower performance was observed for.

Fig. 9

Using Riemannian distance classifying the left-hand verses right hand movement.

ROIs 1 (95.54%), ROIs 4 (95.39%), and 3 (95.67%), while higher performance was achieved for ROIs 2 (96.02%), 5 (96.04%), and 6 (96.06%). Overall, ROI demonstrated the best performance with an accuracy of 96.06%.

Fig. 10

Confusion metric of (a) Decision Tree, (b) RF, (c) LR, (d) SVM, (e) KNN, and (f) CNN.

As illustrated in the Fig. 7 the ROC curves, the proposed Hybrid LSTM + CNN model achieved the highest AUC of 0.92, indicating its superior performance in classifying EEG signals compared to other models. This is closely followed by Logistic Regression and SVM, both with an AUC of 0.91. The CNN model also performed well, with an AUC of 0.90, showing its strength in handling the spatial features of EEG data.

Random Forest and k-Nearest Neighbors demonstrated moderate performance, with AUCs of 0.86 and 0.87, respectively. The Decision Tree model, with an AUC of 0.85, performed slightly worse than Random Forest and KNN. However, the LSTM model, despite its theoretical strength in modeling sequential data, achieved a relatively low AUC of 0.74, reflecting its difficulty in effectively capturing the temporal dependencies in this particular EEG classification task. The ROC analysis highlights the robustness of the proposed Hybrid LSTM + CNN model, which not only achieved the highest classification accuracy but also demonstrated the best discriminative ability as indicated by its AUC value. This suggests that the combination of LSTM and.

Table 6 Proposed hybrid model training epochs and performance indices of the different rois averaged overall classes.

Fig. 11

Confusion metric of (g) LSTM, and (h) proposed hybrid LSTM + CNN.

CNN effectively captures both the temporal and spatial features of EEG signals, making it a promising approach for EEG-based BCIs.

Table 7 details the performance of each class within ROI. The Hybrid model consistently achieved high accuracy rates (> 95%) across all classes, including the baseline category.

Table 7 Classification task: metrics of ROI for each class.

The proposed hybrid CNN-LSTM model was evaluated using varying epochs and segment lengths and shown in Fig. 12. Table 8 indicate a significant improvement in mean test accuracy with increased epochs and segment length. With 50 epochs and a 5-second segment length, the model achieved a peak mean test accuracy of 96.04%. This demonstrates the effectiveness of the hybrid model in capturing both spatial and temporal features within the data for enhanced classification performance.

The hybrid model has demonstrated exceptional performance in classifying left- and right-hand movements based on EEG data. In the Fig. 8 the t-SNE visualization clearly illustrates distinct and well-separated clusters corresponding to each movement type, indicating a high degree of accuracy and robustness. This outcome suggests that the model has effectively learned to extract relevant features from the EEG signals and utilize them to make precise classifications. Blue represent Left hand and Red represent Right hand.

In this study, this study further analyzed the classification of imaginary movements, specifically focusing on the discrimination between right-hand and left-hand movements using Riemannian geometry. Figure 9 illustrates the Riemannian distances to the “Right hand” and “Left hand” classes for each trial, represented by red circles and blue squares, respectively (Fig. 10).

Fig. 12

Classification task: loss and accuracy graph for proposed model.

Table 8 Impact of epochs and segment length on accuracy for the proposed model.

The scatter plot clearly shows the distribution of Riemannian distances for the two classes. The diagonal dashed line represents the decision boundary where the distance to the “Right hand” class equals the distance to the “Left hand” class. Points above this line are classified as “Right hand,” while points below are classified as “Left hand.”

The results demonstrate a clear separation between the right-hand and left-hand imaginary movements. The majority of the red circles (right-hand movements) are positioned above the diagonal line, indicating a greater Riemannian distance to the “Left hand” class compared to the “Right hand” class. Conversely, the blue squares (left-hand movements) are predominantly below the diagonal line, showing a greater distance to the “Right hand” class.

This separation suggests that the Riemannian distance metric effectively captures the intrinsic differences in the EEG signals associated with the two types of imaginary movements. The clustering of points on either side of the decision boundary indicates that the metric is highly discriminative, with most data points being correctly classified.

The use of Riemannian geometry for classifying EEG signals offers several advantages. It accounts for the non-Euclidean nature of the data, allowing for a more accurate representation of the underlying signal structure. This approach is particularly useful for handling the variability and complexity inherent in EEG data, providing a robust architecture for distinguishing between different classes of movements. In the context of this study, the Riemannian distance analysis has shown to be an effective method for classifying right-hand and left-hand imaginary movements. The results from this analysis support the overall findings of the study, reinforcing the conclusion that advanced mathematical techniques, such as Riemannian geometry, can significantly enhance the performance of BCI systems.

The Decision Tree model achieved a solid accuracy of 0.860. However, the model has a moderate number of false positives (20) and false negatives (25). This indicates that while the model is relatively good at correctly identifying both classes. The Hybrid LSTM + CNN model stands out with the highest accuracy of 0.873 among all the evaluated models. It has a moderate number of false positives (30) and a relatively low number of false negatives (31). While this model combines the strengths of both LSTM and CNN, it still faces challenges in classification accuracy, particularly in reducing the number of misclassifications. However, its balanced performance suggests it has strong potential in distinguishing between the two classes, and with further fine-tuning, it could achieve even better precision and recall.

Traditional classification results

The Logistic Regression model performs well with a decent accuracy of 0.860. However, it has a moderate number of false positives (18) and false negatives (27), suggesting some challenges in correctly classifying instances, particularly in maintaining a balance between precision and recall. The model may need further tuning to reduce these misclassifications. The SVM model exhibits moderate accuracy of 0.842 but suffers from a higher number of false positives (21) and false negatives (29). This indicates potential issues with overfitting or underfitting, impacting the model’s ability to generalize well across different data points.

The KNN model shows slightly lower accuracy at 0.818 compared to other models. It has a considerable number of false positives (21) and a relatively high number of false negatives (38). This suggests that the model may struggle with distinguishing between the two classes, leading to lower precision and recall. The CNN model demonstrates reasonable accuracy at 0.810 but is challenged by a significant number of false positives (28) and false negatives (32). This results in a moderate balance between precision and recall, with room for improvement in classification accuracy. The LSTM model has the lowest accuracy at 0.706 among the evaluated models, with a high number of false positives (39) and false negatives (52). This indicates that the model struggles significantly in correctly classifying instances, particularly in maintaining a balance between the true positive rate and true negative rate.

The performance of the proposed hybrid CNN-LSTM model was evaluated across 50 training epochs, as depicted in Fig. 11. The model’s training and validation loss, along with accuracy metrics, were meticulously tracked to gauge its learning efficacy.

The graph on the left in Fig. 12 illustrates the loss trajectory for both the training and validation datasets. The training loss consistently decreases, starting at approximately 0.04 and steadily declining to under 0.01 by the 50th epoch. This trend indicates that the model effectively minimizes error during training. Similarly, the validation loss follows a decreasing pattern, from around 0.05 at the beginning to about 0.015 by the end of training, demonstrating the model’s capability to generalize well to unseen data without overfitting.

The right side of Fig. 12 showcases the accuracy progression throughout the training process. The training accuracy begins at approximately 85% and reaches just above 96% by the final epoch, indicating a strong model fit to the training data. Concurrently, the validation accuracy improves from an initial 84% to about 96%, reinforcing the robustness and precision of the proposed hybrid CNN-LSTM model in motor imagery classification tasks. These results underscore the hybrid model’s superior performance, significantly surpassing the accuracies achieved by traditional machine learning classifiers, such as the Random Forest, and other deep learning approaches tested in this study.

Fig. 13

Comparative analysis of various state-of-the-art models based on their accuracy percentages. Source: Prepared by the authors based on^24,42.

Comparative analysis

The bar chart in Fig. 13 illustrates a comparative analysis of various state-of-the-art models based on their accuracy percentages. Among the models evaluated, the Proposed Hybrid model CNN + LSTM stands out with the highest accuracy, achieving an impressive 96.06%. This model significantly outperforms other models, including the Random Forest, which also shows strong performance with an accuracy of 91.74%.

Other models, such as the CNN, SVM, and Logistic Regression, exhibit lower accuracies, ranging from 80.81 to 85.16%, indicating a moderate level of performance. The LSTM model, while powerful in certain contexts, shows a lower accuracy of 84.59% compared to the hybrid approach. Traditional models like K-Nearest Neighbors (KNN), Naive Bayes, and Linear Discriminant Analysis (LDA) also demonstrate moderate performance with accuracies close to the 80% mark.

On the lower end of the spectrum, models like Generalized Feedforward Neural Network (GFNN) and Stacked Autoencoder (SAE) show the least accuracy, with 78.37% and 74.18% respectively. This analysis highlights the superior performance of the hybrid CNN + LSTM model, making it the most effective in terms of accuracy among the evaluated models.

Conclusion

This research demonstrates that hybrid architectures, particularly the combination of CNN with LSTM networks, offer significant advantages for motor imagery EEG-based BCI systems. Our proposed Hybrid CNN + LSTM model achieved the highest accuracy of 96.06%, outperforming other models such as Random Forest and traditional machine learning approaches. Notably, this superior performance was achieved with remarkable efficiency, requiring only 30–50 epochs, each lasting just 5 s. This efficiency highlights the model’s ability to rapidly converge while maintaining high accuracy, making it highly suitable for real-time BCI applications. The integration of CNN’s powerful feature extraction capabilities with LSTM’s strength in processing sequential data has proven to be a highly effective approach. However, while our proposed hybrid model demonstrated exceptional accuracy, there is still potential for further refinement and improvement. Future research could explore several avenues to enhance the performance and applicability of the Hybrid CNN + LSTM model. One promising direction is the investigation of alternative network architectures, such as Transformer-based networks or attention mechanisms, which could provide new insights into capturing complex patterns in EEG data. These advanced models might further improve classification accuracy and computational efficiency. Optimizing the preprocessing pipeline is another crucial area for future work. Exploring alternative signal enhancement methods, such as ICA, could improve the quality of input data, leading to better model performance. Finally, the practical implementation of the hybrid model in real-time BCI systems presents an exciting challenge. Future work could involve developing optimized hardware or software solutions to deploy this model in portable or wearable devices, making it accessible for everyday use. Techniques for reducing computational load without sacrificing accuracy will be key in making BCIs more user-friendly and widely adopted.