Abstract

Cognitive abilities are responsible for performing various simple and complex activities that affect a person's mental performance. These are also responsible for different day-to-day actions in human life. In the past few years, studies on cognitive ability, mental performance, and mindfulness meditation have been seen more frequently. The electroencephalogram (EEG) is an effective technique to study brain dynamics while executing any cognitive task and leads to new possibilities in the brain-computer interface (BCI) field. In this study, twenty-seven (27) healthy subjects performed a designed cognitive task having three different states (i.e., rest, meditation, and arithmetic) to stimulate the brain's cognitive functions. BIOPAC-MP-160 has been used for the EEG signal acquisition of the designed cognitive task according to the international 10-20 data acquisition system. The EEGLAB has been used to visualize, pre-process, filter, and removal of noise from the data. Then phase-amplitude coupling is performed to extract the features. After completing the feature extraction, the classification has been performed by three different deep learning approaches, i.e., sequential convolutional network (SCN), multi-branch convolutional network (MBCN), and multi-branch convolutional network-bidirectional long short-term memory network (MBCN-Bi-LSTM). The performance of the different classifications model has been estimated in terms of accuracy, precision, F1 score, and recall. The results demonstrated that MBCN-Bi-LSTM performs better than the SCN and MBCN, with a significant improvement in accuracy of 97.99%. The comparative analysis of the previously used deep learning and machine learning approaches to classify the EEG signal of different brain states substantially indicates that the proposed MBCN-Bi-LSTM model performs better in terms of accuracy and error rate. Also, the computational execution time of the proposed MBCN-Bi-LSTM is found to be less than the previous methods. The proposed classification approach may be utilized in future research to classify the various physiological signals.

Keyword

BCI, EEG, Deep learning, Classification, Cognitive task, Mental state classification.

Cite this article

Yadav H, Maini S

Refference

[1][1]Vansteensel MJ, Pels EG, Bleichner MG, Branco MP, Denison T, Freudenburg ZV et al. Fully implanted brain–computer interface in a locked-in patient with ALS. New England Journal of Medicine. 2016; 375(21):2060-6.

[2][2]Wang H, Chang W, Zhang C. Functional brain network and multichannel analysis for the P300-based brain computer interface system of lying detection. Expert Systems with Applications. 2016; 53:117-28.

[3][3]Fujiwara K, Abe E, Kamata K, Nakayama C, Suzuki Y, Yamakawa T, et al. Heart rate variability-based driver drowsiness detection and its validation with EEG. IEEE Transactions on Biomedical Engineering. 2018; 66(6):1769-78.

[4][4]Gao X, Wang Y, Chen X, Gao S. Interface, interaction, and intelligence in generalized brain–computer interfaces. Trends in Cognitive Sciences. 2021; 25(8):671-84.

[5][5]Hramov AE, Maksimenko VA, Pisarchik AN. Physical principles of brain–computer interfaces and their applications for rehabilitation, robotics and control of human brain states. Physics Reports. 2021; 918:1-33.

[6][6]Yadav D, Yadav S, Veer K. A comprehensive assessment of brain computer interfaces: recent trends and challenges. Journal of Neuroscience Methods. 2020; 346(2020):1-20.

[7][7]Gu X, Cao Z, Jolfaei A, Xu P, Wu D, Jung TP, et al. EEG-based brain-computer interfaces (BCIs): A survey of recent studies on signal sensing technologies and computational intelligence approaches and their applications. IEEE/ACM Transactions on Computational Biology and Bioinformatics. 2021; 18(5):1645-66.

[8][8]Bagheri M, Power SD. Simultaneous classification of both mental workload and stress level suitable for an online passive brain–computer interface. Sensors. 2022; 22(2):1-17.

[9][9]Lohani M, Payne BR, Strayer DL. A review of psychophysiological measures to assess cognitive states in real-world driving. Frontiers in Human Neuroscience. 2019; 13:1-27.

[10][10]Kongwudhikunakorn S, Kiatthaveephong S, Thanontip K, Leelaarporn P, Piriyajitakonkij M, Charoenpattarawut T, et al. A pilot study on visually stimulated cognitive tasks for EEG-based dementia recognition. IEEE Transactions on Instrumentation and Measurement. 2021; 70:1-10.

[11][11]Sharma S, Singh G, Sharma M. A comprehensive review and analysis of supervised-learning and soft computing techniques for stress diagnosis in humans. Computers in Biology and Medicine. 2021; 134(2021):1-19.

[12][12]Keil A, Bernat EM, Cohen MX, Ding M, Fabiani M, Gratton G, et al. Recommendations and publication guidelines for studies using frequency domain and time‐frequency domain analyses of neural time series. Psychophysiology. 2022; 59(5):1-37.

[13][13]Nafea MS, Ismail ZH. Supervised machine learning and deep learning techniques for epileptic seizure recognition using EEG signals-a systematic literature review. Bioengineering. 2022; 9(12):1-35.

[14][14]Wang J, Wang M. Review of the emotional feature extraction and classification using EEG signals. Cognitive Robotics. 2021; 1:29-40.

[15][15]Zhou Y, Xu Z, Niu Y, Wang P, Wen X, Wu X, et al. Cross-task cognitive workload recognition based on EEG and domain adaptation. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2022; 30:50-60.

[16][16]Zheng X, Chen W, You Y, Jiang Y, Li M, Zhang T. Ensemble deep learning for automated visual classification using EEG signals. Pattern Recognition. 2020; 102:1-33.

[17][17]Li G, Lee CH, Jung JJ, Youn YC, Camacho D. Deep learning for EEG data analytics: a survey. Concurrency and Computation: Practice and Experience. 2020; 32(18):1-13.

[18][18]Zhao X, Zhang H, Zhu G, You F, Kuang S, Sun L. A multi-branch 3D convolutional neural network for EEG-based motor imagery classification. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2019; 27(10):2164-77.

[19][19]Qiao W, Bi X. Ternary-task convolutional bidirectional neural turing machine for assessment of EEG-based cognitive workload. Biomedical Signal Processing and Control. 2020; 57:1-13.

[20][20]Abbasi SF, Ahmad J, Tahir A, Awais M, Chen C, Irfan M, et al. EEG-based neonatal sleep-wake classification using multilayer perceptron neural network. IEEE Access. 2020; 8:183025-34.

[21][21]Ha KW, Jeong JW. Motor imagery EEG classification using capsule networks. Sensors. 2019; 19(13):1-20.

[22][22]Saccá V, Campolo M, Mirarchi D, Gambardella A, Veltri P, Morabito FC. On the classification of EEG signal by using an SVM based algorithm. Multidisciplinary Approaches to Neural Computing. 2018:271-8.

[23][23]Suchetha M, Madhumitha R, Sruthi R. Sequential convolutional neural networks for classification of cognitive tasks from EEG signals. Applied Soft Computing. 2021; 111(2021):1-14.

[24][24]Al-fahoum AS, Al-fraihat AA. Methods of EEG signal features extraction using linear analysis in frequency and time-frequency domains. International Scholarly Research Notices. 2014; 2014:1-8.

[25][25]Scally B, Burke MR, Bunce D, Delvenne JF. Resting-state EEG power and connectivity are associated with alpha peak frequency slowing in healthy aging. Neurobiology of Aging. 2018; 71:149-55.

[26][26]Jacobs D, Hilton T, Del CM, Carlen PL, Bardakjian BL. Classification of pre-clinical seizure states using scalp EEG cross-frequency coupling features. IEEE Transactions on Biomedical Engineering. 2018; 65(11):2440-9.

[27][27]Nuamah JK, Seong Y. Support vector machine (SVM) classification of cognitive tasks based on electroencephalography (EEG) engagement index. Brain-Computer Interfaces. 2018; 5(1):1-12.

[28][28]Dutta S, Singh M, Kumar A. Automated classification of non-motor mental task in electroencephalogram based brain-computer interface using multivariate autoregressive model in the intrinsic mode function domain. Biomedical Signal Processing and Control. 2018; 43:174-82.

[29][29]Noshadi S, Abootalebi V, Sadeghi MT, Shahvazian MS. Selection of an efficient feature space for EEG-based mental task discrimination. Biocybernetics and Biomedical Engineering. 2014; 34(3):159-68.

[30][30]Gupta A, Agrawal RK, Kirar JS, Andreu-perez J, Ding WP, Lin CT, et al. On the utility of power spectral techniques with feature selection techniques for effective mental task classification in noninvasive BCI. IEEE Transactions on Systems, Man, and Cybernetics: Systems. 2019; 51(5):3080-92.

[31][31]Gaurav AR, Kumar V. EEG-metric based mental stress detection. Network Biology. 2018; 8(1):25-34.

[32][32]Qayyum A, Khan MA, Mazher M, Suresh M. Classification of EEG learning and resting states using 1d-convolutional neural network for cognitive load assessment. In student conference on research and development (SCOReD) 2018 (pp. 1-5). IEEE.

[33][33]Zhang P, Wang X, Zhang W, Chen J. Learning spatial–spectral–temporal EEG features with recurrent 3D convolutional neural networks for cross-task mental workload assessment. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2018; 27(1):31-42.

[34][34]Jiménez-guarneros M, Gómez-gil P. Custom domain adaptation: a new method for cross-subject, EEG-based cognitive load recognition. IEEE Signal Processing Letters. 2020; 27:750-4.

[35][35]Idowu OP, Ilesanmi AE, Li X, Samuel OW, Fang P, Li G. An integrated deep learning model for motor intention recognition of multi-class EEG signals in upper limb amputees. Computer Methods and Programs in Biomedicine. 2021; 206:1-39.

[36][36]Liu Q, Cai J, Fan SZ, Abbod MF, Shieh JS, Kung Y, et al. Spectrum analysis of EEG signals using CNN to model patient’s consciousness level based on anesthesiologists’ experience. IEEE Access. 2019; 7:53731-42.

[37][37]Schirrmeister RT, Springenberg JT, Fiederer LD, Glasstetter M, Eggensperger K, Tangermann M, et al. Deep learning with convolutional neural networks for EEG decoding and visualization. Human Brain Mapping. 2017; 38(11):5391-420.

[38][38]Lawhern VJ, Solon AJ, Waytowich NR, Gordon SM, Hung CP, Lance BJ. EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces. Journal of Neural Engineering. 2018; 15(5):1-30.

[39][39]Yang Y, Wu Q, Qiu M, Wang Y, Chen X. Emotion recognition from multi-channel EEG through parallel convolutional recurrent neural network. In international joint conference on neural networks 2018 (pp. 1-7). IEEE.

[40][40]Bird JJ, Kobylarz J, Faria DR, Ekárt A, Ribeiro EP. Cross-domain MLP and CNN transfer learning for biological signal processing: EEG and EMG. IEEE Access. 2020; 8:54789-801.

[41][41]Samanta K, Chatterjee S, Bose R. Cross-subject motor imagery tasks EEG signal classification employing multiplex weighted visibility graph and deep feature extraction. IEEE Sensors Letters. 2019; 4(1):1-4.

[42][42]Kwon S. CLSTM: deep feature-based speech emotion recognition using the hierarchical ConvLSTM network. Mathematics. 2020; 8(12):1-19.

[43][43]Sajjad M, Kwon S. Clustering-based speech emotion recognition by incorporating learned features and deep BiLSTM. IEEE Access. 2020; 8:79861-75.

[44][44]Roy Y, Banville H, Albuquerque I, Gramfort A, Falk TH, Faubert J. Deep learning-based electroencephalography analysis: a systematic review. Journal of Neural Engineering. 2019; 16(5):1-38.

[45][45]Amin SU, Alsulaiman M, Muhammad G, Mekhtiche MA, Hossain MS. Deep learning for EEG motor imagery classification based on multi-layer CNNs feature fusion. Future Generation Computer Systems. 2019; 101:542-54.

[46][46]Altuwaijri GA, Muhammad G. A multibranch of convolutional neural network models for electroencephalogram-based motor imagery classification. Biosensors. 2022; 12(1):1-21.

[47][47]Farsi L, Siuly S, Kabir E, Wang H. Classification of alcoholic EEG signals using a deep learning method. IEEE Sensors Journal. 2020; 21(3):3552-60.

[48][48]Ambrosini E, Vallesi A. Asymmetry in prefrontal resting-state EEG spectral power underlies individual differences in phasic and sustained cognitive control. Neuroimage. 2016; 124:843-57.

[49][49]Gong R, Wegscheider M, Mühlberg C, Gast R, Fricke C, Rumpf JJ, et al. Spatiotemporal features of β-γ phase-amplitude coupling in Parkinson’s disease derived from scalp EEG. Brain. 2021; 144(2):487-503.

[50][50]Jurkiewicz GJ, Hunt MJ, Żygierewicz J. Addressing pitfalls in phase-amplitude coupling analysis with an extended modulation index toolbox. Neuroinformatics. 2021; 19:319-45.

[51][51]Tort AB, Komorowski R, Eichenbaum H, Kopell N. Measuring phase-amplitude coupling between neuronal oscillations of different frequencies. Journal of Neurophysiology. 2010; 104(2):1195-210.

[52][52]Hülsemann MJ, Naumann E, Rasch B. Quantification of phase-amplitude coupling in neuronal oscillations: comparison of phase-locking value, mean vector length, modulation index, and generalized-linear-modeling-cross-frequency-coupling. Frontiers in Neuroscience. 2019; 13:1-15.

[53][53]Crandall B, Klein GA, Hoffman RR. Working minds: a practitioners guide to cognitive task analysis. Mit Press; 2006.

[54][54]Salmon P, Stanton NA, Gibbon A, Jenkins D, Walker GH. Human factors methods and sports science: a practical guide. CRC Press; 2009.

[55][55]Selim S, Tantawi MM, Shedeed HA, Badr A. A csp am-ba-svm approach for motor imagery BCI system. IEEE Access. 2018; 6:49192-208.

[56][56]Sreeja SR, Samanta D. Classification of multiclass motor imagery EEG signal using sparsity approach. Neurocomputing. 2019; 368:133-45.

[57][57]Sun Y, Xue B, Zhang M, Yen GG. Completely automated CNN architecture design based on blocks. IEEE transactions on neural networks and learning systems. 2019; 31(4):1242-54.

[58][58]Ma J, Cheng JC, Lin C, Tan Y, Zhang J. Improving air quality prediction accuracy at larger temporal resolutions using deep learning and transfer learning techniques. Atmospheric Environment. 2019; 214:1-10.

[59][59]Hu L, Zhang Z. EEG signal processing and feature extraction. Singapore: Springer Singapore; 2019.

[60][60]Fouladi S, Safaei AA, Mammone N, Ghaderi F, Ebadi MJ. Efficient deep neural networks for classification of Alzheimer’s disease and mild cognitive impairment from scalp EEG recordings. Cognitive Computation. 2022; 14(4):1247-68.

[61][61]Siddiqui F, Mohammad A, Alam MA, Naaz S, Agarwal P, Sohail SS, et al. Deep neural network for EEG signal-based subject-independent imaginary mental task classification. Diagnostics. 2023; 13(4):1-23.

[62][62]Wen D, Li R, Tang H, Liu Y, Wan X, Dong X, et al. Task-state EEG signal classification for spatial cognitive evaluation based on multiscale high-density convolutional neural network. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2022; 30:1041-51.

[63][63]Cui Z, Zheng X, Shao X, Cui L. Automatic sleep stage classification based on convolutional neural network and fine-grained segments. Complexity. 2018; 2018:1-14.

[64][64]Michielli N, Acharya UR, Molinari F. Cascaded LSTM recurrent neural network for automated sleep stage classification using single-channel EEG signals. Computers in Biology and Medicine. 2019; 106:71-81.

[65][65]Nagabushanam P, Thomas GS, Radha S. EEG signal classification using LSTM and improved neural network algorithms. Soft Computing. 2020; 24:9981-10003.

[66][66]Dvorak D, Shang A, Abdel-baki S, Suzuki W, Fenton AA. Cognitive behavior classification from scalp EEG signals. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2018; 26(4):729-39.

[67][67]Yıldırım Ö, Baloglu UB, Acharya UR. A deep convolutional neural network model for automated identification of abnormal EEG signals. Neural Computing and Applications. 2020; 32:15857-68.

[68][68]Ac Çİ, Kaya M, Mishchenko Y. Distinguishing mental attention states of humans via an EEG-based passive BCI using machine learning methods. Expert Systems with Applications. 2019; 134:153-66.

[69][69]Toa CK, Sim KS, Tan SC. Electroencephalogram-based attention level classification using convolution attention memory neural network. IEEE Access. 2021; 9:58870-81.

[70][70]Mohanavelu K, Poonguzhali S, Janani A, Vinutha S. Machine learning-based approach for identifying mental workload of pilots. Biomedical Signal Processing and Control. 2022.