advanced
Volume 19 Issue 5
October  2022
Turn off MathJax
Article Contents
Relative Articles
Qian Cai, Guo-Chong Cui, Hai-Xian Wang. EEG-based Emotion Recognition Using Multiple Kernel Learning. Machine Intelligence Research, vol. 19, no. 5, pp.472-484, 2022. https://doi.org/10.1007/s11633-022-1352-1
Citation: Qian Cai, Guo-Chong Cui, Hai-Xian Wang. EEG-based Emotion Recognition Using Multiple Kernel Learning. Machine Intelligence Research, vol. 19, no. 5, pp.472-484, 2022. https://doi.org/10.1007/s11633-022-1352-1
shu

EEG-based Emotion Recognition Using Multiple Kernel Learning

doi: 10.1007/s11633-022-1352-1
  • 1.

    School of Statistics and Data Science, Nanjing Audit University, Nanjing 211815, China

  • 2.

    Key Laboratory of Child Development and Learning Science of Ministry of Education, School of Biological Science & Medical Engineering, Southeast University, Nanjing 210096, China

  • 3.

    Institute of Artificial Intelligence of Hefei Comprehensive National Science Center, Hefei 230094, China

More Information
  • Author Bio:

    Qian Cai received the B. Sc. and M. Sc. degrees in mathematics from Anhui University, China in 2000 and 2003, respectively. Currently, she is with School of Statistics and Data Science, Nanjing Audit University, China. Her research interests include statistical pattern recognition and data science.E-mail: caiq@nau.edu.cn ORCID iD: 0000-0001-7255-6321

    Guo-Chong Cui received the B. Sc. degree in biomedical engineering from Yanshan University, China in 2019. He is a master student in biomedical engineering at Department of Biomedical Engineering, School of Biological Science & Medical Engineering, Southeast University, China. His research interests include EEG-based emotion recognition, machine learning and neural information processing. E-mail: gc_cui@seu.edu.cn (Corresponding author) ORCID iD: 0000-0002-1929-0154

    Hai-Xian Wang received the B. Sc. and M. Sc. degrees in statistics and the Ph. D. degree in computer science from Anhui University, China in 1999, 2002 and 2005, respectively. Currently, he is with Key Laboratory of Child Development and Learning Science of Ministry of Education, School of Biological Science & Medical Engineering, Southeast University, China. His research interests include biomedical signal processing, brain-computer interfaces, and machine learning. E-mail: hxwang@seu.edu.cn (Corresponding author) ORCID iD: 0000-0001-8220-9737

  • Received Date: 2022-03-31
  • Accepted Date: 2022-06-20
  • Publish Online: 2022-09-03
  • Publish Date: 2022-10-01
    Abstract
  • Emotion recognition based on electroencephalography (EEG) has a wide range of applications and has great potential value, so it has received increasing attention from academia and industry in recent years. Meanwhile, multiple kernel learning (MKL) has also been favored by researchers for its data-driven convenience and high accuracy. However, there is little research on MKL in EEG-based emotion recognition. Therefore, this paper is dedicated to exploring the application of MKL methods in the field of EEG emotion recognition and promoting the application of MKL methods in EEG emotion recognition. Thus, we proposed a support vector machine (SVM) classifier based on the MKL algorithm EasyMKL to investigate the feasibility of MKL algorithms in EEG-based emotion recognition problems. We designed two data partition methods, random division to verify the validity of the MKL method and sequential division to simulate practical applications. Then, tri-categorization experiments were performed for neutral, negative and positive emotions based on a commonly used dataset, the Shanghai Jiao Tong University emotional EEG dataset (SEED). The average classification accuracies for random division and sequential division were 92.25% and 74.37%, respectively, which shows better classification performance than the traditional single kernel SVM. The final results show that the MKL method is obviously effective, and the application of MKL in EEG emotion recognition is worthy of further study. Through the analysis of the experimental results, we discovered that the simple mathematical operations of the features on the symmetrical electrodes could not effectively integrate the spatial information of the EEG signals to obtain better performance. It is also confirmed that higher frequency band information is more correlated with emotional state and contributes more to emotion recognition. In summary, this paper explores research on MKL methods in the field of EEG emotion recognition and provides a new way of thinking for EEG-based emotion recognition research.

     

    • FullText(HTML)
  • loading
    • References(51)
  • [1]
    R. Cowie, E. Douglas-Cowie, N. Tsapatsoulis, G. Votsis, S. Kollias, W. Fellenz, J. G. Taylor. Emotion recognition in human-computer interaction. IEEE Signal Processing Magazine, vol. 18, no. 1, pp. 32–80, 2001. DOI: 10.1109/79.911197.
    [2]
    N. Mehendale. Facial emotion recognition using convolutional neural networks (FERC). SN Applied Sciences, vol. 2, no. 3, Article number 446, 2020. DOI: 10.1007/s42452-020-2234-1.
    [3]
    S. Minaee, M. Minaei, A. Abdolrashidi. Deep-emotion: Facial expression recognition using attentional convolutional network. Sensors, vol. 21, no. 9, Article number 3046, 2021. DOI: 10.3390/s21093046.
    [4]
    D. M. Schuller, B. W. Schuller. A review on five recent and near-future developments in computational processing of emotion in the human voice. Emotion Review, vol. 13, no. 1, pp. 44–50, 2021. DOI: 10.1177/1754073919898526.
    [5]
    Y. M. Huang, K. X. Tian, A. Wu, G. B. Zhang. Feature fusion methods research based on deep belief networks for speech emotion recognition under noise condition. Journal of Ambient Intelligence and Humanized Computing, vol. 10, no. 5, pp. 1787–1798, 2019. DOI: 10.1007/s12652-017-0644-8.
    [6]
    J. H. Tao, J. Huang, Y. Li, Z. Lian, M. Y. Niu. Correction to: Semi-supervised ladder networks for speech emotion recognition. International Journal of Automation and Computing, vol. 18, no. 4, Article number 680, 2021. DOI: 10.1007/s11633-019-1215-6.
    [7]
    Y. Q. Yin, X. W. Zheng, B. Hu, Y. Zhang, X. C. Cui. EEG emotion recognition using fusion model of graph convolutional neural networks and LSTM. Applied Soft Computing, vol. 100, Article number 106954, 2021. DOI: 10.1016/j.asoc.2020.106954.
    [8]
    P. Sarkar, A. Etemad. Self-supervised learning for ecg-based emotion recognition. In Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE, Barcelona, Spain, pp. 3217–3221, 2020. DOI: 10.1109/ICASSP40776.2020.9053985.
    [9]
    V. Kehri, R. Ingle, S. Patil, R. N. Awale. Analysis of facial EMG signal for emotion recognition using wavelet packet transform and SVM. Machine Intelligence and Signal Analysis, M. Tanveer, R. B. Pachori, Eds., Singapore: Springer, pp. 247–257, 2019. DOI: 10.1007/978-981-13-0923-6_21.
    [10]
    Q. Zhang, X. X. Chen, Q. Y. Zhan, T. Yang, S. H. Xia. Respiration-based emotion recognition with deep learning. Computers in Industry, vol. 92–93, pp. 84–90, 2017. DOI: 10.1016/j.compind.2017.04.005.
    [11]
    L. Shu, Y. Yu, W. Z. Chen, H. Q. Hua, Q. Li, J. X. Jin, X. M. Xu. Wearable emotion recognition using heart rate data from a smart bracelet. Sensors, vol. 20, no. 3, Article number 718, 2020. DOI: 10.3390/s20030718.
    [12]
    D. Ayata, Y. Yaslan, M. Kamaşak. Emotion recognition via random forest and galvanic skin response: Comparison of time based feature sets, window sizes and wavelet approaches. In Proceedings of Medical Technologies National Congress, IEEE, Antalya, Turkey, 2016. DOI: 10.1109/TIPTEKNO.2016.7863130.
    [13]
    S. Huang, W. Shao, M. L. Wang, D. Q. Zhang. fMRI-based decoding of visual information from human brain activity: A brief review. International Journal of Automation and Computing, vol. 18, no. 2, pp. 170–184, 2021. DOI: 10.1007/s11633-020-1263-y.
    [14]
    L. M. Ward. Synchronous neural oscillations and cognitive processes. Trends in Cognitive Sciences, vol. 7, no. 12, pp. 553–559, 2003. DOI: 10.1016/j.tics.2003.10.012.
    [15]
    J. A. Coan, J. J. B. Allen. Frontal EEG asymmetry as a moderator and mediator of emotion. Biological Psychology, vol. 67, no. 1–2, pp. 7–50, 2004. DOI: 10.1016/j.biopsycho.2004.03.002.
    [16]
    J. Jin, Z. M. Chen, R. Xu, Y. Y. Miao, X. Y. Wang, T. P. Jung. Developing a novel tactile P300 brain-computer interface with a cheeks-stim paradigm. IEEE Transactions on Biomedical Engineering, vol. 67, no. 9, pp. 2585–2593, 2020. DOI: 10.1109/TBME.2020.2965178.
    [17]
    Y. Yu, Y. D. Liu, E. W. Yin, J. Jiang, Z. T. Zhou, D. W. Hu. An asynchronous hybrid spelling approach based on EEG-EOG signals for Chinese character input. IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 27, no. 6, pp. 1292–1302, 2019. DOI: 10.1109/TNSRE.2019.2914916.
    [18]
    Z. Mohammadi, J. Frounchi, M. Amiri. Wavelet-based emotion recognition system using EEG signal. Neural Computing and Applications, vol. 28, no. 8, pp. 1985–1990, 2017. DOI: 10.1007/s00521-015-2149-8.
    [19]
    M. Degirmenci, M. A. Ozdemir, R. Sadighzadeh, A. Akan. Emotion recognition from EEG signals by using empirical mode decomposition. In Proceedings of Medical Technologies National Congress, IEEE, Magusa, Cyprus, 2018. DOI: 10.1109/TIPTEKNO.2018.8597061.
    [20]
    Z. Yin, L. Liu, J. N. Chen, B. X. Zhao, Y. X. Wang. Locally robust EEG feature selection for individual-independent emotion recognition. Expert Systems with Applications, vol. 162, Article number 113768, 2020. DOI: 10.1016/j.eswa.2020.113768.
    [21]
    T. F. Song, W. M. Zheng, P. Song, Z. Cui. EEG emotion recognition using dynamical graph convolutional neural networks. IEEE Transactions on Affective Computing, vol. 11, no. 3, pp. 532–541, 2020. DOI: 10.1109/TAFFC.2018.2817622.
    [22]
    H. Chao, L. Dong, Y. L. Liu, B. Y. Lu. Emotion recognition from multiband EEG signals using CapsNet. Sensors, vol. 19, no. 9, Article number 2212, 2019. DOI: 10.3390/s19092212.
    [23]
    Y. Cimtay, E. Ekmekcioglu. Investigating the use of pretrained convolutional neural network on cross-subject and cross-dataset EEG emotion recognition. Sensors, vol. 20, no. 7, Article number 2034, 2020. DOI: 10.3390/s20072034.
    [24]
    E. Iáñez, J. M. Azorín, A. Úbeda, E. Fernández, J. L. Sirvent. LDA-based classifiers for a mental tasks-based brain-computer interface. In Proceedings of IEEE International Conference on Systems, Man and Cybernetics, Istanbul, Turkey, pp. 546–551, 2010. DOI: 10.1109/ICSMC.2010.5642018.
    [25]
    L. Guo, Y. X. Wu, L. Zhao, T. Cao, W. L. Yan, X. Q. Shen. Classification of mental task from EEG signals using immune feature weighted support vector machines. IEEE Transactions on Magnetics, vol. 47, no. 5, pp. 866–869, 2011. DOI: 10.1109/TMAG.2010.2072775.
    [26]
    M. J. Abdi, S. M. Hosseini, M. Rezghi. A novel weighted support vector machine based on particle swarm optimization for gene selection and tumor classification. Computational and Mathematical Methods in Medicine, vol. 2012, Article number 320698, 2012. DOI: 10.1155/2012/320698.
    [27]
    A. Rakotomamonjy, F. R. Bach, S. Canu, Y. Grandvalet. SimpleMKL. Journal of Machine Learning Research, vol. 9, pp. 2491–2521, 2008.
    [28]
    M. Varma, B. R. Babu. More generality in efficient multiple kernel learning. In Proceedings of the 26th Annual International Conference on Machine Learning, ACM, Montreal, Canada, pp. 1065–1072, 2009. DOI: 10.1145/1553374.1553510.
    [29]
    M. Kloft, U. Brefeld, S. Sonnenburg, A. Zien. Non-sparse regularization and efficient training with multiple kernels. [Online], Available: https://arxiv.org/abs/1003.0079v1, 2010.
    [30]
    Z. L. Xu, R. Jin, H. Q. Yang, I. King, M. R. Lyu. Simple and efficient multiple kernel learning by group lasso. In Proceedings of the 27th International Conference on International Conference on Machine Learning, Haifa, Israel, pp. 1175–1182, 2010.
    [31]
    F. Aiolli, M. Donini. EasyMKL: A scalable multiple kernel learning algorithm. Neurocomputing, vol. 169, pp. 215–224, 2015. DOI: 10.1016/j.neucom.2014.11.078.
    [32]
    W. Samek, A. Binder, K. R. Müller. Multiple kernel learning for brain-computer interfacing. In Proceedings of the 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE, Osaka, Japan, pp. 7048–7051, 2013. DOI: 10.1109/EMBC.2013.6611181.
    [33]
    X. O. Li, X. Chen, Y. N. Yan, W. S. Wei, Z. J. Wang. Classification of EEG signals using a multiple kernel learning support vector machine. Sensors, vol. 14, no. 7, pp. 12784–12802, 2014. DOI: 10.3390/s140712784.
    [34]
    C. A. Frantzidis, C. Bratsas, C. L. Papadelis, E. Konstantinidis, C. Pappas, P. D. Bamidis. Toward emotion aware computing: An integrated approach using multichannel neurophysiological recordings and affective visual stimuli. IEEE Transactions on Information Technology in Biomedicine, vol. 14, no. 3, pp. 589–597, 2010. DOI: 10.1109/TITB.2010.2041553.
    [35]
    P. J. Lang, M. M. Bradley, B. N. Cuthbert. International Affective Picture System (IAPS): Technical Manual and Affective Ratings, Technical Report No. 3, NIMH Center for the Study of Emotion and Attention, USA, pp. 39–58, 1997.
    [36]
    R. N. Duan, J. Y. Zhu, B. L. Lu. Differential entropy feature for EEG-based emotion classification. In Proceedings of the 6th International IEEE/EMBS Conference on Neural Engineering, IEEE, San Diego, USA, pp. 81–84, 2013. DOI: 10.1109/NER.2013.6695876.
    [37]
    W. L. Zheng, B. L. Lu. Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Transactions on Autonomous Mental Development, vol. 7, no. 3, pp. 162–175, 2015. DOI: 10.1109/TAMD.2015.2431497.
    [38]
    S. Koelstra, C. Muhl, M. Soleymani, J. S. Lee, A. Yazdani, T. Ebrahimi, T. Pun, A. Nijholt, I. Patras. DEAP: A database for emotion analysis; using physiological signals. IEEE Transactions on Affective Computing, vol. 3, no. 1, pp. 18–31, 2012. DOI: 10.1109/T-AFFC.2011.15.
    [39]
    X. Chen, X. Y. Xu, A. P. Liu, S. Lee, X. Chen, X. Zhang, M. J. McKeown, Z. J. Wang. Removal of muscle artifacts from the EEG: A review and recommendations. IEEE Sensors Journal, vol. 19, no. 14, pp. 5353–5368, 2019. DOI: 10.1109/JSEN.2019.2906572.
    [40]
    R. Jenke, A. Peer, M. Buss. Feature extraction and selection for emotion recognition from EEG. IEEE Transactions on Affective Computing, vol. 5, no. 3, pp. 327–339, 2014. DOI: 10.1109/TAFFC.2014.2339834.
    [41]
    C. Zhang, H. Wang, R. R. Fu. Automated detection of driver fatigue based on entropy and complexity measures. IEEE Transactions on Intelligent Transportation Systems, vol. 15, no. 1, pp. 168–177, 2014. DOI: 10.1109/TITS.2013.2275192.
    [42]
    V. S. Vijith, J. E. Jacob, T. Iype, K. Gopakumar, D. G. Yohannan. Epileptic seizure detection using non linear analysis of EEG. In Proceedings of International Conference on Inventive Computation Technologies, IEEE, Coimbatore, India, 2016. DOI: 10.1109/INVENTIVE.2016.7830193.
    [43]
    R. C. Guido. A tutorial review on entropy-based handcrafted feature extraction for information fusion. Information Fusion, vol. 41, pp. 161–175, 2018. DOI: 10.1016/j.inffus.2017.09.006.
    [44]
    K. H. Kim, S. W. Bang, S. R. Kim. Emotion recognition system using short-term monitoring of physiological signals. Medical and Biological Engineering and Computing, vol. 42, no. 3, pp. 419–427, 2004. DOI: 10.1007/BF02344719.
    [45]
    X. W. Wang, D. Nie, B. L. Lu. Emotional state classification from EEG data using machine learning approach. Neurocomputing, vol. 129, pp. 94–106, 2014. DOI: 10.1016/j.neucom.2013.06.046.
    [46]
    K. R. Muller, S. Mika, G. Ratsch, K. Tsuda, B. Scholkopf. An introduction to kernel-based learning algorithms. IEEE Transactions on Neural Networks, vol. 12, no. 2, pp. 181–201, 2001. DOI: 10.1109/72.914517.
    [47]
    X. Chen, Z. J. Wang. Pattern recognition of number gestures based on a wireless surface EMG system. Biomedical Signal Processing and Control, vol. 8, no. 2, pp. 184–192, 2013. DOI: 10.1016/j.bspc.2012.08.005.
    [48]
    C. M. Qing, R. Qiao, X. M. Xu, Y. Q. Cheng. Interpretable emotion recognition using EEG signals. IEEE Access, vol. 7, pp. 94160–94170, 2019. DOI: 10.1109/ACCESS.2019.2928691.
    [49]
    L. C. Shi, Y. Y. Jiao, B. L. Lu. Differential entropy feature for EEG-based vigilance estimation. In Proceedings of the 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE, Osaka, Japan, pp. 6627–6630, 2013. DOI: 10.1109/EMBC.2013.6611075.
    [50]
    M. Soleymani, M. Pantic, T. Pun. Multimodal emotion recognition in response to videos. IEEE Transactions on Affective Computing, vol. 3, no. 2, pp. 211–223, 2012. DOI: 10.1109/T-AFFC.2011.37.
    [51]
    W. L. Zheng, J. Y. Zhu, B. L. Lu. Identifying stable patterns over time for emotion recognition from EEG. IEEE Transactions on Affective Computing, vol. 10, no. 3, pp. 417–429, 2019. DOI: 10.1109/TAFFC.2017.2712143.
    • Relative Articles
    • Supplements(0)
    • Cited By
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(6)  / Tables(4)

    Get Citation
    PDF
    XML
    Entire Issue PDF

    用微信扫码二维码

    分享至好友和朋友圈

    Article Metrics

    Article views (1093) PDF downloads(52) Cited by()
    Proportional views
    Related
    Browse MIR
    Published Online Current Issue Special Issue Archive Selected Papers
    About MIR
    Aims and Scopes Index information Editorial Board Contact Us Springer Page

    For Authors
    Guide for Authors Submission Template Copyright Agreement
    For Referees
    Guide for Referees Referees Acknowledgement

    MIR Event
    MIR News MIR Awards
    WeChat: 机器智能研究MIR
    Twitter: MIR_Journal

    © Institute of Automation, Chinese Academy of Sciences. Published by Springer Nature and Science Press. All rights reserved. 京ICP备14019135号-25

    Supported by Beijing Renhe Information Technology Co. Ltd  support: info@rhhz.net  

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return