An improved multi-source domain adaptation network for inter-subject mental fatigue detection based on DANN
-
Kun Chen
,
Zhiyong Liu
Abstract
Objectives
Electroencephalogram (EEG) is often used to detect mental fatigue because of its real-time characteristic and objective nature. However, because of the individual variability of EEG among different individuals, tedious and time-consuming calibration sessions are needed.
Methods
Therefore, we propose a multi-source domain adaptation network for inter-subject mental fatigue detection named FLDANN, which is short for focal loss based domain-adversarial training of neural network. As for mental state feature extraction, power spectrum density is extracted based on the Welch method from four sub-bands of EEG signals. The features of the source domain and target domain are fed into the FLDANN network. The contributions of FLDANN include: (1) It uses the idea of adversarial to reduce feature differences between the source and target domain. (2) A loss function named focal loss is used to assign weights to source and target domain samples
Results
The experiment result shows that when the number of the source domains increases, the classification accuracy of domain-adversarial training of neural network (DANN) gradually decreases and finally tends to be stable. The proposed method achieves an accuracy of 84.10% ± 8.75% on the SEED-VIG dataset and 65.42% ± 7.47% on the self-designed dataset. In addition, the proposed method is compared with other domain adaptation methods and the results show that the proposed method outperforms those state-of-the-art methods.
Conclusions
The result proves that the proposed method is able to solve the problem of individual differences across subjects and to solve the problem of low classification performance of multi-source domain transfer learning.
Funding source: Natural Science Foundation of Hubei Province
Award Identifier / Grant number: 2022CFB896
Funding source: National Natural Science Foundation of China
Award Identifier / Grant number: 52075398
Award Identifier / Grant number: 52275029
-
Research funding: National Natural Science Foundation of China (52275029, 52075398) and Natural Science Foundation of Hubei Province (2022CFB896).
-
Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission.
-
Competing interests: Authors state no conflict of interest.
-
Informed consent: Informed consent was obtained from all individuals included in this study.
-
Ethical approval: The local Institutional Review Board deemed the study exempt from review.
References
1. Tran, Y, Craig, A, Craig, R, Chai, R, Nguyen, H. The influence of mental fatigue on brain activity: evidence from a systematic review with meta-analyses. Psychophysiology 2020;575:e13554. https://doi.org/10.1111/psyp.13554.Search in Google Scholar PubMed
2. Blanco-Díaz, FC, Guerrero-Méndez, CD, Bastos-Filho, T, Jaramillo-Isaza, S, Ruiz-Olaya, AF. Effects of the concentration level, eye fatigue and coffee consumption on the performance of a BCI system based on visual ERP-P300. J Neurosci Methods 2022;382:109722. https://doi.org/10.1016/j.jneumeth.2022.109722.Search in Google Scholar PubMed
3. Li, S, Duan, J, Sun, Y, Sheng, X, Zhu, X, Meng, J. Exploring fatigue effects on performance variation of intensive brain–computer interface practice. Front Neurosci 2021;15:773790. https://doi.org/10.3389/fnins.2021.773790.Search in Google Scholar PubMed PubMed Central
4. Wang, H, Dragomir, A, Abbasi, NI, Li, J, Thakor, NV, Bezerianos, A. A novel real-time driving fatigue detection system based on wireless dry EEG. Cogn Neurodyn 2018;12:365–76. https://doi.org/10.1007/s11571-018-9481-5.Search in Google Scholar PubMed PubMed Central
5. Min, J, Wang, P, Hu, J. Driver fatigue detection through multiple entropy fusion analysis in an EEG-based system. PLoS One 2017;12:e0188756. https://doi.org/10.1371/journal.pone.0188756.Search in Google Scholar PubMed PubMed Central
6. Tuncer, T, Dogan, S, Ertam, F, Subasi, A. A dynamic center and multi threshold point based stable feature extraction network for driver fatigue detection utilizing EEG signals. Cogn Neurodyn 2021;15:223–37. https://doi.org/10.1007/s11571-020-09601-w.Search in Google Scholar PubMed PubMed Central
7. Tuncer, T, Dogan, S, Subasi, A. EEG-based driving fatigue detection using multilevel feature extraction and iterative hybrid feature selection. Biomed Signal Process Control 2021;68:102591. https://doi.org/10.1016/j.bspc.2021.102591.Search in Google Scholar
8. Zhang, X, Liang, W, Ding, T. 2019 individual similarity guided transfer modeling for EEG-based emotion recognition. IEEE int. conf. on bioinformatics and biomedicine; 2019:1156–61 pp.10.1109/BIBM47256.2019.8982972Search in Google Scholar
9. Pan, SJ, Yang, QA. Survey on transfer learning. IEEE Trans Knowl Data Eng. 2010;22:1345–59. https://doi.org/10.1109/tkde.2009.191.Search in Google Scholar
10. Pan, SJ, Tsang, IW, Kwok, JT, Yang, Q. Domain adaptation via transfer component analysis. IEEE Trans Neural Network 2011;22:199–210. https://doi.org/10.1109/tnn.2010.2091281.Search in Google Scholar
11. Redko, I, Morvant, E, Habrard, A, Sebban, M, Bennani, Y. A survey on domain adaptation theory: learning bounds and theoretical guarantees. arXiv preprint arXiv; 2020:11829 p.10.1016/B978-1-78548-236-6.50002-7Search in Google Scholar
12. Lou, XG, Chen, L, Song, ZZ. EEG emotion recognition based on multi-source domain transfer learning. Comput Eng Design 2020;41:2011–8.Search in Google Scholar
13. Liu, Y, Lan, Z, Cui, J, Sourina, O, Müller-Wittig, W. Inter-subject transfer learning for EEG-based mental fatigue recognition. Adv Eng Inf 2020;46:101157. https://doi.org/10.1016/j.aei.2020.101157.Search in Google Scholar
14. Wu, D, Lawhern, VJ, Gordon, S, Lance, BJ, Lin, CT. Driver drowsiness estimation from eeg signals using online weighted adaptation regularization for regression (owarr). IEEE Trans Fuzzy Syst 2017;25:1522–35. https://doi.org/10.1109/tfuzz.2016.2633379.Search in Google Scholar
15. Chen, L, Zhang, A, Lou, X. Cross-subject driver status detection from physiological signals based on hybrid feature selection and transfer learning. Expert Syst Appl 2019;137:266–80.10.1016/j.eswa.2019.02.005Search in Google Scholar
16. Healey, JA, Picard, RW. Detecting stress during real-world driving tasks sing physiological sensors. IEEE Trans Intell Transport Syst 2005;6:156–66. https://doi.org/10.1109/tits.2005.848368.Search in Google Scholar
17. Zheng, WL, Lu, BL. A multimodal approach to estimating vigilance using EEG and forehead EOG. J Neural Eng 2016;14:026017. https://doi.org/10.1088/1741-2552/aa5a98.Search in Google Scholar PubMed
18. Ganin, Y, Ustinova, E, Ajakan, H. Domain-adversarial training of neural networks. J Mach Learn Res 2016;17:2096–30.Search in Google Scholar
19. Li, H, Zheng, WL, Lu, BL. Multimodal vigilance estimation with adversarial domain adaptation networks. In: 2018 International Joint Conference on Neural Networks (IJCNN); 2018:1–6 pp.10.1109/IJCNN.2018.8489212Search in Google Scholar
20. Tong, JJ, Luo, Y, Ma, BQ, Zheng, WL, Lu, BL, Song, X. Sleep quality estimation with adversarial domain adaptation: from laboratory to real scenario. In: 2018 International Joint Conference on Neural Networks (IJCNN); 2018:1–8 pp.10.1109/IJCNN.2018.8489480Search in Google Scholar
21. Zheng, WL, Lu, BL. A multimodal approach to estimating vigilance using EEG and forehead EOG. J Neural Eng 2017;14:026017. https://doi.org/10.1088/1741-2552/aa5a98.Search in Google Scholar
22. Tanaka, M, Mizuno, K, Tajima, S, Sasabe, T, Watanabe, Y. Central nervous system fatigue alters autonomic nerve activity. Life Sci 2009;84:235–9. https://doi.org/10.1016/j.lfs.2008.12.004.Search in Google Scholar PubMed
23. Craig, J. The chalder fatigue scale (CFQ 11). Occup Med 2015;65:86.10.1093/occmed/kqu168Search in Google Scholar PubMed
24. Liu, Q, Liu, Y, Chen, K. Research on channel selection and multi-feature fusion of EEG signals for mental fatigue detection. Entropy 2021;23:457. https://doi.org/10.3390/e23040457.Search in Google Scholar PubMed PubMed Central
25. Welch, PD. The use of fast fourier transform for the estimation of power spectra: a method based on time averaging over short, modified periodograms. IEEE Trans Audio Electroacoust 1967;15:70–3. https://doi.org/10.1109/tau.1967.1161901.Search in Google Scholar
26. Zhu, XW, Zhang, SJ, Liu, QL. Determination to the confidence level of PSD estimation with given DOF based on WELCH algorithm. In: Proc. 4th int. conf. inf. technol. manage.; 2015:617–22 pp.10.2991/icitmi-15.2015.101Search in Google Scholar
27. Lin, TY, Goyal, P, Girshick, R. Focal loss for dense object detection. In: IEEE int. conf. computer vision; 2018:2980–8.10.1109/ICCV.2017.324Search in Google Scholar
28. Gretton, A, Borgwardt, KM, Rasch, MJ. A kernel two-sample test. J Mach Learn Res 2012;13:723–73.Search in Google Scholar
29. Zeng, H, Zhang, J, Zakaria, W. InstanceEasyTL: an improved transfer-learning method for EEG-based cross-subject fatigue detection. Sensors 2020;20:7251. https://doi.org/10.3390/s20247251.Search in Google Scholar PubMed PubMed Central
30. Long, M, Wang, J, Ding, G. Transfer feature learning with joint distribution adaptation. In: Proc. int conf on computer vision; 2013: 2200–7 pp.10.1109/ICCV.2013.274Search in Google Scholar
31. Xiao, J, Wang, R, Teng, G, Hu, Y. A transfer learning based classifier ensemble model for customer credit scoring. In: Proc. 7th Int. Joint Conf. Comput. Sci. Opt.; 2014:64–8 pp.10.1109/CSO.2014.21Search in Google Scholar
32. Sun, B, Saenko, K. Deep coral: Correlation alignment for deep domain adaptation. In: European conference on computer vision; 2016:443–50 pp.10.1007/978-3-319-49409-8_35Search in Google Scholar
33. Chen, S, Zhou, F, Liao, Q. Visual domain adaptation using weighted subspace alignment. In: IEEE Vis. Com. Ima. Proc.; 2016:1–4 pp.10.1109/VCIP.2016.7805516Search in Google Scholar
© 2023 Walter de Gruyter GmbH, Berlin/Boston
Articles in the same Issue
- Frontmatter
- Review
- A review: strategies to reduce infection in tantalum and its derivative applied to implants
- Research Articles
- Biomechanical analysis of different fixed dental restorations on short implants: a finite element study
- Highly sensitive temperature sensor using one-dimensional Bragg Reflector for biomedical applications
- Synchronisation of wearable inertial measurement units based on magnetometer data
- Multiple ECG segments denoising autoencoder model
- Heart sound classification based on equal scale frequency cepstral coefficients and deep learning
- Towards a versatile mental workload modeling using neurometric indices
- An improved multi-source domain adaptation network for inter-subject mental fatigue detection based on DANN
Articles in the same Issue
- Frontmatter
- Review
- A review: strategies to reduce infection in tantalum and its derivative applied to implants
- Research Articles
- Biomechanical analysis of different fixed dental restorations on short implants: a finite element study
- Highly sensitive temperature sensor using one-dimensional Bragg Reflector for biomedical applications
- Synchronisation of wearable inertial measurement units based on magnetometer data
- Multiple ECG segments denoising autoencoder model
- Heart sound classification based on equal scale frequency cepstral coefficients and deep learning
- Towards a versatile mental workload modeling using neurometric indices
- An improved multi-source domain adaptation network for inter-subject mental fatigue detection based on DANN
