Hand gesture recognition with deep residual network using Semg signal
-
Abid Saeed Khattak
,
Azlan bin Mohd Zain
Abstract
Objectives
To design and develop a classifier, named Sewing Driving Training based Optimization-Deep Residual Network (SDTO_DRN) for hand gesture recognition.
Methods
The electrical activity of forearm muscles generates the signals that can be captured with Surface Electromyography (sEMG) sensors and includes meaningful data for decoding both muscle actions and hand movement. This research develops an efficacious scheme for hand gesture recognition using SDTO_DRN. Here, signal pre-processing is done through Gaussian filtering. Thereafter, desired and appropriate features are extracted. Following that, effective features are chosen using SDTO. At last, hand gesture identification is accomplished based on DRN and this network is effectively fine-tuned by SDTO, which is a combination of Sewing Training Based Optimization (STBO) and Driving Training Based Optimization (DTBO). The datasets employed for the implementation of this work are MyoUP Dataset and putEMG: sEMG Gesture and Force Recognition Dataset.
Results
The designed SDTO_DRN model has gained superior performance with magnificent results by delivering a maximum accuracy of 0.943, True Positive Rate (TPR) of 0.929, True Negative Rate (TNR) of 0.919, Positive Predictive Value (PPV) of 0.924, and Negative Predictive Value (NPV) of 0.924.
Conclusions
The hand gesture recognition using the proposed model is accurate and improves the effectiveness of the recognition.
Acknowledgments
I would like to express my very great appreciation to the co-authors of this manuscript for their valuable and constructive suggestions during the planning and development of this research work.
-
Research ethics: Not Applicable.
-
Informed consent: Not Applicable.
-
Author contributions: All authors have made substantial contributions to conception and design, revising the manuscript, and the final approval of the version to be published. Also, all authors agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
-
Competing interests: The authors declare no conflict of interest.
-
Research funding: This research did not receive any specific funding.
-
Data availability: The data underlying this article are available in MyoUP Dataset “https://github.com/tsagkas/MyoUP_dataset”. The data underlying this article are available in putEMG: sEMG Gesture and Force Recognition Datasets “https://biolab.put.poznan.pl/putemg-dataset/#download”.
References
1. Shen, S, Gu, K, Chen, XR, Lv, CX, Wang, RC. Gesture recognition through sEMG with wearable device based on deep learning. Mobile Network Appl 2020;25:2447–58. https://doi.org/10.1007/s11036-020-01590-8.Search in Google Scholar
2. Enoka, RM, Duchateau, J. Inappropriate interpretation of surface EMG signals and muscle fiber characteristics impedes understanding of the control of neuromuscular function. J Appl Physiol 2015;119:1516–18, https://doi.org/10.1152/japplphysiol.00280.2015.Search in Google Scholar PubMed
3. Goen, A, Tiwari, D. Review of surface electromyogram signals: its analysis and applications. Int J Electr Electron Commun, Energy Sci Eng 2013;7:965–73.Search in Google Scholar
4. Bahador, A, Yousefi, M, Marashi, M, Bahador, O. High accurate lightweight deep learning method for gesture recognition based on surface electromyography. Comput Methods Progr Biomed 2020;195:105643. https://doi.org/10.1016/j.cmpb.2020.105643.Search in Google Scholar PubMed
5. Wachs, JP, Kölsch, M, Stern, H, Edan, Y. Vision-based hand gesture applications. Commun ACM 2011;54:60–71. https://doi.org/10.1145/1897816.1897838.Search in Google Scholar
6. Bablani, A, Edla, DR, Dodia, S. Classification of EEG data using k-nearest neighbor approach for concealed information test. Procedia Comput Sci 2018;143:242–9. https://doi.org/10.1016/j.procs.2018.10.392.Search in Google Scholar
7. Lu, Z, Chen, X, Li, Q, Zhang, X, Zhou, P. A hand gesture recognition framework and wearable gesture-based interaction prototype for mobile devices. IEEE Trans Hum Mach System 2014;44:293–9. https://doi.org/10.1109/thms.2014.2302794.Search in Google Scholar
8. Lu, Z, Tong, K, Shin, H, Li, S, Zhou, P. Advanced myoelectric control for robotic hand-assisted training: outcome from a stroke patient. Front Neurol 2017;8:107. https://doi.org/10.3389/fneur.2017.00107.Search in Google Scholar PubMed PubMed Central
9. Tabor, A, Bateman, S, Scheme, E, Flatla, DR, Gerling, K. Designing game-based myoelectric prosthesis training. In: ProcSIGCHI conf hum factor comput syst. Association for Computing Machinery, New York; 2017:1352–63 pp.10.1145/3025453.3025676Search in Google Scholar
10. Karnam, NK, Dubey, SR, Turlapaty, AC, Gokaraju, B. EMGHandNet: a hybrid CNN and Bi-LSTM architecture for hand activity classification using surface EMG signals. Biocybern Biomed Eng 2022;42:325–40. https://doi.org/10.1016/j.bbe.2022.02.005.Search in Google Scholar
11. Phinyomark, AN, Khushaba, R, Scheme, E. Feature extraction and selection for myoelectric control based on wearable EMG sensors. Sensors 2018;18:1615. https://doi.org/10.3390/s18051615.Search in Google Scholar PubMed PubMed Central
12. Parajuli, N, Sreenivasan, N, Bifulco, P, Cesarelli, M, Savino, S, Niola, V, et al.. Real-time EMG based pattern recognition control for hand prostheses: a review on existing methods, challenges and future implementation. Sensors 2019;19:4596. https://doi.org/10.3390/s19204596.Search in Google Scholar PubMed PubMed Central
13. Balasubramanian, V, Adalarasu, K. EMG-based analysis of change in muscle activity during simulated driving. J Bodyw Mov Ther 2007;11:151–8. https://doi.org/10.1016/j.jbmt.2006.12.005.Search in Google Scholar
14. Scheme, E, Englehart, K. Electromyogram pattern recognition for control of powered upper-limb prostheses: state of the art and challenges for clinical use. J Rehabilitation Res Dev 2011;48:643–59. https://doi.org/10.1682/jrrd.2010.09.0177.Search in Google Scholar PubMed
15. Yu, Z, Zhao, J, Wang, Y, He, L, Wang, S. Surface EMG-based instantaneous hand gesture recognition using convolutional neural network with the transfer learning method. Sensors 2021;21:2540. https://doi.org/10.3390/s21072540.Search in Google Scholar PubMed PubMed Central
16. Shim, H, An, H, Lee, S, Lee, E, Min, H, Lee, S. EMG pattern classification by split and merge deep belief network. Symmetry 2016;8:148. https://doi.org/10.3390/sym8120148.Search in Google Scholar
17. Krizhevsky, A, Sutskever, I, Hinton, GE. ImageNet classification with deep convolutional neural networks. In: Proceedings of the annual conference on neural information processing systems. Association for Computing Machinery, New York; 2012:1097–105 pp.Search in Google Scholar
18. Abdel-Hamid, O, Mohamed, AR, Jiang, H, Deng, L, Penn, G, Yu, D. Convolutional neural networks for speech recognition. IEEE/ACM Trans Audio, Speech, Language Process 2014;22:1533–45. https://doi.org/10.1109/taslp.2014.2339736.Search in Google Scholar
19. Karpathy, A, Toderici, G, Shetty, S, Leung, T, Sukthankar, R, Li, F. Large scale video classification with convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, New York City; 2014:1725–32 pp.10.1109/CVPR.2014.223Search in Google Scholar
20. Yang, J, Nguyen, MN, San, PP, Li, X, Krishnaswamy, S. Deep convolutional neural networks on multichannel time series for human activity recognition. In: Proceedings of the International joint conference on artificial intelligence. AAAI Press/International Joint Conferences on Artificial Intelligence, Palo Alto, California USA; 2015:3995–4001 pp.Search in Google Scholar
21. Wei, W, Wong, Y, Du, Y, Hu, Y, Kankanhalli, M, Geng, W. A multi-stream convolutional neural network for sEMG-based gesture recognition in muscle-computer interface. Pattern Recogn Lett 2019;119:131–8. https://doi.org/10.1016/j.patrec.2017.12.005.Search in Google Scholar
22. MyoUP dataset. https://github.com/tsagkas/MyoUP_dataset [Accessed Dec 2022].Search in Google Scholar
23. Tsinganos, P, Cornelis, B, Cornelis, J, Jansen, B, Skodras, A. Hilbert sEMG data scanning for hand gesture recognition based on deep learning. Neural Comput Appl 2021;33:2645–66. https://doi.org/10.1007/s00521-020-05128-7.Search in Google Scholar
24. Hu, Y, Wong, Y, Wei, W, Du, Y, Kankanhalli, M, Geng, W. A novel attention-based hybrid CNN-RNN architecture for sEMG-based gesture recognition. PLoS One 2018;13:e0206049. https://doi.org/10.1371/journal.pone.0206049.Search in Google Scholar PubMed PubMed Central
25. Qi, S, Wu, X, Chen, WH, Liu, J, Zhang, J, Wang, J. sEMG-based recognition of composite motion with convolutional neural network. Sensor Actuator Phys 2020;311:112046. https://doi.org/10.1016/j.sna.2020.112046.Search in Google Scholar
26. Bullmann, M, Fetzer, T, Ebner, F, Deinzer, F, Grzegorzek, M. Fast kernel density estimation using Gaussian filter approximation. In: 21st International conference on information fusion (FUSION). IEEE, New York City; 2018:1233–40 pp.10.23919/ICIF.2018.8455686Search in Google Scholar
27. Kathirvel, P, Manikandan, MS, Senthilkumar, S, Soman, KP. Noise robust zerocrossing rate computation for audio signal classification. In: 3rd International conference on Trendz in information sciences & computing (TISC2011). IEEE, New York City; 2011:65–9 pp.10.1109/TISC.2011.6169086Search in Google Scholar
28. Toufiq, DM, Sagheer, AM, Veisi, H. Brain tumor identification with a hybrid feature extraction method based on discrete wavelet transform and principal component analysis. Bull Electr Eng Inform 2021;10:2588–97. https://doi.org/10.11591/eei.v10i5.3013.Search in Google Scholar
29. Itoh, T, Yamauchi, N. Surface morphology characterization of pentacene thin film and its substrate with under-layers by power spectral density using fast Fourier transform algorithms. Appl Surf Sci 2007;253:6196–202. https://doi.org/10.1016/j.apsusc.2007.01.056.Search in Google Scholar
30. Delina, M, Nurhusni, PA. Feature extraction of noise signal in motorcycle by Fast Fourier Transform. J Phys Conf Ser 2021;1869:012198. https://doi.org/10.1088/1742-6596/1869/1/012198.Search in Google Scholar
31. Dehghani, M, Trojovská, E, Trojovský, P. Driving training-based optimization: a new human-based metaheuristic algorithm for solving optimization problems; 2022.10.21203/rs.3.rs-1506972/v1Search in Google Scholar
32. Dehghani, M, Trojovska, E, Zuscak, T. A new human-inspired metaheuristic algorithm for solving optimization problems based on mimicking sewing training. Scientific reports 2022;12.10.1038/s41598-022-22458-9Search in Google Scholar PubMed PubMed Central
33. Chen, Z, Chen, Y, Wu, L, Cheng, S, Lin, P. Deep residual network based fault detection and diagnosis of photovoltaic arrays using current-voltage curves and ambient conditions. Energy Convers Manag 2019;198:111793. https://doi.org/10.1016/j.enconman.2019.111793.Search in Google Scholar
34. putEMG: sEMG gesture and force recognition datasets. https://biolab.put.poznan.pl/putemg-dataset/#download [Accessed Dec 2022].Search in Google Scholar
© 2023 Walter de Gruyter GmbH, Berlin/Boston
Articles in the same Issue
- Frontmatter
- Review
- Computational models of bone fracture healing and applications: a review
- Research Articles
- Assessing standing balance with MOTI: a validation study
- Effect of non-thermal plasma treatment and resin cements on the bond strength of zirconia ceramics with different yttria concentrations
- A new approach towards extracorporeal gas exchange and first in vitro results
- Estimation of heart rate and respiratory rate by monitoring cardiopulmonary signals with flexible sensor
- Hand gesture recognition with deep residual network using Semg signal
- Crowdsourcing image segmentation for deep learning: integrated platform for citizen science, paid microtask, and gamification
- Image segmentation of mouse eye in vivo with optical coherence tomography based on Bayesian classification
- Impact of the new European medical device regulation: a two-year comparison
Articles in the same Issue
- Frontmatter
- Review
- Computational models of bone fracture healing and applications: a review
- Research Articles
- Assessing standing balance with MOTI: a validation study
- Effect of non-thermal plasma treatment and resin cements on the bond strength of zirconia ceramics with different yttria concentrations
- A new approach towards extracorporeal gas exchange and first in vitro results
- Estimation of heart rate and respiratory rate by monitoring cardiopulmonary signals with flexible sensor
- Hand gesture recognition with deep residual network using Semg signal
- Crowdsourcing image segmentation for deep learning: integrated platform for citizen science, paid microtask, and gamification
- Image segmentation of mouse eye in vivo with optical coherence tomography based on Bayesian classification
- Impact of the new European medical device regulation: a two-year comparison
