A diagnostic method for cardiomyopathy based on multimodal data
-
Linshan Shen
,
Bing Wu
Abstract
Objectives
Currently, a multitude of machine learning techniques are available for the diagnosis of hypertrophic cardiomyopathy (HCM) and dilated cardiomyopathy (DCM) by utilizing electrocardiography (ECG) data. However, these methods rely on digital versions of ECG data, while in practice, numerous ECG data still exist in paper form. As a result, the accuracy of the existing machine learning diagnostic models is suboptimal in practical scenarios. In order to enhance the accuracy of machine learning models for diagnosing cardiomyopathy, we propose a multimodal machine learning model capable of diagnosing both HCM and DCM.
Methods
Our study employed an artificial neural network (ANN) for feature extraction from both the echocardiogram report form and biochemical examination data. Furthermore, a convolutional neural network (CNN) was utilized for feature extraction from the electrocardiogram (ECG). The resulting extracted features were subsequently integrated and inputted into a multilayer perceptron (MLP) for diagnostic classification.
Results
Our multimodal fusion model achieved a precision of 89.87%, recall of 91.20%, F1 score of 89.13%, and precision of 89.72%.
Conclusions
Compared to existing machine learning models, our proposed multimodal fusion model has achieved superior results in various performance metrics. We believe that our method is effective.
Funding source: Fundamental Research Funds for the Central Universities
Award Identifier / Grant number: 3072022TS0604
-
Research funding: This paper is supported by the Fundamental Research Funds for the Central Universities (3072022TS0604).
-
Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission.
-
Conflicts of interests: Authors state no conflict of interest.
-
Informed consent: Informed consent was obtained from all individuals included in this study.
-
Ethical approval: The local Institutional Review Board deemed the study exempt from review.
References
1. Kakadiaris, IA, editor. Keynote lecture 2: cardiovascular informatics: how to stop a heart attack before it happens. In: Proceedings of the 15th american conference on Applied mathematics; 2009.Search in Google Scholar
2. Kovalevskaya, E, Krylova, N, Poteshkina, N. Hypertrophic cardiomyopathy and ischemic heart disease: the problem of pathology combination. Kardiologiia 2018:31–5. https://doi.org/10.18087/cardio.2386.Search in Google Scholar
3. Schultheiss, H-P, Fairweather, D, Caforio, AL, Escher, F, Hershberger, RE, Lipshultz, SE, et al.. Dilated cardiomyopathy. Nat Rev Dis Prim 2019;5:1–19. https://doi.org/10.1038/s41572-019-0084-1.Search in Google Scholar PubMed PubMed Central
4. Qaisar, SM, Subasi, A Cloud-based cardiac health monitoring using event-driven ECG processing and ensemble classification techniques. Modelling and Analysis of Active Biopotential Signals in Healthcare 2020;1.10.1088/978-0-7503-3279-8ch11Search in Google Scholar
5. Alickovic, E, Subasi, A. Medical decision support system for diagnosis of heart arrhythmia using DWT and random forests classifier. J Med Syst 2016;40:108. https://doi.org/10.1007/s10916-016-0467-8.Search in Google Scholar PubMed
6. Alickovic, E, Subasi, A. Effect of multiscale PCA de-noising in ECG beat classification for diagnosis of cardiovascular diseases. Circ Syst Signal Process 2015;34:513–33. https://doi.org/10.1007/s00034-014-9864-8.Search in Google Scholar
7. Masetic, Z, Subasi, A. Congestive heart failure detection using random forest classifier. Comput Methods Progr Biomed 2016;130:54–64. https://doi.org/10.1016/j.cmpb.2016.03.020.Search in Google Scholar PubMed
8. Jing, XJ, Tian, RX. Advances in the application of speckle tracking imaging in the evaluation of myocardial abnormalities. Chinese J Gen Pract 2019;17:276–9.Search in Google Scholar
9. Yue, H, Feng, Y, Center, HM, Hospital, RP. Diagnostic value of electrocardiogram in hypertensive left ventricular hypertrophy and hypertrophic cardiomyopathy. China Health Standard Management; 2018.Search in Google Scholar
10. Lyon, A, Ariga, R, Mincholé, A, Mahmod, M, Ormondroyd, E, Laguna, P, et al.. Distinct ECG phenotypes identified in hypertrophic cardiomyopathy using machine learning associate with arrhythmic risk markers. Front Physiol 2018;9:213. https://doi.org/10.3389/fphys.2018.00213.Search in Google Scholar PubMed PubMed Central
11. Begum, R, Ramesh, M. Detection of cardiomyopathy using support vector machine and artificial neural network. Int J Comput Appl 2016;133:29–34. https://doi.org/10.5120/ijca2016908178.Search in Google Scholar
12. Rahman, QA, Tereshchenko, LG, Kongkatong, M, Abraham, T, Abraham, MR, Shatkay, H. Utilizing ECG-based heartbeat classification for hypertrophic cardiomyopathy identification. IEEE Trans NanoBioscience 2015;14:505–12. https://doi.org/10.1109/tnb.2015.2426213.Search in Google Scholar
13. Sengupta, PP, Kulkarni, H, Narula, J. Prediction of abnormal myocardial relaxation from signal processed surface ECG. J Am Coll Cardiol 2018;71:1650–60. https://doi.org/10.1016/j.jacc.2018.02.024.Search in Google Scholar PubMed
14. Ko, W-Y, Siontis, KC, Attia, ZI, Carter, RE, Kapa, S, Ommen, SR, et al.. Detection of hypertrophic cardiomyopathy using a convolutional neural network-enabled electrocardiogram. J Am Coll Cardiol 2020;75:722–33. https://doi.org/10.1016/j.jacc.2019.12.030.Search in Google Scholar PubMed
15. MacGregor, RM, Guo, A, Masood, MF, Cupps, BP, Ewald, GA, Pasque, MK, et al.. Machine learning outcome prediction in dilated cardiomyopathy using regional left ventricular multiparametric strain. Ann Biomed Eng 2021;49:922–32. https://doi.org/10.1007/s10439-020-02639-1.Search in Google Scholar PubMed PubMed Central
16. Jimenez-Perez, G, Loncaric, F, Marti Castellote, P, Ramos Jovani, M, Gonzales Lopez, E, Gonzales Mirelis, J, et al.. Machine learning-based phenotyping and risk assessment of hypertrophic cardiomyopathy-linking ECGs, morphology and genotypes. Eur Heart J Cardiovasc Imaging 2022;23:431. https://doi.org/10.1093/ehjci/jeab289.431.Search in Google Scholar
17. Baloglu, UB, Talo, M, Yildirim, O, San Tan, R, Acharya, UR. Classification of myocardial infarction with multi-lead ECG signals and deep CNN. Pattern Recogn Lett 2019;122:23–30. https://doi.org/10.1016/j.patrec.2019.02.016.Search in Google Scholar
18. Lih, OS, Jahmunah, V, San, TR, Ciaccio, EJ, Yamakawa, T, Tanabe, M, et al.. Comprehensive electrocardiographic diagnosis based on deep learning. Artif Intell Med 2020;103:101789. https://doi.org/10.1016/j.artmed.2019.101789.Search in Google Scholar PubMed
19. Liu, W, Wang, F, Huang, Q, Chang, S, Wang, H, He, J. MFB-CBRNN: a hybrid network for MI detection using 12-lead ECGs. IEEE J Biomed Health Inform 2019;24:503–14. https://doi.org/10.1109/jbhi.2019.2910082.Search in Google Scholar PubMed
20. Han, C, Shi, L. ML–ResNet: A novel network to detect and locate myocardial infarction using 12 leads ECG. Comput Methods Progr Biomed 2020;185:105138. https://doi.org/10.1016/j.cmpb.2019.105138.Search in Google Scholar PubMed
21. Mian Qaisar, S, Subasi, A. Cloud-based ECG monitoring using event-driven ECG acquisition and machine learning techniques. Phy Eng Sci Med 2020;43:623–34. https://doi.org/10.1007/s13246-020-00863-6.Search in Google Scholar PubMed
22. Hammad, M, Iliyasu, AM, Subasi, A, Ho, ES, Abd El-Latif, AA. A multitier deep learning model for arrhythmia detection. IEEE Trans Instrum Meas 2020;70:1–9. https://doi.org/10.1109/tim.2020.3033072.Search in Google Scholar
23. Subasi, A, Dogan, S, Tuncer, T. A novel automated tower graph based ECG signal classification method with hexadecimal local adaptive binary pattern and deep learning. J Ambient Intell Hum Comput 2021:1–15. https://doi.org/10.1007/s12652-021-03324-4.Search in Google Scholar
24. Mroueh, Y, Marcheret, E, Goel, V, editors. Deep multimodal learning for audio-visual speech recognition. In: IEEE International conference on acoustics, speech and signal processing (ICASSP). 2015.10.1109/ICASSP.2015.7178347Search in Google Scholar
25. Neverova, N, Wolf, C, Taylor, G, Nebout, F. Moddrop: adaptive multi-modal gesture recognition. IEEE Trans Pattern Anal Mach Intell 2015;38:1692–706. https://doi.org/10.1109/tpami.2015.2461544.Search in Google Scholar
26. Akhtar, MS, Chauhan, DS, Ghosal, D, Poria, S, Ekbal, A, Bhattacharyya, P. Multi-task learning for multi-modal emotion recognition and sentiment analysis. arXiv preprint arXiv:190505812. 2019.10.18653/v1/N19-1034Search in Google Scholar
27. Mirabnahrazam, G, Ma, D, Lee, S, Popuri, K, Lee, H, Cao, J, et al.. Machine learning based multimodal neuroimaging genomics dementia score for predicting future conversion to alzheimer’s disease. J Alzheim Dis 2022:1–21. https://doi.org/10.3233/jad-220021.Search in Google Scholar
28. Trockman, A, Kolter, JZ. Patches are all you need? arXiv preprint arXiv:220109792. 2022.Search in Google Scholar
29. He, K, Zhang, X, Ren, S, Sun, J, editors. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2016.10.1109/CVPR.2016.90Search in Google Scholar
30. Huang, G, Liu, Z, Van Der Maaten, L, Weinberger, KQ, editors. Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2017.10.1109/CVPR.2017.243Search in Google Scholar
31. Howard, AG, Zhu, M, Chen, B, Kalenichenko, D, Wang, W, Weyand, T, et al.. Mobilenets: Efficient convolutional neural networks for mobile vision applications; 2017. arXiv preprint arXiv:170404861.Search in Google Scholar
32. Simonyan, K, Zisserman, A. Very deep convolutional networks for large-scale image recognition; 2014. arXiv preprint arXiv:14091556.Search in Google Scholar
33. Szegedy, C, Liu, W, Jia, Y, Sermanet, P, Reed, S, Anguelov, D, et al., editors. Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2015.10.1109/CVPR.2015.7298594Search in Google Scholar
34. Zou, Y, Hui, R, Song, L, Cardiology, DO. Interpretation of guidelines for diagnosis and treatment of Chinese adult patients with hypertrophic cardiomyopathy. J Shanghai Univ (Engl Ed) 2018;18:2396–400.Search in Google Scholar
35. Liao, Y, Cardiology, DO, Hospital, U. Chinese guidelines for diagnosis and treatment of dilated cardiomayopathy:innovation and transformation. J Clin Cardiol 2018;34:435–6.Search in Google Scholar
© 2023 Walter de Gruyter GmbH, Berlin/Boston
Articles in the same Issue
- Frontmatter
- Review
- Effectiveness of FES-supported leg exercise for promotion of paralysed lower limb muscle and bone health—a systematic review
- Research Articles
- Stimulation of spinal cord according to recorded theta hippocampal rhythm during rat move on treadmill
- EEG-based driver states discrimination by noise fraction analysis and novel clustering algorithm
- Active fault tolerant deep brain stimulator for epilepsy using deep neural network
- Stacked machine learning models to classify atrial disorders based on clinical ECG features: a method to predict early atrial fibrillation
- A diagnostic method for cardiomyopathy based on multimodal data
- Hyperspectral imaging enables the differentiation of differentially inflated and perfused pulmonary tissue: a proof-of-concept study in pulmonary lobectomies for intersegmental plane mapping
- Hyperspectral imaging-based cutaneous wound classification using neighbourhood extraction 3D convolutional neural network
- The effects of heating rate and sintering time on the biaxial flexural strength of monolithic zirconia ceramics
Articles in the same Issue
- Frontmatter
- Review
- Effectiveness of FES-supported leg exercise for promotion of paralysed lower limb muscle and bone health—a systematic review
- Research Articles
- Stimulation of spinal cord according to recorded theta hippocampal rhythm during rat move on treadmill
- EEG-based driver states discrimination by noise fraction analysis and novel clustering algorithm
- Active fault tolerant deep brain stimulator for epilepsy using deep neural network
- Stacked machine learning models to classify atrial disorders based on clinical ECG features: a method to predict early atrial fibrillation
- A diagnostic method for cardiomyopathy based on multimodal data
- Hyperspectral imaging enables the differentiation of differentially inflated and perfused pulmonary tissue: a proof-of-concept study in pulmonary lobectomies for intersegmental plane mapping
- Hyperspectral imaging-based cutaneous wound classification using neighbourhood extraction 3D convolutional neural network
- The effects of heating rate and sintering time on the biaxial flexural strength of monolithic zirconia ceramics
