Demystifying multiple sclerosis diagnosis using interpretable and understandable artificial intelligence

1 Introduction

Multiple sclerosis (MS), a complex and often unpredictable autoimmune disease, disrupts communication within the central nervous system, leading to debilitating neurological symptoms [1]. This disease occurs when the body unconditionally targets itself [2]. Myelin, the lipid-rich multilamellar membrane ensheathed around neural fibers [3], undergoes focal, segmental breakdown in MS patients [4]. This demyelination process weakens axonal signal propagation, consequently hindering neurotransmission between central and peripheral organ systems.

The precise origin of MS is currently undetermined. Nevertheless, robust evidence implicates the multifaceted interaction of genetic vulnerabilities, environmental modulators, and potential infectious insults in its initiation and progression [5]. Studies have indicated that viruses such as Epstein–Barr have induced MS in patients [6]. According to some researchers, a lack of vitamin D can also increase the chance of MS in patients [7]. The symptoms of MS are generally erratic. How they manifest depends on the particular part of the impacted neurological system. The first signs of MS include complete/partial loss of vision, trouble walking, and numbness in the body. Other symptoms include muscle pain, trouble coordinating, spasticity, tiredness, loss of sensation, problems speaking, tremors, fainting sensation, trouble hearing, bowel and urinary issues, and mental illness. Patients have experienced issues with cognition, including difficulties with memory retention, concentration, and decision-making abilities. Coping with a chronic ailment such as MS can be very challenging, and this can lead to depression and other mental illnesses [8].

To date, there is no permanent solution for MS. However, several treatments and medicines have been included to reduce the severe symptoms and slow their progression [9]. These include disease-modifying therapies, relapse management treatments, physical rehabilitation, and counselling. The flare-ups can also be reduced by leading a healthy lifestyle. This includes eating healthy, exercising regularly, managing stress, and limiting smoking and alcohol intake.

There is no specific technique to diagnose MS. However, a combination of techniques such as magnetic resonance imaging (MRI) scans, evoked potentials, cerebral spinal fluid analysis, blood tests, and OCT (optical coherence tomography) scans have been used to detect MS [10]. The details mentioned above are summarized in Figure 1.

Figure 1

The common symptoms, causes, treatments, and prevention of multiple sclerosis.

Artificial intelligence (AI) techniques are being heavily deployed in various sectors, and machine learning (ML) models are being used for predicting diagnosis, prognosis, mortality, patient care, preliminary screening, and medical resource utilization [11]. Explainable AI (XAI) makes the ML classifiers interpretable [12]. Ensuring transparent and reliable predictions through clear visual communication would unlock the potential of these models for real-world application in medical settings, empowering doctors with valuable decision-making tools [13].

Several studies have already been conducted that use ML algorithms to diagnose MS. In a study by Law et al. [14], ML was used to predict secondary progressive MS. Three ML classifiers were chosen for prediction. The number of patients selected was 485. The decision tree obtained optimal results with an area under the curve (AUC) of 0.61. Another research used XAI to differentiate MS phenotypes [15]. The subjects considered in this research were from Raffaele Scientific Institute, Italy. A graphic clustering framework was used for detection. Zhao et al. [16] used ML classifiers to predict MS prognosis. Support vector machine (SVM) and logistic regression were the classifiers utilized. The algorithms obtained a maximum sensitivity of 86%. The critical features were family history, race, brain T2 lesion volume, and brain parenchymal fraction. Denissen et al. [17] conducted a systematic review of the literature investigating the potential of ML algorithms for prognostication of MS, with a significant emphasis on studies employing biological markers as predictors.

Building upon the demonstrated capability of ML algorithms in predicting MS disease course [18], this investigation broadened its scope to encompass pertinent considerations for real-world implementation, including data limitations, bias mitigation, missing data imputation, data integration, and the imperative for interpretable AI models. Prioritizing informative features through dimensionality reduction proved crucial in another study [18], where deep learning algorithms successfully predicted MS with an accuracy of 80%. This approach streamlined model complexity and potentially enhanced interpretability. Another study leveraged radiological and clinical markers to predict MS prognosis in a cohort of 163 patients, with the best-performing classifier achieving an accuracy of 79% [19]. This multimodal approach demonstrates the potential for integrating diverse data sources to enhance predictive accuracy. A recent investigation [20] employed an SVM model to classify MS based on 11 gait variables, attaining a noteworthy accuracy of 81%. Pinto et al. [21] used ML to predict the disease progression. The maximum AUC, recall, and specificity obtained were 0.89, 0.84, and 0.81, respectively. According to the study, the most key features were age, disability status scale value, and affected functions during relapse.

Building upon the established role of ML in predicting disease progression, this investigation tackles the crucial challenge of model interpretability. We propose a novel framework for developing transparent and explainable models, addressing a significant unmet need in this field. The contributions made in this study are listed as follows.

• Prior to model training, comprehensive descriptive and inferential statistical analysis was conducted using JAMOVI to uncover salient patterns and relationships within the dataset.

• Mutual information was employed to identify the most informative features, ensuring model parsimony and potentially enhancing interpretability.

• Bayesian optimization was adopted for fine-tuning model hyperparameters, offering a potentially more efficient approach than conventional grid search and randomized search methods.

• A diverse array of ML classifiers was rigorously assessed, including a customized stacking model, a deep neural network (DNN), and a long short-term memory (LSTM) network to harness their unique strengths and identify the best-performing architecture.

• To demystify model predictions and build trust, we leveraged four diverse XAI methods (SHAP, LIME, Eli5, and QLattice) for valuable insights into their decision-making processes.

• We explored how our findings could refine diagnosis, personalize treatment, and deepen medical understanding of the disease.

2 Materials and methods

In this section, we begin by exploring the data through descriptive and inferential statistics and then refine its features using mutual information. ML terminology specific to our analysis will be clarified later.

2.2 Statistical analysis, data preprocessing, and feature selection

Initially, we removed four unimportant attributes. “Patient ID” is irrelevant to ML analysis and was not included for further analysis. In this study, “schooling” was also not considered. The variables “Initial EDSS” and “Final EDSS” were directly correlated with the target variable and would make the results biased [35]. Hence, they were not considered. Sixteen features, including the target variable, were considered for further research. Further, statistical analysis was conducted on the data. Since most were categorical variables, bar graphs were used to describe them. Some of these attributes are described in Figure 2. It is observed that male patients were at a greater risk compared to female patients. The virus “Varicella” was not an essential factor since patients who contracted the virus and those who did not were at risk. A positive test for “oligoclonal bands” was observed in patients undergoing investigations for MS. Patients who tested positive for “LLSSEP” and “ULSSEP” were also at significant risk. Among the MRI scans, the most important were “periventricular MRI,” “cortical MRI,” and “infratentorial MRI.” Further, hypothesis testing was performed to gain more inferences from the data [36]. If the p-value is less than 0.001, the variable is considered necessary. Most markers were considered crucial except “breastfeeding,” “mono or polysymptomatic,” “BAEP” and “spinal cord MRI” (Table 2).

Figure 2

Bar plots to visualize the count of categorical variables. (a) Gender, (b) varicella, (c) oligoclonal bands, (d) LLSSEP, (e) ULSSEP, (f) periventricular MRI, (g) cortical MRI, (h) infratentorial MRI, and (i) spinal cord MRI.

Table 2

Chi square test results

Attribute	p-value	Attribute	p-value
Gender	<0.001	ULSSEP	0.001
Breast feeding	0.004	VEP	<0.001
Varicella	<0.001	BAEP	0.389
Initial symptom	<0.001	Periventricular MRI	<0.001
Mono or polysymptomatic	0.021	Cortical MRI	<0.001
Oligoclonal bands	<0.001	Infratentorial MRI	<0.001
LLSSEP	<0.001	Spinal cord MRI	0.046

Two variables were continuous: age and initial symptoms. The descriptive statistics conducted for the above features are described in Table 3. Violin plots for the two continuous attributes are observed in Figure 3. There is no drastic difference in age between the two groups. When there were more than eight initial symptoms, MS was diagnosed. Further, t-tests were conducted to obtain more inferences on the data. The attribute is considered critical if the obtained p-value is less than 0.001. Among the two attributes, initial symptoms were considered crucial by the Student’s t-test (Table 4).

Table 3

Descriptive statistical testing of continuous variables

	Group	N	Mean	Median	SD	Variance	IQR	Range	Minimum	Maximum
Age (years)	CDMS	125	34.84	34	11.42	130.4	14	55	15	70
	Non-CDMS	148	33.41	32.5	10.85	117.7	14.25	61	16	77
Initial symptom	CDMS	125	8.21	8	4.32	18.6	8	14	1	15
	Non-CDMS	148	4.92	4	3.5	12.2	6	14	1	15

Figure 3

Violin plots to represent the two groups: (a) age (years) and (b) initial symptoms.

Table 4

Student’s t-test

Attribute	Test type	p
Age (years)	Student’s t	0.289
Initial symptom	Student’s t	<0.001

Since most variables are categorical, they can only be used with appropriate encoding [37]. This study utilized one hot encoding since they are efficient [38]. The continuous attributes have to be scaled before model training. This is because ML classifiers favor attributes with higher values. To prevent this, we utilized a method called “standardization.” In standardization, all the values are transformed into the range of “−1” to “1” using the respective attribute’s mean and standard deviation [39]. This dataset was immune to data imbalance since both the classes had similar instances. Hence, no data balancing technique was utilized.

Choosing the correct number of features is critical in supervised ML [40]. The accuracy decreases when unwanted features are included. In this study, mutual information was chosen for feature selection [41]. It uses the concept of “information gain” and “entropy.” When the entropy decreases, information gain increases. Figure 4 depicts the most critical attributes according to mutual information. The attributes are organized, with the best one being at the top. The figure shows that the most important attributes are “periventricular MRI,” “initial symptom,” and “infratentorial MRI.” It could also be seen that the attributes “Age,” “BAEP_Positive,” “mono or polysymptomatic unknown,” and “spinal cord MRI_Positive” do not contribute to differentiating the two classes. Hence, these features were dropped from further analysis. The data were then split into training/testing at an 80:20 ratio.

Figure 4

Feature selection using mutual information.

2.3 Common ML terminologies

In this research, supervised ML algorithms, including bagging, boosting, and stacking, have been used. Labelled data are utilized in supervised ML to make accurate predictions [42]. The models are trained with the results to make accurate predictions. After the training phase, the classifiers accurately predict the new data (test data). Bagging (bootstrap aggregation) is an ensemble technique integrating multiple classifiers to make accurate predictions [43]. This technique makes the predictions more robust and accurate. Further, it uses the concept of “bootstrap sampling,” in which several subsets of the original data are trained separately. Boosting is an ensemble technique, which is similar to bagging. In contrast to bagging, which trains algorithms on their own before combining them, boosting trains classifiers sequentially, with every classifier striving to fix the previous one [44]. Further, they use concepts such as weak learners, weighted training data, sequential model training, iterative processes, and combining predictions. Stacking is an ensemble technique that trains multiple classifiers using a meta-learner [45]. The meta-learner optimizes the diagnostic predictions made by the base algorithms. It is successful when the initial algorithms differ, causing various errors in the collected information. Combining such heterogeneous models also tends to surpass any single classifier and be more resilient. In this study, we have utilized eight different classifiers. The architecture of the final STACK is described in Figure 5.

Figure 5

Stacking architecture utilized to predict MS.

Hyperparameter tuning is an essential step in ML [46]. The user sets hyperparameters before the beginning of model training. Choosing the best hyperparameters can optimize the results obtained by the algorithm. In this study, three hyperparameter tuning techniques have been utilized. They are grid search, randomized search, and Bayesian optimization search. To identify the most effective combinations, grid search extensively looks through a subset of hyperparameters [47]. While grid search comprehensively explores hyperparameter combinations, often proving effective but time-consuming, random search offers potentially faster optimization for large datasets by sampling combinations randomly [48]. In Bayesian optimization, the objective function is modelled using probabilistic combinations and is known to generate promising hyperparameters [49].

Deep learning is an aspect of ML. It entails teaching artificial neural networks to acquire knowledge from information, which is motivated by the anatomy and workings of the brain [50]. A neural network has many layers, including input, output, and hidden layers. The output of the previous layer serves as input for the next layer. Every layer contains neurons that activate based on the specific input. LSTMs are recurrent networks that work on sequential data [51]. It consists of a memory cell and many gates, such as input, output, and forget gates. Its ability to completely avoid the vanishing gradient issue is its biggest strength [51].

XAI makes the models interpretable. It uses various algorithms and visualization techniques to interpret the predictions [12,13]. It can be beneficial in fields such as healthcare and medicine. In this study, SHAP, LIME, Eli5, and QLattice explainers have been utilized. SHAP uses Shapley values based on cooperative game theory, where each player is assigned points based on their combination [52]. SHAP can be utilized for global transparency. Various plots, such as beeswarm, force, summary, and dependence plots, are easily obtained using this algorithm. LIME is an explainer that uses linear regression to make local explanations [53]. It is model-agnostic and can be used to demystify any model. Eli5 is a Python package used to understand the importance of the features [54]. It uses the concept of bias and tree-based learners to make predictions. QLattice was developed by a company Abzu. It uses quantum computing principles to generate valuable explanations [55]. The above four explainers have been utilized to understand the predictions. Various classification and loss metrics have been utilized to test the efficiency of the models, as described in Table 5. The process flow model of this study is pictorially depicted in Figure 6.

Table 5

Classification and loss metrics

Metric	Meaning	Formula
Accuracy	It calculates the percentage of instances that were rightly classified among all instances	Accuracy = Number of correctly predicted values Total predictions made (1)
Precision	It is a measure that determines the correctness of a classifier’s positive prediction	Precision = True positive results True positive results + False positive results (2)
Recall	It is a measure that determines the model’s ability to avoid false negative results	Recall =   True positive reults True positive results + False negative results (3)
F1-score	It balances the trade-off between recall and precision	F 1 ‒ score = Precision  × Recall Precision + Recall × 2 (4)
AUC	At various points, false positive rate is plotted against the true positive rate. The area under the curve is called AUC	—
Precision recall (PR) curve	At various points, precision is plotted against recall	—
Hamming loss (HL)	It measures the percentage of labels wrongly predicted	Hamming loss = 1 N ∑ i = 1 N 1 | L | ∑ j = 1 | L | hamming ( y i j , y i j ˆ ) , (5)
		where N is the number of instances, ∣ L ∣ are the labels, y i j and y i j ˆ are actual and predicted labels
Jaccard score (JS)	It measures the similarity between true and predicted values	Jaccard score = y i ∩ y i ˆ y i ∪ y i ˆ , (6)
		where y i ∩ y i ˆ is the intersection between true and predicted labels, and y i ∪ y i ˆ is the union between true and predicted labels
Log loss (LL)	It assesses the efficacy of a classifier in which the predicted outcome is a probability value ranging from 0 to 1	Log loss = ‒ 1 N ∑ i = 1 N ( y i log ( p i ) + ( 1 ‒ y i ) log ( 1 ‒ p i ) ) , (7)
		where N is the number of instances, y i is the actual outcome, and p i is the predicted probability
Mathews correlation coefficient (MCC)	It is a metric that uses true positive, true negative, false positive and false negative results	MCC = True positive × True negative ( True positive + False positive ) ( True positive + False negative ) ( True negative + False positive ) ( True negative + False negative )

Figure 6

ML work flow utilized in this study.

3 Results

This section tests the various ML models using three hyperparameter tuning techniques. The results are also compared with deep learning algorithms. The diagnostic results obtained by the algorithms were further explored using four XAI methodologies. Table 6 describes the performance of all algorithms for the three search techniques. Among the three, the Bayesian optimization search technique provided the optimal results. The random forest and xgboost achieved an accuracy of 89%. The STACK attained an accuracy of 88%. The grid search technique also performed well. A maximum accuracy of 87% was obtained by logistic regression, adaboost, catboost, and lightgbm. The ensemble STACK model obtained an accuracy of 84%. When the randomized search technique was utilized, an 87% accuracy was obtained by the logistic regression, adaboost, and lightgbm models. The final STACK obtained an accuracy of 86%. Among all the models, logistic regression, catboost, and lightgbm performed well consistently. The hyperparameters chosen by every algorithm for all three search techniques are detailed in Table 7. The AUC curves for the final STACK classifier for the three search techniques are pictorially shown in Figure 7. From the figure, it can be inferred that most instances were correctly predicted. The precision–recall curves for the final STACK for the three search methods are detailed in Figure 8. From the graph, it can be inferred that the precision–recall trade-off was meagre. The confusion matrices of the final STACK for the three search techniques are detailed in Figure 9. Most cases were predicted correctly according to this metric. Further, the number of false positive cases was zero.

Figure 7

AUC curves: (a) Bayesian optimization search, (b) grid search, and (c) randomized search.

Figure 8

Precision–recall curves: (a) Bayesian optimization search, (b) grid search, and (c) randomized search.

Figure 9

Confusion matrices: (a) Bayesian optimization search, (b) grid search, and (c) randomized search.

Two deep learning models (DNN and LSTM) were also tested. Figure 10 describes the architecture of the two models. The results and the hyperparameters used are tabulated in Table 8. Among the two, DNN performed better, with an accuracy of 80%. In this study, the ML classifiers outperformed deep learning networks. This is observed in many studies when the sample size is small. DNNs thrive when vast amounts of data are available to them. Further, the accuracy and loss curves for the predictors are described in Figure 11. Since the deviation in training and validation accuracy/loss was very less, it can be inferred that overfitting was minimal.

Figure 10

Architecture of deep learning models: (a) DNN and (b) LSTM.

Figure 11

Accuracy/loss curves: (a) accuracy curve of DNN, (b) accuracy curve of LSTM, (c) loss curve of DNN, and (d) loss curve of LSTM.

The predictions made by the above models were deciphered using four explainers. The first explainer used was SHAP. Figure 12 describes the mean bar plots obtained by the SHAP explainer. The best features are at the top, and the least important are at the bottom. The graphs show that the critical markers are periventricular MRI, initial symptoms, oligoclonal bands, infratentorial MRI, and age. SHAP is also utilized to decrypt local interpretations. Figure 13 depicts the force plots for all positively predicted MS patients. It can be seen that markers such as infratentorial MRI, oligoclonal bands, varicella, and breastfeeding push the markers toward a positive diagnosis.

Figure 12

SHAP mean bar plots: (a) Bayesian optimization search, (b) grid search, and (c) randomized search.

Figure 13

SHAP force plots: (a) Bayesian optimization search, (b) grid search, and (c) randomized search.

LIME was utilized to understand the marker variation. Figure 14 depicts the LIME plots for all three search techniques. Figure 12(a) indicates a negative diagnosis and Figure 12(b) and (c) indicates a positive diagnosis. Markers such as periventricular MRI, initial symptoms, oligoclonal bands, and infratentorial MRI were considered necessary by the LIME explainer.

Figure 14

LIME interpretation: (a) Bayesian optimization search, (b) grid search, and (c) randomized search.

Eli5 was the third following explainer used, and its interpretations are described in Figure 15. According to Eli5, spinal cord MRI, breastfeeding, oligoclonal bands, and periventricular MRI were considered crucial.

Figure 15

Eli5 interpretation: (a) Bayesian optimization search, (b) grid search, and (c) randomized search.

Finally, the quantum lattice model was utilized. The QGraphs generated by it are described in Figure 16. According to it, periventricular MRI, infratentorial MRI, and oligoclonal bands were considered as deciding factors. It also made use of addition and multiplication activation functions.

Figure 16

QLattice interpretation: (a) Bayesian optimization search, (b) grid search, and (c) randomized search.

Four explainers were used, and according to them, periventricular MRI, infratentorial MRI, oligoclonal bands, spinal cord MRI, breastfeeding, varicella, and initial symptoms were crucial in identifying MS. These markers can be used along with the ML classifiers to detect this disease accurately.

4 Discussion

This research made use of multiple ML techniques to predict MS in patients. The dataset was obtained from a public data repository called Mendeley. Initially, we performed statistical analysis, preprocessing, and feature selection. Further, several classifiers were utilized for prediction, including a customized STACK architecture. Three searching techniques were utilized to choose the best hyperparameters. The highest accuracy of 89% was achieved by the logistic regression and xgboost model when the Bayesian optimization search technique was used. The predictions made by the ML algorithms were also compared with two deep learning models. Since the dataset was small, deep learning classifiers could not perform better than ML models. The results were also interpreted using multiple XAI methods. According to them, periventricular MRI, infratentorial MRI, oligoclonal bands, spinal cord MRI, breastfeeding, varicella, and initial symptoms were critical in detecting MS. A description of the above markers concerning MS is described below.

Periventricular MRI is necessary when diagnosing MS [30]. Lesions of the periventricular region of the brain are often found in MS patients. Demyelinating lesions, which harm or eliminate the protective layer of myelin, may show up as bright spots during the MRI scan. The region below the tentorium cerebelli (cerebellum, brain stem) is scanned in infratentorial MRI [32]. In the case of MS, an infratentorial MRI is used to find lesions in the lower regions of the central nervous system. The brain produces oligoclonal bands in patients with MS and other neurological conditions [25]. This marker can be captured using a unique technique named isoelectric focusing. Lesions are also found in the spinal cord in MS patients. This can be detected using the spinal cord MRI. The extent and severity of MS can also be detected using this method. According to many studies, breastfeeding can lower the occurrence of MS in women [56]. Research suggests that hormonal changes during breastfeeding, particularly elevated prolactin levels, might enhance immune function and potentially reduce the risk of developing MS. Similarly, studies hint at a potential link between varicella infection and MS development. Varicella, also known as chickenpox, is known to trigger/contribute to MS in some patients [23]. Initial symptoms observed in MS patients include visual, sensor, and motor-based variations. According to many studies, the combination of the above markers can be instrumental in diagnosing MS.

Several studies have used ML for MS. However, most research has focused on disease progression, and diagnosis using clinical markers has yet to be explored. Further, most ML studies have yet to utilize XAI to interpret the results. Hu et al. [20] used gait markers to diagnose this condition. Eleven markers were obtained using various embedded sensors. The SVM acquired an accuracy of 81%. Seccia et al. [18] utilized a deep learning approach for MS diagnosis. Genomic data from 144 individuals were considered for this research. The artificial neural network obtained an accuracy of 79%. In another research, OCT images were used to diagnose this deadly disease [57]. The number of patients considered in this study was 212. The ensemble classifier obtained an accuracy of 87.7%. The comparison of our research with the above studies is presented in Table 9.

To enhance the quality of the research, it would have been beneficial to augment the pool of patient data samples. AI algorithms necessitate a substantial amount of data to produce accurate results, yet this study only utilized 273 samples. Moreover, the researchers did not incorporate unsupervised and reinforcement learning algorithms, which would have provided a basis for comparison with the supervised learning models mentioned earlier. Unsupervised and reinforcement learning algorithms were not utilized in this research. The above classifiers could be used and compared with the supervised learning models. A few clinical markers can vary because of other conditions, too. Hence, multiple modalities, such as gait, genomic, and OCT data, could be combined in the future. The doctors did not perform medical validation in this study. Medical validation is essential before deploying the models in real-time in healthcare facilities. In the future, to revolutionize MS research, it is crucial to establish a reliable, cloud-based data infrastructure that integrates a wide range of datasets from hospitals worldwide while prioritizing patient privacy through implementing strong data governance and anonymization measures.

5 Conclusion

This study diagnosed MS in patients using several machine-learning classifiers. In MS, the brain’s cognitive function is severely affected due to the damage caused to myelin (a protective layer that protects the nerves). MS is a complex disease to diagnose, and a series of tests such as medical history and physical examination, MRI scans, lumbar puncture, evoked potentials, blood tests, and clinical presentations are used for disease detection. Therefore, an alternative diagnosis method using AI is proposed in this research. Several classifiers, including a customized STACK model, were trained and tested, and most models performed well in differentiating MS from other conditions. The results were also demystified using various XAI techniques. According to them, important markers were periventricular MRI, infratentorial MRI, oligoclonal bands, spinal cord MRI, breastfeeding, varicella, and initial symptoms. The combination of these markers and ML could be used in identifying this deadly illness.