Explainable deep learning approach for recognizing “Egyptian Cobra” bite in real-time

3.1 IoT-based snake-bit detection

Traditional methods of snake bite detection often rely on visual identification of bite marks or symptoms, which can lead to delays in treatment and potentially fatal outcomes. To address this challenge, we propose the integration of IoT technology with wearable sensors for real-time snake bite detection and identification. Our approach utilizes wearable devices equipped with a combination of advanced sensors capable of detecting key indicators of snake bites. These sensors include pressure sensors, temperature sensors, accelerometers/gyroscopes, electrochemical sensors, and multispectral imaging technology.

• Pressure sensors: Detect sudden pressure spikes indicative of a snake bite.

• Temperature sensors: Monitor localized changes in skin temperature caused by venom injection.

• Accelerometers/gyroscopes: Detect abnormal movement patterns associated with snake bites.

Upon detecting a potential snake bite, the wearable sensors immediately transmit data to a central monitoring system via wireless communication protocols such as Bluetooth or Wi-Fi. This real-time data transmission ensures prompt notification of snake bite incidents, enabling rapid response and medical intervention.

We develop equations to detect snake bites based on sensor data that require a deep understanding of snake venom effects on the body and how these effects manifest in sensor readings, as shown in equations (1)–(4). Let us say we have sensors measuring pressure (P), temperature (T), and biochemical markers (B). We want to determine the likelihood of a snake bite based on these sensor readings. We determine typical readings for each sensor in the absence of a snake bite (normal case-reading). This baseline can vary depending on the individual and environmental conditions, as shown in Table 2. After that, we determine how sensor readings deviate from baseline values in the presence of a snake bite. For example, pressure might increase due to swelling at the bite site, temperature might rise due to localized inflammation, and certain biochemical markers might increase due to venom effects

Here, w ₁, w ₂, and w ₃ are weighting factors representing the relative importance of each sensor in detecting a snake bite. These weighting values need to be determined through experimentation and validation; ΔP, ΔT, and ΔB are the deviations in pressure, temperature, and biochemical marker reading, respectively. Also, P _baseline is the baseline pressure reading, T _baseline is the baseline temperature reading, and B _baseline is the baseline biomedical reading, P _{Snake Bite} is the probability that the victim was exposed to a snake bite. After that, we gather real-world data from sensor readings in both snake bite and non-bite scenarios and then adjust weighting factors and threshold values based on the observed correlation between sensor readings and snake bites. We continuously refine the equation based on feedback from experts and real-world observations.

3.2 Case study for our proposed equation

John, while hiking in a forest known to be inhabited by venomous snakes, suddenly felt a sharp pain in his leg [21]. He quickly examined his leg and noticed fang marks and swelling around the bite site.

Here are the readings taken immediately after the suspected snake bite:

1. Pressure reading (P):

   • Current pressure reading: 1,055 mmHg (higher than the baseline of 1,000 mmHg)

2. Temperature reading (T):

   • Current temperature reading: 38.5°C (higher than the baseline of 37°C)

3. Biochemical marker reading (B):

   • Current biochemical marker reading: 20 units (elevated from the baseline of 0 units)

Now, let us plug these values into equation (4):

Likelihood of snake bite = 1⋅(1,055−1,000) + 1⋅(38.5−37) + 1⋅(20−0)

Likelihood of snake bite = 1⋅(1,055−1,000) + 1⋅(38.5−37) + 1⋅(20−0)

Likelihood of snake bite = 55 + 1.5 + 20 likelihood of snake bite = 55 + 1.5 + 20

Likelihood of snake bite = 76.5

Since the likelihood score is above a predetermined threshold (let us say 50 for this example), it indicates a probable snake bite. Based on this evaluation, it is likely that John has been bitten by a snake.

3.3 Dataset collecting

Obtaining a suitable dataset posed a significant challenge for our research. Due to the sensitive nature of bite mark images, there is a scarcity of readily available datasets that align with our specific objectives. Access to such images is restricted and requires patient consent. Consequently, extensive and meticulous online searches were conducted to locate bite mark images caused by Egyptian cobra bites, amplifying the complexity of our task. To compile our dataset, we extensively scoured zoological and medical scientific papers, as well as online medical websites [17]. Through these endeavors, we successfully gathered a total of 847 images showing bite marks from Egyptian cobras and 1,005 images of bite marks from other snake species, as illustrated in Figure 2. Given the challenges mentioned earlier, this compilation stands as a noteworthy accomplishment. However, we recognize the need to address this challenge to attain the highest possible accuracy. To address the diversity in image sizes within the dataset, we employed DA techniques and leveraged transfer learning. This approach involved resizing all bite mark images to a uniform dimension of 1,000 × 1,000 pixels. Consequently, we achieved RGB reordering, resulting in an input format of 1,000 × 1,000 × 3 for our proposed model. Among the included cases, 73% corresponded to male individuals, while 27% pertained to female individuals. Furthermore, 54% of the cases involved young individuals, whereas 46% encompassed older individuals, some of whom suffered from chronic diseases. Overcoming these challenges through the utilization of DA and transfer learning techniques has enabled us to address the intricacies inherent in our dataset. By employing these strategies, our objective is to improve the accuracy and resilience of our model, making it suitable for various scenarios and delivering the most dependable results.

3.4 Dataset pre-processing

Enhancing the generalization of deep learning is a critical challenge in our research. Generalization refers to a model’s ability to perform well on unseen data (testing data) after being trained on a set of known data (training data). Typically, achieving strong generalization necessitates a large dataset that encompasses a diverse range of features. However, in certain scenarios, dataset availability may be limited, presenting a challenge in enhancing model accuracy. In order to tackle this challenge and improve our model’s precision, we incorporate a method known as DA. DA enables us to artificially expand the size of the training dataset by generating altered variations of the existing data DA offers a mechanism to artificially expand the size of the training dataset by generating modified versions of existing data. In our study, we employ specific augmentation techniques, namely image flipping and also zooming-in, to augment our dataset size, as outlined in Table 3. By applying these augmentation techniques, we aim to increase the diversity and variability within our dataset, enabling our model to learn more robust and representative features.

The application of a two-dimensional Gaussian filter is commonly employed for noise reduction and smoothing purposes [2]. However, its implementation demands substantial processing resources and performance. The Gaussian operator is typically utilized, with Gaussian smoothing accomplished through convolution, as shown in equation (5)

The Gaussian operators in two dimensions (circularly symmetric) are showcased by

In this context, σ (Sigma) represents the standard deviation of the Gaussian function, with higher values yielding superior smoothing effects. The coordinates (x, y) denote the Cartesian coordinates within the image, indicating the dimensions of the window. These filters involve summation and multiplication operations between the kernel and image, with the image typically represented in a matrix format ranging from 0 to 255 on an 8-bit scale. The kernel is a standardized square matrix ranging between 0 and 1. Following this, the kernel’s size is expressed in bits. During the convolution process, the total bits of the kernel and all elements of the image are divided by the power of 2.

3.5 Transfer learning

Transfer learning is a highly effective approach that leverages pre-trained deep learning models as a starting point for new tasks. It involves reusing a model previously trained on one task and applying it to a related task, resulting in accelerated progress during the modeling process. By utilizing transfer learning, we can achieve improved performance compared to training a model from scratch, especially when working with given limited data availability; our snake species identification model incorporates three well-established pre-trained models: VGG19, VGG-16, and ResNet-101. Notably, VGG-19 is a deep CNN comprising 19 layers, including 5 convolutional blocks and 3 fully connected layers. In comparison, ResNet-101 is a variant of the ResNet CNN with a remarkable depth of 101 layers, featuring 33 residual blocks. In our model, the convolutional layer plays a crucial role as it conducts feature extraction. This layer employs the convolution operation, which replaces conventional matrix multiplication and serves as the fundamental component of CNNs. The convolutional layer’s parameters consist of a set of trainable filters, also known as kernels. Its primary purpose is to detect features within various regions of the input image and generate corresponding feature maps. This operation involves applying a specific filter over the input image, with typical filter sizes being 3 × 3, 5 × 5, or 7 × 7 pixels. The outcome of this convolution operation results in the creation of activation maps, which contain locally distinctive features relevant to the specific task, as shown in equation (7)

In this context, each convolution layer is equipped with a filter represented as m ₁. The output of Layer l comprises m 1 ( l ) feature maps, denoted as Y i ( l ) , with dimensions m 2 l × m 3 l . The i-th feature map, indicated as Y i ( l ) , is calculated using equation (1), with B i ( l ) representing the deviation matrix, and K i , j l as the filter dimension. The operation of the pooling layer is described in equations (8)–(10)

In the pooling operation, a vector “v” is reduced to a single scalar “f(v)” through the pooling process “f.” Two types of pooling are commonly used: average pooling and max pooling. In this case, max pooling is employed. Fully connected layers serve the following purposes: (1) they convert the feature maps from the last convolution or pooling layer into a one-dimensional vector, (2) connect to one or more dense layers, (3) adjust the weights, and (4) provide the final classification prediction.

3.6 Visualization

The lack of tools for understanding the functioning of black-box models poses a challenge to the integration of deep learning in medical imaging. In the medical field, where trust among clinicians and patients is paramount, interpretability and reliability are essential. Moreover, deep learning models have the potential to unveil biologically distinctive patterns by identifying underlying features that conventional diagnosis might overlook. However, these valuable insights can only be leveraged if the model’s decision-making process is interpretable and comprehensible to those examining the results. When the reasoning behind a model’s predictions is unknown or poorly understood, it raises questions about the level of trust in those predictions. In our research, we employ a gradient-weighted class activation mapping technique called Grad-CAM. This method is both class-specific and proficient at pinpointing relevant regions within an image. Grad-CAM operates by using gradients to assign scores to various parts of an image based on a trained model’s assessment of a given image category. As demonstrated in a prior study [22], Grad-CAM outperformed other explainable techniques, such as CAM and Grad-CAM++. Consequently, we have opted to utilize Grad-CAM for interpreting our model’s decisions and highlighting the salient features within the bite marks.

4.1 Confusion matrix

The confusion matrix is a valuable tool for evaluating the performance of deep learning models in classification tasks. This matrix enables the comparison of predicted outcomes with actual values, consisting of four key components: False Positives (FP), True Positives (TP), True Negatives (TN), and False Negatives (FN), as outlined in Table 5. The matrix’s rows correspond to “Real class values,” while the columns represent “Predicted class values.” This analysis aids in evaluating the model’s performance. Based on the validation dataset’s confusion matrix, our model attained a sensitivity of 96.7% and a specificity of 71.5%. Sensitivity assesses the model’s capability to accurately detect Egyptian cobra bite marks (True Positives, TP) out of the total number of such bites in the test dataset. Our model made accurate predictions for 170 out of 220 Egyptian cobra bites, indicating a 96.7% success rate. This result signifies a minimal error rate of 3.3% and underscores the model’s effectiveness in identifying Egyptian cobra bite marks. Specificity, on the other hand, refers to the model’s ability to correctly identify non-Egyptian cobra bites (True Negatives, TN). Our model achieved a specificity rate of 71.4%. By calculating the overall accuracy based on the confusion matrix, our model achieved an accuracy of 90.9%. This demonstrates its high accuracy compared to other models for snake species identification, as shown in Table 3. The increased accuracy can be ascribed to the utilization of bite mark images in the training data, which provide the model with effective features for precise bite mark classification. The unique structural characteristics and functional properties of venom components in various snake species play a crucial role in the successful classification, as they rely on well-defined parameters [23].

4.2 Accuracy

The validation accuracy ranged from 74.6 to 91.5%. Similarly, the training accuracy ranged from 77.8 to 94.01%. This indicates the model’s ability to generalize, a crucial aspect of deep learning models. In comparison to existing approaches outlined in Table 6, our model surpasses them in terms of performance. Unlike these approaches that rely on snake image classes, our model overcomes the limitations caused by partial snake visibility or coiled positions. These constraints result in missing important snake features in training images.

4.3 Loss

With regard to the loss factor, our model consistently exhibits impressive outcomes when employing different models in our transfer learning approach, as illustrated in Figure 3. These findings highlight the efficacy of our technique in promoting a resilient learning process and in harnessing the dataset to extract meaningful features, thereby enabling our model to achieve accurate identification of “Egyptian Cobra” bites.

Figure 3

Loss of the proposed model. Source: Proposed, created and drawn by the authors.

4.4 Time decision

Our primary objective is the swift identification of the snake species responsible for the attack, as this is a critical initial step in saving the victim’s life. Therefore, expediting this process is of paramount importance. To evaluate the time aspect, we conducted a comparative analysis with prior research endeavors. The time taken for a decision is influenced by two main factors: transmission time and prediction time. Concerning transmission time, our approach follows the edge node paradigm, siting the classifier in close proximity to the user, which results in reduced transmission time. In terms of prediction time, as demonstrated in Table 6, our approach stands out for its rapid predictions due to its straightforward implementation.

4.5 Interpretation parameter

This parameter is recent and important as it provides insights and justifications for the decisions made by deep learning models. By understanding why a model makes certain predictions or classifications, we gain transparency and trust in its output. As shown in Figure 4, our model considers only the injury place and neglects all other possible features. This reflects our model interests.

Figure 4

Heatmap for our model. Source: Proposed, created and drawn by the authors.