Enhancing IoT device security: CNN-SVM hybrid approach for real-time detection of DoS and DDoS attacks

3.1 Dataset collection and description

In the study by Singh et al. [35], an authentic IoT attack dataset was utilized to cultivate practical security analytics applications within real IoT operations. This approach involved employing actual IoT devices both as attackers and victims, fostering a genuine representation of the IoT environment. The primary objective was to empower researchers in developing robust security models to counteract IoT-based attacks effectively. To accomplish this, a comprehensive experimentation was conducted, encompassing a total of 33 distinct attacks. These attacks were executed within a meticulously designed IoT topology comprising 105 devices of different types and brands. The overarching goal was two-fold: first, to comprehend the intricate dynamics of these attacks within an IoT setting, and second, to systematically categorize them into seven distinct categories. These categories encompassed DDoS, DoS, Recon, Web-based, Brute Force, Spoofing, and Mirai attacks. This categorization aimed to facilitate the accurate identification and classification of IoT traffic as either malicious or benign. Table 2 discuss DDoS and DoS attacks affect in IOT environment as follows.

3.2 Countermeasures

Countermeasures indicated in Table 3 for IoT attacks include a combination of technological and organizational measures targeted at improving security and protecting IoT devices and networks from potential threats. By implementing these measures, enterprises may significantly enhance the security posture of their IoT ecosystem and lower the chance of falling victim to IoT-related cyber-attacks.

3.3 Preprocessing

All rows containing a missing value have been removed which consists of five rows, and each row that contains a missing value has also been removed. Also, due to the discrepancy in the values in the data, we decided to use Min Max scaler [36] which contributes to scaling the data. And before entering data into a CNN or traditional models in ML, it is better that all data and values be in the same range with the least degree of variation, as Min Max scaler was used, and all related values were confined between [0, 1]. Equation (1) shows Min-max Scaling.

3.4 Hybrid CNN: Integrating SVM and ELM

Our proposed hybrid model 1D CNN + SVM was built for classification between DDoS, DoS, and Benign. The hybrid model 1D CNN + SVM, proposed in this study for classifying DDoS, DoS, and Benign attacks, operates through two distinct stages. In the initial stage, a CNN is employed to extract intricate features from the data. The CNN architecture in this approach is composed of eight layers. The model initiates with three convolutional layers (Conv1D), each having 32, 64, and 128 filters, respectively, with a kernel size of (3 × 3). This configuration enables the network to discern intricate patterns, aiding in the prevention of overfitting. Furthermore, the architecture incorporates MP with a size of 2, effectively reducing the spatial dimensions of the data. The activation function Rectified linear unit (ReLU) is applied, which rectifies negative values to zero while retaining positive values unchanged. This choice helps mitigate the vanishing gradient problem. The final layer comprises Dense layers housing 512 output units. For binary classification, the sigmoid activation function is employed as the output layer, and the optimizer Adam is selected. The training dataset is partitioned into 75% for training, 10% for validation, and 15% for testing purposes. Moving to the second stage, pivotal features extracted by the initial CNN stage are utilized to represent the data comprehensively. The classification task is then undertaken by the SVM ML algorithm. Renowned for its adeptness at categorizing data using hyperplanes, SVM facilitates effective separation of data into distinct categories as binary classification [53] as shown in Table 4. The study also touches on the concept of ELM [54], which serves as an interesting avenue that can enhance the performance of the hybrid model. It is one of the deep learning algorithms that are characterized by speed and performance in classification, as it works without modifications during the training phase, unlike other traditional methods, where the number of neurons is determined within the hidden layers, and this is what distinguishes it because it assigns weights that link between the input layers and the hidden layers, and therefore, does not need to adjust parameters a lot, and if we look at it when it consists of a NN that contains one hidden layer, let us assume we have S arbitrary samples ( x j , t j ) where x j = [ x j 1 , x 2 j , x 3 j , … x jn ] T ε R n , t j = [ t j 1 , t 2 j , t 3 j , … t ju ] T ε R u ,

If we have K nodes hidden layer, then the NN containing one hidden layer is expressed in formula (2):

where β i is the output weight, f(x) is the activation function indicated in formula (3), W i = [ w i 1 , w i , w i 3 … w i n ] T is input of weight, and b i is the bias. The goal of having one hidden layer is to reduce the output error, we can express it using equation (4)

3.5 Performance assessment

The performance evaluation of the model unfolds through the lens of the confusion matrix, shown in Table 5. To compare and evaluate a model’s performance, a test dataset was applied to estimate the performance metrics: accuracy, specificity, sensitivity, and F1-score. These estimates were calculated using the number of True Positives (TPs), True Negatives (TNs), FPs, and False Negatives (FNs) resulting from applying an undertaken model to the test dataset, as explained in equations (2)–(6).

In this study, TPs show that the model’s perceptiveness is breathtakingly precise, with its discernment seamlessly coinciding with the reality at hand. It astutely identifies and classifies malicious IoT traffic, which is truly malicious, TNs come into play when the model exhibits its acumen by discerning and categorizing benign IoT traffic as exactly benign. This alignment mirrors the veritable actual classification, The FPs narrative is where the model falters a misstep in which it erroneously labels benign IoT traffic as malicious. This misjudgment births a disconcerting divergence, fueling misleading classification. In this study, several metrics were used to assess the performance of both the suggested and compared models, with accuracy being the most important. As described in equation (5), accuracy is the fraction of accurately anticipated instances compared to the total number of cases. Precision refers to TPs relative to TPs and FPs as shown in equation (6). Recall indicates the ratio of positive cases to FPs and FNs as indicated in equations (7) and F1-score refers to a measure of the level of precision and recall, which achieves a balanced scale of rating as presented in equation (8). Equation (9) shows the calculation of the error rate that results from the ratio of false predictions to the total.

The computation of the area under the receiver operating characteristic (ROC) curve (AUC) serves as a crucial numerical measure that enhances our understanding of the model’s capabilities. This supplementary metric occupies a central role in our study’s evaluation framework. An AUC value below 0.5 indicates that a classifier exhibits weaker performance. In the range of 0.7–0.8, a classifier falls within the spectrum of mediocrity and commendable performance – a zone denoting moderate achievement. Finally, the interval from 0.8 to 1 characterizes a classifier with excellent performance.

3.6 ML algorithms

Table 6 summarizes the working principle of each of the following ML algorithms: SVM, NB, KNN, and LR.

4.1 Experiments

In this section, the basis of the proposed model presented in Table 7 shows the parameters used for our CNN + SVM model. Also, the proposed model was evaluated based on four other ML algorithms which are SVM, KNN, NB, and LR, where the data were divided into 80% for training and 20% for testing. All experiment in this study were conducted using Python tool on a laptop with i5-10H CPU, NVIDIA GeForce GTX 1660Ti 4GB, 8GB RAM, 250 GB SSD, and 1 TB HDD.

4.2 Results analysis

A detailed summary of the model evaluation findings is given in Table 8. Based on the models’ F1-score, accuracy, precision, recall, and error rate percentages, they are compared. A boxplot of the performance metrics for each assessed model is shown in Figure 3. The table makes it evident that the CNN + SVM model is the most accurate and precise, showcasing its exceptional ability to classify instances with precision while maintaining a low error rate. The accuracy surged to an impressive 99.8%, matched by equally remarkable precision, recall, and F1-score values, all standing at 99%. Another noteworthy performance was recorded by the CNN + ELM model, with a promising accuracy of 99% alongside precision, recall, and F1-score scores of 99%. The SVM model secured solid results with an accuracy of 98%, while precision, recall, and F1-score held steady at 98, 97, and 98%, respectively. Figure 3 demonstrates the boxplot of performance metrics of different models.

Figure 3

Boxplot of performance metrics of different models.

Meanwhile, the CNN model demonstrated a notable accuracy of 99%, while boasting precision, recall, and F1-score values of 98, 99, and 99%, respectively. In contrast, the KNN model showcased commendable results with a strong accuracy of 97%, while precision, recall, and F1-score were pegged at 96, 98, and 97%. On the other hand, LR and NB struggled to keep pace, sharing the lowest accuracy of 96%. For NB, this translated to precision, recall, and F1-score of 95, 96, and 95%, respectively. Meanwhile, LR exhibited a precision score of 94%, recall of 98%, and an F1-score of 96%. This powerful validation of our approach stems from the idea of merging features extracted from CNN and integrating them with the original features. This ingenious fusion, coupled with the utilization of the SVM classifier on an authentic IoT attack dataset, yielded results that hold immense promise. The practicality of utilizing real-world IoT scenarios, with actual devices assuming both attacking and victim roles, underscores the potential of these findings in bolstering the development of security analytics applications within real IoT operations. Figure 4 presents with a straightforward visual depiction of how our models fared. A swift assessment of the outcomes distinctly highlights the dominance of the (CNN + SVM) model, outshining all other algorithms. Close on its heels are three contenders CNN, CNN + ELM, and SVM each equally promising performances, all clocking in at an accuracy of 98%. The KNN model secures its place at an accuracy of 97%. Finally, we see the NB and LR models rounding off the chart, both achieving an accuracy of 96%.

Figure 4

Performance metrics of different models.

Figure 5 presents a visual depiction that lays out the training and validation accuracy, alongside the training and validation loss of the CNN model that underwent meticulous training on our dataset. The aim behind this endeavor encompassed extracting pivotal features from the dataset over the course of 20 epochs. The outcome of this endeavor is nothing short of remarkable training accuracy that soared to an impressive 99.45%. A noteworthy point arises when we closely examine the validation accuracy, as it hovered tantalizingly close to the 98.7% mark during the final epoch. This achievement distinctly underscores the robustness embedded within our proposed methodology. Moreover, a truly remarkable observation surfaces when we consider the validation loss. It experienced a staggering drop, plummeting to an almost negligible value of 0.04 at the 20 epochs. This revelation serves as a powerful testament to the efficacy of our approach. Collectively, these findings interlock to furnish a compelling body of evidence that attests to the effectiveness and promise deeply embedded within our chosen methodology.

Figure 5

(a) Training and validation accuracy and (b) training and validation loss.

Figure 6 presents the error rates associated with each model. These rates effectively reflect the percentage of accurate predictions made by each model, offering a complementary view to the concept of accuracy itself. To add depth to this understanding, Figure 6 complements the narrative by emphasizing the importance of lower error rates in elevating accuracy and overall effectiveness. Notably, our proposed model, the hybrid (CNN + SVM) configuration, emerges as the standout performer, boasting an impressive error rate of 1.0. This places it ahead of the other models, with the CNN model closely trailing at an error rate of 2.0, paralleled by the SVM model at the same error rate of 2.0. As the sequence unfolds, the KNN model enters the scene with an error rate of 3.0, while the two models, NB and LR, wrap up the series with the highest error rate of 4.0. The crux of these data lies in the stark contrast exhibited by the (CNN + SVM) model. Beyond its superior performance, it manages to underscore its prowess by maintaining a significantly lower error rate in comparison to its counterparts.

Figure 6

Error rates of different models for accuracy.

Figure 7 presents a confusion matrix that encompasses all the models employed for classification. What catches our attention is how our proposed model (CNN + SVM) stands out as a frontrunner in terms of accuracy and various performance metrics. It is worth noting that while other models also demonstrate their strengths, our (CNN + SVM) model truly shines. On the flip side, certain errors are observed in the classification performed by the LR and NB models. However, despite these results, when considering the practical landscape, they show substantial promise as compared to earlier research findings. In our analysis, we dedicated our efforts to evaluating the prowess of the (CNN + SVM) hybrid model, which outshines its counterparts in terms of effectiveness. Our focus zoomed in on the specific confusion matrix linked to the (CNN + SVM) model insights as the best model. This matrix played a pivotal role in gauging the model’s accuracy when it came to distinguishing between the realms of DoS and DDoS conditions.

Figure 7

A confusion matrix that encompasses all the models.

In our confusion matrix, we bore witness to the (CNN + SVM) model amassing an impressive tally of TPs in specific categories, a testament to its precision in making accurate forecasts. For instance, during this close inspection, the hybrid model’s commendable strength truly shone. It adeptly identified a substantial 3,399 instances as malicious attacks, aligning its affirmative “malicious” (positive) predictions remarkably well with the actual instances of malicious activities. This spotlights the model’s innate capacity to astutely detect and adeptly label genuine threats. The model’s proficiency further manifested as it correctly classified 3,413 instances as benign. These instances, tagged as “benign” (negative) by the model, impeccably harmonized with their genuinely harmless nature. This underscores the model’s finesse in meticulously discerning and categorizing routine activities. However, the model’s journey encountered an intriguing twist as it attempted to predict 42 instances of benign behavior. Contrary to the model’s classification of these instances as “benign” (negative), their actual intent was malicious. This revelation shines a light on instances where the model grappled with accurately identifying truly malicious actions. Our study’s findings found a poignant echo in the matrix’s narrative. While the hybrid model effectively attained notable TPs and TNs, emblematic of its precision in classification, sporadic instances of misclassification emerged as well.

Nevertheless, the hybrid model resounds with significant potential in reinforcing IoT device security by adeptly detecting and addressing the intricacies entailed in DoS and DDoS attacks.

It appears that the ROC curve is applied to various models as shown in Figure 8. The ROC curve is a graphical depiction illustrating a binary classification model’s performance at different classification thresholds. An AUC value of 1.0 signifies flawless classification performance. Both SVM and KNN models have a ROC curve AUC of 0.98, implying that these models demonstrate highly commendable performance. Moreover, the CNN + SVM model boasts an AUC of 0.999, indicating impeccable classification performance based on the ROC curve assessment. This compellingly underscores the model’s proficiency in accurately distinguishing between positive and negative instances.

Figure 8

ROC curve.

4.3 Discussion

In this section, we discuss the performance evaluation of various ML models for classifying IoT traffic as either malicious or benign. Our study successfully identifies models that exhibit strong performance in terms of accuracy, recall, F-score, and the AUC. The hallmark of our innovation lies in the seamless integration of CNN and SVM, a union meticulously orchestrated within our groundbreaking CNN + SVM framework. This fusion acts as an invigorating catalyst, consistently propelling us beyond the benchmarks set by our peers. Our model boasts an impressive accuracy rate of 99.8%, accompanied by precision, recall, and F1-score metrics of 100, 99, and 99%, respectively. These outcomes stand as a testament to the strength and efficacy embedded within our approach, affirming its proficiency in precise classification.

Table 9 offers a comprehensive comparison between our research and other related studies within the realm of intrusion detection. This table provides insights into each study’s research number, source, best model employed, and the corresponding accuracy achieved. The studies showcased encompass a diverse array of ML models, spanning from DNN, CNN, and SVM, to Smart Grid Networks, DBNs, and RF. This reflects the wide spectrum of methodologies explored within the intrusion detection field. The recurrent use of deep learning models such as DNN, CNN, and DBFs across various studies underscores their robust performance, highlighting the potential of deep learning in uncovering intricate patterns within network data. Worth noting is that some studies have chosen the same model more than once, amplifying the model’s proven efficacy. For instance, both Asad et al. [18] and Vinayakumar et al. [23] have leveraged the DNN model as their optimal choice, achieving notable accuracies of 98 and 99.2%, respectively. Similarly, SVM emerged as the preferred model in previous studies [20,22], yielding accuracies of 85 and 80%, respectively.

However, it is our research that takes center stage in this comparative narrative. Through the ingenious fusion of CNN and SVM, we achieved an astonishing accuracy of 99.8%, as depicted in Figure 9. This bold amalgamation exemplifies how hybrid models can harness the strengths of both NNs and traditional ML methods, yielding unparalleled outcomes in accurately identifying intrusion patterns. Our research propels us forward, establishing new benchmarks in the realms of intrusion detection and network security. The comprehensive comparison detailed in Table 9 highlights the gradual evolution of methodologies, progressing from NNs to hybrid approaches. Our contribution, showcased by the CNN-SVM hybrid model, not only raises the bar for accuracy but also encapsulates the synergy between established and cutting-edge techniques. Through this comparative analysis, we embrace the essence of progress in fortifying network environments against evolving threats. Future research endeavors could delve into exploring the model’s adaptability to evolving attack strategies, along with strategies to enhance its resilience against adversarial attacks. Moreover, extending the model’s capabilities to encompass a broader array of attack types and subjecting it to real-world deployment scenarios would substantially validate its practical utility and robustness. Additionally, optimizing the model’s computational efficiency to cater to resource constrained IoT devices, coupled with the exploration of collaborative security mechanisms, holds the potential to pave the way for the development of more comprehensive and effective IoT security solutions. The outcomes of this study serve as a robust foundation for these future directions, contributing to the ongoing evolution of IoT security measures.

Figure 9

Comparison of our model with previous studies.

4.4 Trust of network (ToN)-IoT dataset

In this section, we also tested the proposed approach on another dataset to ensure scalability and to see how robust the proposed CNN-SVM model is against other attacks that may not have been present in the CIC IoT 2023 dataset.

The ToN-IoT dataset used in this study compiles a wide range of data from a whole industrial internet of things (IIoT) system. This comprises telemetry data from linked IoT devices, operational records from the Linux and Windows operating systems, and detailed network traffic data for the IIoT system. The dataset, created by UNSW Canberra IoT Labs and the Cyber Range, is formatted in CSV format and intended to reflect activities inside a medium-scale IoT network. Each entry in the dataset is labeled to indicate whether it corresponds to normal operations or attack behaviors, and includes a sub-category that identifies the type of attack ranging from ransomware, password attacks, and scanning to DDoS, data injection, backdoor, cross-site scripting (XSS), man-in-the-middle (MITM) attacks, and more.

The ToN-IoT dataset used in this study collects a wide range of data from a full IIoT system. This comprises telemetry data from linked IoT devices, operational records from the Linux and Windows operating systems, and detailed network traffic data particular to the IIoT system. The dataset, which was created by UNSW Canberra IoT Labs and the Cyber Range, is formatted in CSV format and intended to depict activities inside a medium-scale IoT network. Each entry in the dataset is labeled to indicate whether it corresponds to normal operations or attack behaviors, and includes a sub-category that identifies the type of attack – ranging from ransomware, password attacks, and scanning to DoS, DDoS, data injection, backdoor, XSS, MITM attacks, and more. This structure aids in the detailed analysis and identification of various threats launched against different IoT and IIoT sensors, with full details available in the ToN-IoT repository.

In our research, we verified the CNN-SVM model using the ToN-IoT dataset, achieving impressive performance metrics that demonstrate the resilience of our suggested approach. The confusion matrix, a critical tool for assessing the efficacy of classification models, exhibited excellent predictive power. Our model achieved an accuracy of 99.85%, suggesting that practically all predictions matched the actual classifications. Precision was calculated at 99.70%, demonstrating the model’s high level of accuracy in identifying TPs. Surprisingly, the model obtained a recall of 100%, indicating that all instances of targeted attacks in the test dataset were properly detected. Furthermore, the F1-score was 99.85%, demonstrating a balance between precision and recall (Figure 10).

Figure 10

Matrix of ToN-IoT dataset.

These indicators, particularly those derived from a complex and diverse set of IoT attack scenarios in the ToN-IoT dataset, demonstrate the model’s promise as a highly reliable tool for real-time threat identification in IoT systems.

4.5 Scalability considerations

In addressing the key feature of scalability for our CNN-SVM hybrid solution, we recognize the various needs and complexities of bigger IoT networks. Scalability in IoT security refers to the capacity to efficiently adapt to a rising number of devices while retaining high detection accuracy and low latency. Our solution, which is based on modular components, allows scalability by default due to its layered architecture. The CNN component enables feature extraction that may be efficiently parallelized across several devices or computational nodes, addressing the heterogeneous nature and volume of IoT traffic.

The subsequent SVM phase provides a robust classification mechanism that can be scaled vertically, depending on the complexity and volume of the detected features. While our current study does not empirically evaluate these scalability aspects, the theoretical underpinnings of our model suggest that by leveraging distributed computing resources and optimizing algorithmic efficiency, our approach could be extended to larger networks. Future work will aim to validate these scalability propositions through extensive testing across varied IoT environments, focusing on metrics such as throughput, detection time, and resource utilization to ensure the model’s effectiveness and efficiency at scale.

We adopted several essential measures to ensure that the CNN-SVM hybrid model is scalable and computationally efficient, making it appropriate for deployment in a variety of IoT scenarios. First, we created the model with a lightweight architecture that decreases computational burden and memory utilization, making it suitable for resource-constrained devices. This entails using optimized convolutional processes and fewer parameters to achieve high detection accuracy while reducing computing needs. Additionally, we used model compression approaches including pruning and quantization. Pruning helped eliminate extraneous model weights, and quantization reduced the precision of the numerical parameters, resulting in a substantially smaller model size and faster inference times. These adjustments are critical for deployment on devices with limited computing power.

To improve scalability, the model allows distributed computing, which distributes detection jobs among numerous IoT devices in the network. This technique not only takes advantage of the processing capability of several nodes, but it also minimizes data transmission burden, which is critical for system responsiveness and efficiency.

Furthermore, edge computing techniques were used to process data at the network’s edge, lowering latency and bandwidth consumption by removing the need to send data to centralized cloud servers. This local processing means that the model can quickly respond to attacks in real time, which is crucial for IoT security. Finally, we implemented incremental learning capabilities, which enable the model to dynamically react to new threats without requiring substantial retraining. This component ensures that the model stays successful when network behaviors and attack techniques change, ensuring its durability over time. These innovations collectively ensure that our CNN-SVM hybrid model not only meets the stringent criteria of IoT security in terms of detection accuracy but also tackles the practical obstacles of implementation in diverse IoT ecosystems, which are characterized by their diverse computational constraints.

In addition to the tactics described above, we evaluated our model’s scalability and efficiency by using an additional dataset throughout the training phase. This dataset, which simulates real-world IoT network traffic, was used to assess the robustness and adaptability of our CNN-SVM hybrid model across a variety of IoT scenarios. By training our model on a broader set of data, we demonstrated its capacity to efficiently manage a wide range of network conditions and attack behaviors. This method not only improves the generalizability of our model, but also demonstrates its potential to expand over a broader range of IoT networks without sacrificing computing efficiency. The successful application of our approach to numerous datasets emphasizes its potential for wider implementation in IoT devices with diverse processing capabilities, affirming our commitment to creating a scalable and computationally efficient security solution for the growing IoT landscape.