Framework for identifying network attacks through packet inspection using machine learning

1.1 Anomaly detection

Anomaly identification can be separated into two steps as a technique. The first step involves developing a specification for traditional network traffic. Model generation algorithms or mathematical simulations may be used to generate or learn this model from training data. Traffic is monitored for variations from the standard model in the second step. Using features from the traffic, a model of regular network traffic is developed. A value or symbol that represents network traffic is referred to as a feature in the sense of anomaly detection. These characteristics should reflect traffic behavior and characteristics, while still containing no redundant information to be as light as possible [2].

1.2 Statistical-based anomaly detection

In a process that depends on statistics, deviations are found using statistics. Mathematical methods are used to create historical models. Then, any discrepancies between these simulations and the real-world situation are categorized as anomalies. After a deviation is tracked, its severity is measured and rated. The severity of the phenomenon increases with grade level. For example, the total amount of time a person has spent using the network is compared to the existing count. If the current number of network accesses is one or two more than usual, it is not always a severe situation. Nevertheless, a significant phenomenon exists regardless of how much higher the number is, and in some cases, it is 100 times greater. In general, it depends on how the grading guidelines are written [3].

1.3 Rule modeling-based anomaly detection

In a rule modeling-based approach, the system’s rules are specified, and when they are violated, those instances are labeled as anomalies [4]. In principle, this is similar to how firewalls work. Predefined guidelines are balanced against network traffic by firewalls. If the traffic is not in violation of these laws, it is permitted to proceed. Anything that violates these guidelines is discarded. This means that something that deviates from the norm is considered an exception in anomaly detection [5].

To summarize, the objective of the article is to increase the right distribution ratio while achieving high precision, which is still a proving area. Numerous writers have investigated a number of algorithms, including basic classification, K-Means, genetic algorithms, and Support Vector Machine (SVM) techniques, and they have demonstrated the effectiveness and accuracy of employing these algorithms. We have introduced the feature extraction method known as “k-means clustering” in this study. This method, which has roots in signal processing, divides a set of n observations into k clusters, each of which derives its origin from the observation with the closest mean. This study uses Python and a dataset from KDDcup99 to analyze the stream and its implementation and applications. The efficiency of the result reveals how well the intended job was performed in relation to other available alternatives.

Organization: The article is structured into many modules, where the first section states the introduction, followed by Section 2, which identifies network layer attacks. Section 3 states the literature survey, followed by Section 4, which discusses the problem definition. Section 5 states the proposed algorithm. The penultimate section of the manuscript discusses the result analysis, and the ultimate section is the conclusion and the future work section.

2.1 Data collection

The intrusion detection system (IDS) is tasked with receiving the data, which this module fulfills. A chronicle is kept, and then the material is written down. For example, network-based intrusion detection system (NIDS) (structure-based IDS) keeps track of how the framework is used and how it is implemented.

2.2 Feature selection

To choose the right part, there is a lot of information in the structure, and they are usually measured for resistance. For example, the Internet Protocol (IP) addresses, display type, header length, and size of the source and target structures can all be used as interference keys [8].

2.3 Analysis

To discover the accuracy, the investigation of information is finished. Standard-based IDSs breaks down the information where approaching traffic is inspected against a predefined mark or an example. Anomaly-based IDS is another approach that makes use of numerical models to analyze the behavior of the framework.

2.4 Action

This capability is concerned with the framework’s response to an attack that is destructive and intolerable. Depending on what is required, it will either send an email/alert symbol or have a functional effect on the framework by falling packets so they do not reach it or closing the ports. Interruptions can be defined as a series of activities that attempt to balance the integrity, privacy, and accessibility of assets on a framework [9].

• Integrity: Data transparency refers to making sure the data has not been tampered with either in transit or storage. When networking terminals and servers are checked and their physical environments are controlled, their data access is limited, and procedures of robustness in authentication are ensured, all these steps are taken to ensure maximum privacy.

• Confidentiality: Unauthorized users have not downloaded or revealed records indicating secrecy.

• Availability: On the basis of opportunities, accessibility ensures that systems are sufficiently strong and open to customers (e.g., when clients require them). Refusal of administration, on the other hand, occurs when clients are unable to obtain the funds they need on a timely basis.

5.1 Classification algorithm

The information extraction is performed using the negative selection algorithm, which is then extended to the K-MEANS classification. Finally, depending on the final classification stage, intrusion or non-intrusion data is assigned.

5.2 Diagram of the process

The method flow is depicted in Figure 2.

Figure 2

Flow diagram of the complete process.

The flow map of the device that takes feedback and processes the attack detection for further classification is seen in Figure 2. KDDCup99 dataset is taken and pre-processed to identify if there are any NaN or null values. Once the data are preprocessed, it checks for all the features that lead to the identification of network attacks. Once the features grouping for such negative data classification is done, a label is set for each category of attack in the training data. The machine learning approach is applied after classification to detect attacks. In this work, k-mean classification is done, and the process repeats until the training is done. This training data can be further applied to real-time network packets to identify new attacks.

5.3 Pseudocode

The pseudocode functionality of the proposed approach.

Input: KDD dataset library initialization, indexed dataset storage.

Output: Data classification, computational parameters, confusion matrix, comparison analysis graphically and statically.

Steps:

Begin [

Loading DS (i-n) KDD  #KDDCup dataset is used

{

DataloadModel ()  #Processing for any NaN or Null values and to fit into ML

}

Processing of protocol Selection ()  # Feature selection applied for each type of attack with label

{

NegativeprotocolCollect ();  # Attack and benign classification based on the features is done.

Filtration ();

NegativeSelection;

NegSelRefine ();

}

Obtaining refined data;

Processing of K-MEANS:  #Training to the algorithm with reduced selected features

K-MEANS ()

{

Initializing of N layer values;

Computation of sigmoid Function ();

hidden LayerInit ();

weight Optimization ();

}

Performing K-MEANS Classify();  # applying the K-Means to check the efficiency

Return classification results;

Computations()

{

confusionMatrix();

efficiencyParam();

}

Return comparison analysis;

]

end;

Thus, the above pseudocode discusses the core functions of the algorithm performed over the execution of the proposed system.

6.1 Programming environment

6.2 KDD Cup 99 dataset

With the help of the KDD Cup 99 dataset, our standard estimations were carried out. The KDD Cup 99 dataset is a standard set of independently verifiable data, including a wide range of simulated intrusions into a military network environment, and is used to carry out the investigations by employing the proposed techniques in our research. For testing IDS, it serves as a standard. In 1998, the software was developed and was named DARPA Intrusion Detection Assessment Software. MIT Lincoln Laboratory was the key operator of this software. The aim of the evaluation program is to evaluate the research and investigation in the interruption exploration. A common dataset was implemented by the testing program, e.g., the dataset of DARPA98. This dataset was inclusive of interruptions of a large variety that were recreated in the army device scenarios. To evaluate the exhibition of IDS, the DARPA98 dataset was used as both a preparation and research dataset. Lincoln Laboratory arranged for 9 weeks to dump transmission control protocol (TCP) data that was supposed to be extracted. Data for 7 weeks of planning are included in the DARPA98 dataset, and in the same way, 2 weeks were its study period. For the DARPA98 dataset, the KDD cup 99 dataset is considered as an updated version. In collaboration with KDD99, a competition was organized and named as Third International Knowledge Discovery and Data Mining Tools Competition. On Knowledge Discovery and Data mining, the fifth international conference was also conducted with the help of the dataset of KDD Cup 99. The aim of the test was to create a device interruption marker, a predictive model capable of detecting bad associations, also known as interruptions or dangerous untolerable attacks, as well as great typical associations.

• The interruption recognition dataset KDD 99 is made up of several parts: kdd cup data: It contains full preparing information, e.g., traffic systems and their 7 weeks. The number of records presented is 4,940,210.

• kddcup.data_10_percent: comprises 10% of dataset in full preparing. The number of records presented is 494,021.

• kddcup.testdata.unlabeled: fully tested information and data is present in it. In this set, each of the test information is unlabeled. The number of records presented is 2,984,153.

• kddcup.testdata.unlabeled_10_percent: for fully unlabeled and tested data, it contains only 10%. The number of records presented is 311,029.

• kddcup.newtestdata_10_percent_unlabeled: the refined number of 10% is present in it for the fully unlabeled testing data. The number of records presented is 311,079.

• Corrected: in its named structure, it contains the testing data or information. The number of records presented is 311,029.

6.3 Different attacks

In KDD Cup 99 dataset, there are 25 different forms of attacks. There are four categories of multiple assaults. U2R attacks, R2L, and probe attacks are all included in DoS attacks.

6.4 Training and testing dataset for the proposed framework

On the subset of 10% KDD dataset, this system is generally tested on the basis of a larger number of records of training sets in the 10% KDD'99 data collection. One categorical highlight in the Labeled Dataset indicates whether the attack is general or precise in nature. The planning dataset was kddcup.data 10%, and the checking dataset was rectified. There were 490,015 connections in the subset used for preparation, with 392,737 assaults and 97,278 regular ones. There were several different networking attacks in this given subset, and these are named as: ipsweep, neptune, pod, smurf, teardrop, back, guess passwd, ipsweep, neptune, pod, smurf, teardrop 2,203 return, guess passwd, 1,247 ipsweep, 107,201 neptune, 264 pod, 280,790 smurf, and 979 teardrop attacks are some of the main kinds of attacks that are included in this particular training dataset. In the research or investigative dataset, there were as many as 288,555 connections; the total number of attacks was 227,962; and the total attempts recorded were 60,593. There were as many as 1,098 back cases; those who guessed passwords were 4,367; ipsweep was 306; Neptune was 58,001; pod was 87; smurf was estimated to be 164,091; teardrop attacks were 12 kinds; and all these are included in the dataset of this investigation.

6.5 DoS

The DoS assault takes place. The interloper or programmer sends a lot of solicitations, making the server highly occupied, and in the same way, it keeps the memory assets occupied as well for the consideration of serving some highly genuine structures and administrative demands. Finally, it also denies the entrance of any client into a particular machine. Under DoS, attacks of many kinds are included:

• Back attack: Under DoS class, the bad attack comes at first. It was initially launched on the Apache web server, which at the time was flooded with numerous requests with front slashes (/) in large numbers, and this is generally a character in the URL description.

• Neptune attack: The Neptune attack is also included in the DoS class. Memory resources can be made unreasonably full for any unexpected type of loss by delivering the TCP packet to begin the session of TCP as previously described. It means memory resources can be made extremely full for any unforeseen sort of loss by delivering the TCP packet to begin the session of TCP as previously explained. Simultaneously, this package contains a three-way handshake, and it is intended that it will result in TCP relationships being established between two hosts. As a result, the unfortunate fatality is unable to finish the handshake, despite the fact that a significant amount of framework memory was distributed for this relationship. After a huge number of these packages have been sent, the injured person runs out of memory resources.

• Pod attack: “The ping of death” occurs when an attacker delivers an excessively large packet size (more than 65,535) as a ping request. In spite of the fact that it is illegal to send a ping heap of this size, a package of this size can be sent if it is separated. The victim’s computer frequently crashes as a result of a cushion flood in this attack.

• Smurf attack: Internet Control Message Protocol (ICMP) is included in attacks of this kind in enormous quantities. These bundles are there with ridiculed sources of the suggested injured individuals for IPs. These communicate to the system with the help of broadcasting IP addresses. On the system, as a result of this, all hosts are supposed to answer to the demand of ICMP, and hence, this makes the traffic genuine for the PC’s unfortunate casualty. If n hosts, e.g., are connected to the network, in this case, the attacker can also make a simple host list for packet reply to the victim, which can be done by sending packets to the concerned network.

• Teardrop attack: In the case of teardrop attack, the IP of the attacker is added with a puzzled value offset in the fragments subsequent to it. If the system fails to recognize a way to handle circumstances like this, the system is likely to crash as well.

• DoS Slowloris attack: On Wednesday, DoS Slowloris attacks are likely to happen, and they also correspond to other DoS attacks in the data set named as CICIDS2017. The depletion of device resources of victim is the main aim of this kind of attack, and it can block legal or legitimate users from data reception.

• DoS SlowHTTPTest attack: Every Wednesday, the DoS SlowHTTPTest is held, and to other DoS attacks in the dataset of CICIDS2017, these kinds of attacks correspond. The TCP window size functionality provides additional benefits for attacks of this kind, and hence, the resources of the victim’s server are wasted and legitimate users fail to use their provided services.

• Heartbleed attack: Heartleech, a tool written in the C programming language, is exploited in the Heartbleed exploit tool, and in this assault, it is used. This type of attack occurs on Wednesday afternoon, as confirmed by the CICIDS2017 dataset report.

6.6 U2R or client-to-root

Attacks of this kind are highly intolerable and harmful. With a typical client record and endeavors, the programmer starts on a PC, and the aim is to abuse the PC vulnerabilities for the enhancement of client benefits predominantly.

6.7 R2L or remote-to-user

Attacks of this kind are harmful and intolerable, where packets are sent to gadgets by a client to the computer organizer. There is no such entrance for the attacker, and therefore, to uncover the vulnerabilities of the gadget and endeavor benefits the client has, his system is badly affected.

• Guess_Passwd attack: R2L attacks also include Guess_passwd attacks. These special kinds of attacks comprise an intruder that keeps on guessing the user’s password and potential passwords; the main aim is to gain access to the account of any user. For an attack, any such service requiring a password to enter is often the priority.

6.8 Probing

This is the case when hackers check the networking structures and computers for any useful bugs that can be utilized for the system breach.

• Ipsweep attack: IPSweep strike is included in the Probe class. In the sweep monitoring processes, the attack of IPSweep makes a decision regarding the networking host. The operating hosts are specified by this along with various kinds of services; the details are then used by attackers for the attack and for this purpose, and they always look for compromised computers.

6.9 False positives and false negatives

There are four potential events whose ratio is tracked to determine the IDS’s performance and identification accuracy. These events are represented in the Figure 3.

There are some legitimate events that are regarded as false negatives, and these are also labeled mistakenly as anomalous. Those events that are identified with accuracy are known as true positives. The anomalous events are also known as false negative events, whereby the detectors are ignored and hence anomalous attacks cannot be identified [25]. The events that are true negatives are those that have been properly identified as lawful actions. A network operator must evaluate the phenomenon or interference to decide if it is a false positive or false negative.

6.10 Network data mining

Network data mining (NDM) may be used for a variety of purposes. The first step is to produce information about the monitored data that have been processed. This allows for the identification of dominant traits and outliers in data records that may be considered anomalous or suspect. Second, NDM may be used to describe rules or patterns that are unique to some types of traffic, such as standard web traffic or traffic seen during a DoS attack. These principles and trends can be extended to new collections of monitoring data to see if they have the same properties and features as the first. NIDS and traffic analyzers that define and identify traffic flows are two obvious implementations that benefit from such laws and patterns.

• Accuracy analysis:

  A “True Positive” is a fact if interference is precisely predicted. We will term it “True Negative” if we do not find any dangers. If IDS detects an intruder but the statement is inaccurate, a “False Positive” alert is raised. A “False Negative” (FN) incident occurs when a non-intruder is found and the intruder continues to operate. This is the worst-case scenario, in which all identifying conditions are operating well and a false negative is issued. To test our IDS, we used these words: accuracy and identification rate. Total results divided by total intruders are termed as Accuracy (ACC).

  ACC = TP + TN TP + FP + FN + TN .

• Detection rate:

  On the other side, the detection rate (DR) is the percentage of alarms that lead to real intrusions.

  DR = TP TP + FN .

• Statistical analysis:

Table 1 summarizes the results of our study in accordance with the above notions. Our IDS is capable of detecting abnormalities with a high degree of precision and a high rate of detection.

Precision = TP/(TP + FP)

Recall = TP/(TP + FN)

F-Measure = 2 × ((Precision × Recall)/(Precision + Recall))

There is a difference analysis between the methods, which can be seen in Table 2.

6.11 Graphical representation for comparison analysis of algorithms

An analysis of the result is presented in Figure 3. This section gives an understanding of the statistical graphical analysis.

Figure 3

Comparison between the analyses of the algorithms.

This section compares the results of the old technique and the suggested approach based on the implementation’s measured outcomes. KNN Algorithm results are shown in Figure 4, and K-Means Algorithm results are shown in Figure 5.

Figure 4

KNN algorithm graph.

Figure 5

K-Means algorithm graph.