DBSCANI: Noise-Resistant Method for Missing Value Imputation

1 Introduction

Real-world databases are highly susceptible to noisy, missing, and inconsistent data. Missing values are present in many instances of data set if there is no recorded value for several attributes and noise is a random error or variances in a measured attribute or variable [11, 20]. These are attributed to their huge size and their origin from multiple heterogeneous sources [11]. Low-quality data leads to low-quality mining results. Missing values occur due to a number of reasons such as errors in the manual data entry procedures, equipment errors, or incorrect calculations. The presence of missing values (MVs) in data mining produces several problems in the knowledge extraction process such as loss of efficiency, complications in the management, and analysis of data. Data mining algorithms cannot be directly applied on the incomplete data set. Three types of techniques have been discussed in the literature to deal with the missing value. The first approach deletes the instances that are having missing entries. But this method fails when the data set consists of a huge number of incomplete values. The second kind of approach such as model based defines a model for the partially missing data and base inferences on the likelihood under that model. These models estimate the parameters by maximum likelihood, expected maximization, and other methods [13]. The third type of imputation techniques replaces the missing values by imputation such as the mean for numerical attributes and the mode for nominal attributes. These imputation techniques are independent of data mining algorithms used for the purpose of classification, clustering, or association. Hence, data miners can select any of the imputation techniques that suit their data set and apply learning algorithms on the imputed data set.

Although, a lot of work has been done in the area of missing value imputation, a slight attention except [1, 22, 25] has been given on noise in the present MV literature. The noise may come from many sources such as data transformation, data entry, or data compilation [23]. The presence of noise in the data can lead to false results during imputation. Therefore, there is a need to analyze the current imputation techniques in the presence of noise.

In this paper, we have proposed a novel imputation technique using density-based clustering approach and hypothesize that we can predict missing values correctly in noisy environment. Very few researchers have imputed missing values in the presence of noise. Density-based clustering algorithm [4] proposed by Kriegal groups the objects according to their density in spatial data bases. The high-density regions are known as clusters, and the low-density regions refer to the noise objects in the data set. A lot of experiments have been performed on two data sets, i.e. the Iris data set from the life science domain and 2D data set from shape data sets [2, 10]. We also have compared our results with the K-means imputation technique [13].

The rest of the paper is divided into four sections. Section 2 discusses the related work in MV literature. Section 3 presents the proposed work for imputation. Section 4 gives the experimental results and analysis drawn from them. Finally, Section 5 concludes the complete paper.

2 Related Work

The missing values present in the data set generate several problems like loss of efficiency and biased estimates in knowledge extraction processes. Different approaches for missing value imputation in MV literature are employed to prepare data in order to avoid the negative effects in the analysis of data mining algorithms. Broadly, they can be classified into the following categories:

• List wise deletion: The first and the most trivial approach is deleting the objects, which consists of incomplete entries. This method is based on list wise deletion of those examples, which have incomplete column values. But this approach is appropriate only when there are a couple of missing values in the data set. Although as the quantity of missing values increase, significant data may be lost by deletion of the incomplete instances [13].

• Mean/mode substitution: This is a simple way to impute the missing data. It replaces the missing values by the mean or mode of all the observations or a subgroup at the same variable. It consists of replacing the unknown value for a given attribute by the mean (quantitative attribute) or mode (qualitative attribute) of all known values of that attribute. But replacing all missing records with a single value distorts the input data distribution [21].

• Maximum likelihood procedures [3]: These methods are used for estimation of parameters of model by making use of the expectation–maximization algorithm [13, 21]. These methods make assumption of model distribution for the variables, such as a multivariate normal model, which are very much sensitive to outliers [21].

• Multiple imputation [5, 18]: This method imputes the value of a missing variable more than once. Then, analysis is done by averaging them [21].

• Machine learning-based imputation [13]: There are a number of approaches for missing value imputation include choosing the most common value among the neighbors, weighted imputation based on k-nearest neighbors, decision tree based, association based, fuzzy logic, neural networks, SVM, and clustering-based imputation, e.g., as K-means, weighted K-means, fuzzy c means [6, 17, 19], etc. Recently, Ref. [24] has proposed an approach for mixed data set consisting of discrete as well as continuous attributes. Moreover, a novel clustering-based multiple imputations via Grey relational analysis has been stated in Ref. [21].

Most of the approaches discussed above ignore an important issue like noise that may be present in the data set in addition to missing values. The noise may arise from various resources such as data transformation, the process of data collection, data entry, etc. The presence of noise may introduce some harmful outcomes. A little work like robust imputation based on group method of data handling (RIBG) [1, 7–9, 16, 25] has been done in the MV literature with noise. RIBG proposed by Ref. [25] is based on group method of data handling. This method initializes the missing values with mean imputation to generate the initial data set. This data set could be initialized with better imputation technique so that the number of iterations will be less. Missing value imputation by Ref. [25] takes more execution time. Moreover, Van Hulse and Khoshgoftaar [7] have only investigated the performance of five popular imputation techniques on noisy software measurement data set, but they have not proposed any technique to deal with missing values in the presence of noise.

In this paper, we have proposed an imputation technique based on DBSCAN mechanism. We hypothesize that the missing values can be calculated more accurately in the presence of noise using our proposed method. We have validated our hypothesis by experimenting on the Iris data set from standard repository [2] as well as 2D data set [10]. Moreover, we have examined the impact of noise by introducing different noise levels and different missing rates. Experimental results show that our algorithm performs better compared to K-means imputation [13] in the presence of noise.

3 Proposed Work

Our motivation for this study is based on the clustering algorithms that groups instances within a specified radius on the basis of distance principle such as the Euclidean distance. Density-based spatial clustering of applications with noise (DBSCAN) [4] proposed by Kriegel is a clustering technique that locates the cluster on the basis of density. This approach calculates the density in the data set by counting the number of points within a specified radius. It is relatively noise resistant and handle clusters of arbitrary shape as against K-means clustering algorithm.

We have applied density-based clustering method on whole data set in order to group the instances in cluster based on their distances. It also separates out the noisy instances, thus, minimizing their impact on MVI. This tends to better accuracy compared to the case when no elimination of noisy data is made prior to MVI. Later on, we make use of cluster information to find the nearest neighbors of missing value attributes instance. The average of missing attribute values of the nearest neighbors is used to impute missing value attributes.

4 Experimental Framework, Results, and Analysis

In this section, we have investigated the impact of noise on imputation techniques with a different missing rate to validate the efficiency of the proposed approach. Moreover, we have compared the proposed method with a popular K-mean imputation technique (KMI) [13].

4.4 Results and Discussion

We have performed the experiments on two data sets, namely, Iris and 2D. Figures 1 and 2 illustrate all the combinations of the different missing rates and noise levels in the Iris as well as the 2D data set, respectively, where the x-axis indicates the level of missing rate, and the y-axis represents the RMSE calculated by eq. (4). There are two curves in each part of Figures 1 and 2. The red curve represents the result of the K-means imputation [13], and the blue curve shows the result of the proposed method, namely, DBSCANI.

Figure 1:

Experimental Results on Iris Data Set.

(A) Noise level = 0%. (B) Noise level = 3%. (C) Noise level = 6%. (D) Noise level = 12%.

Figure 2:

Experimental Results on the 2D Data Set.

(A) Noise level = 0%. (B) Noise level = 3%. (C) Noise level = 6%. (D) Noise level = 12%.

We have carried out numerous experiments by generating four levels of noise such as 0%, 3%, 6%, and 12% in one variable. For each level of noise, different levels of missing rates such as 2%, 4%, 6%, 8%, 10%, and 12% have been used to control the missingness in the data set. Moreover, we have conducted the experiments for KMI to compare with the proposed approach.

4.4.1 Discussion for Iris Data Set

The following are the observations from Figures 1 and 2:

• For DBSCANI at 0% noise level as shown in Figure 1A, at a small missing rate (2%), RMSE is more compared to the medium missing rate, i.e. 4%. For a higher missing rate (6%, 8%, 10%, and 12%), the error is more compared to the medium missing rate as well as the small missing rate. For KMI at 0% noise level, RMSE increases as we increase the missing rate from 2% to 12%.

• For DBSCANI at 3% noise level as shown in Figure 1B, increasing the missing rate from 2% to 12%, there is no significant change in RMSE. But KMI gives the highest imputation error at 4% compared to the other level of missing rate.

• At 6% of noise level as shown in Figure 1C, the impact of noise is significant, and imputation error increases at a higher missing rate (6%, 8%, 10%, and 12%). For KMI, the impact of noise is insignificant and the average imputation error is less compared to the 3% level of noise.

• At 12% of the noise level as shown in Figure 1D, DBSCANI performs the same compared to 6% of the noise level. KMI also performs better compared to the 6% level of noise.

• For DBSCANI, at a low noise level (3%) as shown in Figure 3, the impact of the noise is limited, and imputation errors are less. Sometimes, a small amount of noise (3%) even seems to improve the results as in Figures 1B,C and 3. However, when the level of noise is relatively high, the introduction of more noise (6% and 12%) will weaken the imputation results. For KMI, increasing the noise level decreases the imputation errors. But overall, the imputation error achieved by the proposed method is much less compared to KMI.

Figure 3:

Average RMSE versus Noise Level for Iris Data Set.

Therefore, we can conclude that DBSCANI performs better compared to KMI at different noise levels, i.e. 0%, 3%, 6%, and 12% as well as at a different missing rate.

4.4.2 Discussion for 2D Data Set

The following are the observations from Figures 2 and 4.

• For DBSCANI, at a small missing rate (2%), RMSE is more compared to other missing rates. The value of RMSE keeps on decreasing until 10% of the missing rate. At 12% of the missing rate, RMSE is a little higher compared to 4%, 6%, and 10% missing rate. For the highest missing rate, RMSE is further reduced. For KMI, RMSE is less at a lower missing rate (2%) at all noise levels, but it is drastically increased for all other missing rates.

• For DBSCANI, at a low noise level (3%) as shown in Figure 4, the impact of noise is limited, and imputation errors are less. As we increase the noise in the data set, imputation errors are also increased. For KMI, increasing the noise level decreases the imputation errors. But overall, the imputation error achieved by the proposed method is much lesser compared to KMI.

Figure 4:

Average RMSE versus Noise Level for the 2D Data Set.

Therefore, we can conclude that DBSCANI performs better compared to KMI at different noise levels, i.e. 0%, 3%, 6%, and 12% as well as at different missing rates.

5 Conclusion

Missing value imputation has been a key area in data mining research. A great deal of missing value techniques have been discussed in the literature. The impact of noise has been neglected in most of the methods. Popular clustering methods such as K-means, fuzzy K-means, etc., have been used for MV imputation. This paper propounds a new technique that uses the DBSCAN clustering algorithm instead of the K-mean clustering to deal with the missing value in the presence of noise. We have systematically conducted experiments to analyze the impact of noise with the given missing rate. Moreover, we have illustrated our approach with a different missing rate at various noise levels. We conclude that DBSCANI performs better compared to KMI in a noisy as well as in a non-noisy environment. In the future, we would investigate the proposed approach with the noise as well as the missing values distributed throughout the data set.