Research on surface defect detection method and optimization of paper-plastic composite bag based on improved combined segmentation algorithm

1 Introduction

Paper-plastic composite bags are widely used in various fields such as industry, food packaging, and electronics [1] as practical general carriers. Due to the effect of production equipment, raw materials, and processing processes, it is difficult to avoid that there are more types of defects on the bag’s surfaces, which seriously affect the beauty and quality of the bag products. Some common surface defects are shown in Figure 1. In order to ensure the final quality of the paper-plastic composite bag, the surface quality must be detected in the production process.

Figure 1

Common defect patterns on the surface of paper-plastic composite bags.

At present, most of the intelligent automatic monitoring systems used in surface quality detection are developed based on machine vision systems with imaging processing and recognition technology as the core. Based on this technology, some researchers have carried out a lot of studies on defect detection of rolling elements, leather, fabrics, tiles, pipes, etc. For example, Ren et al. [2] conducted research on the study status of defect detection technology based on machine vision, and expounded the latest progress of machine vision in the field of industrial defect detection. Prappacher et al. [3] carried out scanning defect detection on rolling element surfaces based on neural image segmentation. Based on fast convergence particle group optimization integrated classifier, Jawahar et al. [4] studied the visual inspection system of leather surface defects. Fouda [5], Daniel et al. [6], Lizarraga-Morales et al. [7], and Yassine and Mohamed [8] studied fabric flaw spots and pattern fabric defects based on some related methods of machine vision such as image analysis. Hanzaei et al. [9] used a multi-class support vector machine (SVM) classifier based on statistical pattern recognition theory to accurately identify the surface defects of ceramic tiles. Mohamed et al. [10] monitored physical defects such as dents, cracks, corrosion in oil and gas pipelines based on image classification of decision tree. Mengkun et al. [11] detected defects of irregular industrial products with some methods of machine vision, it successfully realized the defect detection of Bluetooth headset and the defect recognition accuracy reached 98%. Cong et al. [12] proposed an improved Sobel algorithm for the edge detection of image defects, which split the defects in the image with Gaussian window adaptive threshold segmentation method. After morphological processing was performed, the outline of defects was accurately drawn. Wu and Lu [13] detected the surface defects of printing boxes by the way of the SVM training model with the multi-layer perception kernel function. Thakkar et al. [14] performed an image classification method based on data analysis to detect longitudinal surface crack defects generated in the continuous casting stage. All these studies have achieved good applications in the respective fields, which have laid a good theoretical and empirical foundation for surface defects of paper-plastic composite bags, although there are few on such kind of research. However, due to the different detection objects and defect detection methods, accuracy and classification effects will be quite different.

One fast image screening and processing method based on the gradient projection difference was adopted for paper-plastic composite surface defects in this article, which is combined with the algorithm [1] of edge detection and adaptive region growing combined segmentation. As a result, five common defects including broken skin, holes, dirty spots, and convex ribs were handled and compared with the normal surface images, so as to realize the fast and effective screening of defect images. Aiming at the classification problem of surface defect images of paper-plastic composite bags, effective feature selection was carried out, and three parameter optimization models of SVM classifiers for surface defects of paper-plastic composite bags were established; in the meantime, the most suitable kernel function was selected by means of comparative calculation to improve the accuracy of the classifier. Finally, one detection platform for the paper-plastic composite bags surface defect was performed and the defect detection method proposed in this article was verified by experiment.

2 Theoretical basis of improved combinatorial algorithms

In this study, through a series of pre-experiments, it is found that, compared with other edge detection operators, the edges of Sobel operator are relatively clear, most of them are real defect edges, and the calculation speed of Sobel operator [15] is faster. While removing the surface image noise of paper-plastic composite bags, Gaussian filter retains relatively complete edge details of the defective part, which has the same denoising effect as bilateral filter, but the running speed is much faster than bilateral filter. Therefore, Sobel operator and Gaussian filter algorithm are selected as the algorithms for image edge detection and denoising. The Sobel operator is often used for edge detection and, in coarse precision, is the most commonly used edge detection operator to be widely used for decades. The Sobel operator consists of two 3 × 3 convolution kernels, which are used to calculate the gray weight difference of the center pixel neighborhood. It is divided into vertical and horizontal Sauber filters Gx and Gy. Gaussian filter is a kind of linear smoothing filter whose value is selected according to the shape of Gaussian function. Gaussian smoothing filters are very effective for suppressing noise that follows a normal distribution. Second, this paper inherits the basic idea of combining pixels with similar qualities in the traditional region growth method. The initial seed points of the traditional region growth algorithm [16] need to be manually selected or specified, which cannot meet the requirements of automatic segmentation. In addition, the image after segmentation is affected by image noise, resulting in uneven edges and holes, which can lead to a decrease in image quality the adaptive region growing method with the combined segmentation algorithm of edge detection and adaptive region growing can reduce these problems and improve the performance of the algorithm.

At first, it is found that the flatness can distinguish the fold and rib images from the other three kinds of defect images, and this article uses Sobel operator to distinguish the wrinkle and convex rib images from the other three defect images with Gauss filtering. After that, the adaptive region growing method with the combined segmentation algorithm of edge detection and adaptive region growing is used to segment the images of broken skin, hole, and dirty point defects. Then, the minimum circumscribed rectangle is extracted to achieve accurate segmentation of the other three types of defect images of broken skin, hole, and dirty spot. At the same time, edge detection segmentation algorithm is used to accurately segment the images of convex ribs and wrinkles. Mean filtering and median filtering [17] are also two denoising algorithms often used in image denoising. OTSU method [18] is an algorithm to determine the threshold of binary image segmentation, which was proposed by the Japanese scholar OTSU in 1979, and it is considered to be the best algorithm for threshold selection in image segmentation. It is simple to calculate and is not affected by image brightness and contrast, so it has been widely used in digital image processing. In order to quantitatively evaluate the performance of various segmentation algorithms, the segmentation algorithm described in this study is now compared with the OTSU, the Sobel operator combined with the median filter algorithm, and the traditional region growing algorithm (the initial seed point is manually selected at the centroid), and introduce the evaluation index – Dice coefficient. The closer the value is to 1, the higher the accuracy of defect segmentation. The calculation formula is as follows:

where A is the defect area drawn by hand and B is the defect area obtained by algorithm segmentation.

The quantitative evaluation of the quality of segmentation results of different algorithms is shown in Figure 2, where the Dice coefficient is the average of 20 sets of experimental results.

Figure 2

Quality assessment of segmentation results of different algorithms.

As can be seen from Figure 2, OTSU method and Sobel combined median filter algorithm perform poorly in defect detection. Although the traditional regional growth method has a high detection accuracy for broken skin, holes, and dirty spots, it has a very low accuracy in detecting convex rib and fold defects and cannot meet the requirements. In contrast, the algorithm proposed in this work has a good performance in the evaluation indicators of various defects, the Dice coefficients are all above 0.93, and the advantages of the algorithm proposed in this work for image segmentation of surface defects of paper-plastic composite bags are shown.

After accurately segmenting the surface defect areas, the shape feature and invariant moment feature of the defects from the extracted binary images are studied as the object, so as to complete the defect feature extraction. The area of the defect region is given by the following formula:

where f ( i , j ) denotes the defect area, M and N denote the quantity of the abscissa and ordinate of the pixel points in the area, respectively.

The perimeter of the defect region is represented by the following formula:

Flatness is expressed as

where H and W are the height and width of the smallest circumscribed rectangle of the defect area, respectively.

Duty Ratio calculation formula is as follows:

The formula for calculating the circumference ratio is as follows:

The circularity calculation formula is as follows:

The central moment μ pq of order p + q of f ( x , y ) is defined as

where x c and y c are the barycentric coordinates of the defect area.

The normalized central moment of order p + q of f ( x , y ) is defined as

where r = p + q 2 + 1 .

Based on the moments of the region defined above, the normalized second-order central moments η 02 , η 11 , η 20 and the normalized third-order central moments η 03 , η 12 , η 21 , η 30 can be obtained. Therefore, seven invariant moments to translation, rotation, and scale transformation of the defects can be defined as φ 1 , φ 2 , φ 3 , φ 4 , φ 5 , φ 6 , and φ 7 .

It can be seen from Eq. (9) that when calculating the normalized central moment of the image, the number of powers of different orders varies with the order, so the obtained central moments of different orders will be quite different in magnitude. In order to reduce the sensitivity of high-order central moments to pixel coordinate changes, the logarithm of seven invariant moments is used to describe the target. In this way, the final seven invariant moment feature vectors can be expressed as follows:

The calculation flow chart in detail is shown in Figure 3.

Figure 3

Calculation flow chart.

3 Feature extraction and selection

According to the above feature calculation formulas, the binary images of the surface defect areas on the paper-plastic composite bags are extracted by the method of image segmentation, then the shape features and invariant moment features of five types of defects are extracted. This article conducts feature extraction experiments on five types of defect samples such as wrinkle, broken skin, hole, dirty spot, and convex rib. Binary images of some samples are shown in Figure 4. The 6 shape features and 7 invariant moment features of the corresponding samples are list in Table 1.

Figure 4

Two groups of common defects of binary image. From left to right are wrinkles, broken skins, holes, dirty spots, and convex ribs.

It can be seen from Table 1 that due to the relatively slender shape of the wrinkle and the convex rib, the corresponding flatness is smaller than that of the three types of defects such as broken skin, hole, and dirty spot. In addition, the defects such as broken skin and hole most appear in large blocks, so their areas are generally larger than that of wrinkle, dirty spot, and convex rib as well. The dirty spot defect area is small, so its perimeter is generally smaller than that of other defects. Hole and dirty point defect areas are more compact than that of wrinkle and convex rib defects, so their circularity is larger. For the feature of perimeter ratio, most of all kinds of defects are gathered in the same small range, and it is not separable. Although the duty ratios of wrinkle and convex rib defects are basically smaller than that of other defects, their distinguishing characteristics and flatness are repeated. The first two items of the invariant moment of the same type defect image are relatively close, and those for the different types defect images are quite different. In addition, the value of the last five items of the invariant moment of the same type defect image varies greatly, which is unreliable. Therefore, six feature parameters in the defect areas are finally selected such as the area, perimeter, flatness, circularity, φ 1 , and φ 2 , and they will be input as the feature vectors of the classification and identification algorithm.

4 Surface defect classification of SVM based on optimization model

Because the probability of surface defects of paper-plastic composite bags is small, only limited defect samples can be obtained on site, so the classification of surface defects of paper-plastic composite bags can only be carried out in the state of small samples. Support Vector Machine (SVM) [19] is based on statistical learning theory, Vapnik-Chervonenkis Dimension (VC dimension) theory and structural risk minimization principle. The basic idea is to find the optimal classification surface to obtain the maximum classification interval between two types of samples. The SVM method is based on the optimal classification surface in the case of linear separability. SVM has the strong promotion ability in solving the problems of small sample decision-making, so when classifying the surface defects of paper-plastic composite bags in the state of small samples, it can be used to classify the surface defect images. At the same time, the choice of kernel function is also one of the key factors to achieve the best performance of the classifier. The classification model is trained with four kernel functions, respectively, under the same penalty factor (penalty factor is 2). After comparing the influence of different kernel functions on the accuracy of the classifier, the Radial basis function (RBF) kernel function K ( x i , x j ) = exp − 1 σ 2 ∥ x i − x j ∥ 2 is selected as the kernel function of the SVM. RBF is one of the commonly used kernel functions. It can map primitive features to infinite dimensional spaces. This function resembles a Gaussian distribution and is therefore called a Gaussian kernel function. The RBF kernel function is often used in SVM because it can map samples to higher-dimensional spaces and handle examples of nonlinear relationships. At this time, the defect classification accuracy of the SVM model is about 90.5263%. The research by Cherkassky [20] showed that the penalty factor and kernel function parameters are the key to determine the performance of SVM. Therefore, the recognition effect of the SVM model in this article largely depends on the selection of the model parameters C and g , and there is no unified standard or theory to guide the optimization of the SVM parameters.

The classifier performance is first validated by K-Cross Validation (K-CV). The general steps are as follows: First, the original data are equally divided into K disjoint subsets, then each subset data are used as a verification set, and the remaining K-1 subset data are used as a training set to obtain K models. Finally, the average classification accuracy rate of the final verification set of the K models is used as the performance index of the K-CV classifier. It effectively avoids over-learning and under-learning and the final results obtained are also more convincing.

In current research, Genetic Algorithm (GA) [21,22,23] and the emerging Grey Wolf Optimization (GWO) [24,25,26] are widely used in optimizing SVM parameters. In the article, Particle Swarm Optimization (PSO) [27,28,29] is used to improve the results of SVM parameter optimization, which makes the results more convincing.

4.1 GA-based SVM parameter optimization

GA, also known as evolutionary optimization algorithm, simulates the evolutionary mechanism of organisms and the natural law of survival of the fittest, through its selection operator, crossover operator, and mutation operator, and it continuously evolves the original population to obtain the optimal solution of the optimization problem. It is a heuristic global search algorithm with wide application, high efficiency and strong robustness. The process of parameter optimization for SVM classifier with the help of GA is shown in Figure 5.

Figure 5

GA optimization SVM parameter flow chart.

Selection operator, crossover operator, and mutation operator are used in three basic links of GA, respectively: heredity, crossover, and mutation. The selection operator is used to select individuals from the population, and the crossover operator is used to cross two individuals to generate a new individual. Mutation operators are used to mutate individuals to increase the diversity of the population. Selection operators, crossover operators, and mutation operators in GA are an organic whole, and their rational use is the key to achieve GA.

According to the parameter optimization idea of GA, the model parameters of SVM are optimized and selected. Here 285 samples are used to train the SVM model. The specific parameters of the model are set as follows: the maximum evolutionary generation is set to 200, the population size is 20, the penalty factor C ∈ [ 0 , 100 ] , the kernel function parameter g ∈ [ 0 , 100 ] , the crossover probability is 0.7, the mutation probability is 0.01,the accuracy of five-fold cross validation is selected as the fitness function, and the maximum evolutionary algebra is selected as the iteration end condition of the algorithm. The results of ten runs of GA optimizing the parameters of the SVM model are recorded in Table 2.

Table 2

GA optimization results for SVM parameters

Number of experiments	Best c	Best g	CV accuracy (%)	Average running time (s)
1	30.0728	6.1778	95.3333	18.78337
2	51.6945	9.3692	94.0862
3	21.0836	28.8268	95.3333
4	22.9131	8.3937	95.3333
5	35.0282	11.6646	94.0862
6	28.7776	27.7609	95.3333
7	41.9443	1.0428	96.6274
8	34.0490	12.2337	93.8265
9	60.0082	0.8847	93.8265
10	25.5672	29.2689	95.3333

4.2 PSO-based SVM parameter optimization

PSO is an optimization based on swarm intelligence. Similar to GA, it is an evolutionary algorithm, which realizes the search for the optimal solution in a complex space through competition and cooperation between individuals. The flowchart of PSO optimizing SVM parameters is shown in Figure 6.

Figure 6

PSO optimization SVM parameter flow chart.

According to the parameter optimization idea of PSO, the model parameters of SVM are optimized and selected. Here 285 samples are used to train the SVM model. The specific parameters of the model are set as follows: the maximum evolutionary generation is set to 200, the population size is 20, the penalty factor C ∈ [ 0 , 100 ] , the kernel function parameter g ∈ [ 0.01 , 1000 ] , the inertia weight w = 0.6 , the learning factors c 1 = 1.5 , c 2 = 1.7 , the accuracy of five-fold CV is selected as the fitness function, and the maximum evolutionary algebra is selected as the iteration end condition of the algorithm. The results of ten runs of PSO optimizing the parameters of the SVM model are recorded in Table 3.

Table 3

PSO optimization results for SVM parameters

Number of experiments	Best c	Best g	CV accuracy (%)	Average running time (s)
1	29.7124	12.6681	95.3333	39.30805
2	40.5832	28.2934	95.3333
3	22.3836	27.9543	94.6481
4	52.2047	9.7179	96.0536
5	15.9174	14.8806	95.3333
6	83.1571	6.1733	93.8629
7	21.4805	15.4898	95.3333
8	31.8098	23.1130	95.3333
9	26.9917	8.2189	96.2427
10	17.9965	10.2794	96.2427

4.3 GWO-based SVM parameter optimization

GWO is a group intelligence optimization algorithm proposed by Mirjalili et al. to simulate the unique hierarchical system and collective hunting behavior of gray wolf populations in nature. Compared with the early intelligent optimization algorithms, it has extremely significant advantages in terms of convergence, and at the same time, the algorithm has lower complexity and fewer control parameters, and takes into account the ability of individual and global optimal solutions. To a certain extent, it can avoid the problem of falling into local optimum in the optimization process, so it is also widely used in various practical optimization problems.

In the gray wolf group, there are mainly four levels: α , β , δ , and ω . Among them, wolf α is the optimal solution, wolf β is the second optimal solution, wolf δ is the third optimal solution, and Wolf ω is a candidate solution in the population. The detailed hunting and solving process of gray wolf can be found in the definition of GWO [24,25,26]. The flowchart of GWO optimizing SVM parameters is shown in Figure 7.

Figure 7

GWO optimization SVM parameter flow chart.

According to the parameter optimization idea of GWO, the model parameters of SVM are optimized and selected. Here, 285 samples are used to train the SVM model. The specific parameters of the model are set as follows: the maximum evolutionary generation is set to 200, the population size is 20, the penalty factor C ∈ [ 0 , 100 ] , the accuracy of five-fold cross validation is selected as the fitness function, and the maximum evolutionary algebra is selected as the iteration end condition of the algorithm. The results of ten runs of GWO optimizing the parameters of the SVM model are recorded in Table 4.

Table 4

GA optimization results for SVM parameters

Number of experiments	Best c	Best g	CV accuracy (%)	Average running time (s)
1	97.5852	0.0460	98.6094	7.34617
2	100	0.0460	98.6094
3	100	0.0460	98.6094
4	97.8856	0.0460	98.6094
5	98.2243	0.0460	98.6094
6	100	0.0460	98.6094
7	100	0.0460	98.6094
8	100	0.0460	98.6094
9	96.1015	0.0460	98.6094
10	100	0.0460	98.6094

After the calculated comparison of the three algorithms, it is found that the SVM model based on GWO has the highest accuracy and the shortest running time; however, the penalty factor C of the model is always close to the maximum value of the boundary, which indicates that the generalization ability of the model is poor. For most paper-plastic composite bag surface defects, the accuracy of the SVM model based on GA is the same as that of the SVM model based on PSO. But the running time of the model based on GA is nearly 1/2 shorter than that of the model based on PSO.

In order to finally explore the influence of the three optimization algorithms on the classification performance of the SVM model of the paper-plastic composite bag surface defects, the SVM models established based on GA, PSO, and GWO were used to classify the test set data, respectively. The training set containing 285 samples was used to train the SVM model (c = 2, g = 1), and the test set containing 95 samples was used for model testing. The classification results of the best parameter training classifier obtained by using GA is shown in Figure 8, respectively. The abscissas 1–95 in the figure are the sample numbers of the test set. The abscissas 1–95 in the figure are the sample numbers of the test set, and the ordinates “1,” “2,” “3,” “4,” and “5” numbers represent wrinkles, broken skins, holes, dirty spots, and convex ribs, respectively. For the ease of comparison, the classification results of the three different optimization algorithms are summarized in Table 5.

Figure 8

Test set classification results based on GA.

Table 5

Comparison of classification results of different optimization algorithms

Optimization method	Best c	Best g	Test set accuracy
GA	22.9131	8.3937	95.7894% (91/95)
PSO	26.9917	8.2189	94.7368% (90/95)
GWO	100	0.0460	92.6316% (88/95)

From the above chart and figures, it can be seen that the SVM model based on GWO for the surface defects of paper-plastic composite bags has a poor performance on the test samples under the condition of high model accuracy, it may be that the defect feature extraction is not perfect, resulting in the poor generalization ability of the model. Considering the huge advantages of GA in the process of parameter optimization, time-consuming, and generalization ability, for the detection of surface defects of paper-plastic composite bags which require strict algorithm robustness and running time, it might be more suitable to adopt GA as the parameter optimization algorithm of SVM, where the recognition accuracy and speed, and the most ideal test results in a shorter time can be obtained.

5 Experimental test

The paper-plastic composite bag surface defects detection platform is mainly composed of industrial cameras, industrial lenses, light sources, motors, etc. The schematic diagram of the detection platform structure is shown in Figure 9. The motor controls the conveyor belt to transport the uncut paper-plastic composite bag to the detection area, the encoder measures the position of the paper-plastic composite bag, and determines whether the paper-plastic composite bag reaches the detection area. When it is determined that the paper-plastic composite bag reaches the detection area, the camera is triggered to obtain the surface image of the paper-plastic composite bag and is transmitted to the upper computer to complete the detection of the surface image of the paper-plastic composite bag.

Figure 9

Schematic diagram of the surface defect detection platform for paper-plastic composite bags.

In order to verify the theory and platform feature requirements in the article, a paper-plastic composite bag defect detection platform is designed and built as shown in Figure 10. The experimental equipment is shown in Table 6.

Figure 10

Experimental platform for defect detection of paper-plastic composite bags.

6 Conclusion and prospects

This work studies the segmentation, screening, feature selection, and SVM model optimization of surface defect images of paper-plastic composite bags. The main conclusions are as follows:

1. The segmentation algorithm combining edge detection and adaptive region growth is used to realize the accurate classification of various typical defect images on the surface of paper plastic composite bags, which improves the problem that a single segmentation method can not achieve the ideal segmentation effect for various surface defect images of paper plastic composite bags, and has practical significance.

2. The research compares the optimization effects of the SVM models established by GA, PSO, and GWO, respectively. The test finds that although GWO has the highest model accuracy and the shortest running time, its generalization ability is poor. PSO runs for a long time. GA can take into account the recognition speed and accuracy, so it is more suitable for the optimization of the SVM model.

3. This study established a paper-plastic composite bag defect detection platform, and experiments are carried out on the theory proposed in this work. The experimental results show that the overall defect detection and classification accuracy rate reaches 96.83%, which meets the actual production requirements and the algorithm proposed in this study has a good performance in the evaluation indicators of various defects, the Dice coefficients are all above 0.93, which is superior to other comparison algorithms.

4. A sample screening experiment was carried out by using one fast image screening, and processing method based on the gradient projection difference was adopted for paper-plastic composite surface defects. Experimental results show that the detection and classification accuracy rates without the screening method or with the screening method are the same, but the detection speed with the screening method is 3.75 times faster than that without the screening method, it verifies the reliability of the theory about the screening method in this work.