Real-time image defect detection system of cloth digital printing machine

2.1.1 Common printing fabric defects

In the process of digital printing production, if the jet printing ink is not filtered thoroughly and contains small particles or motor operation deviation, the nozzle will be blocked; Ink leakage and uneven ink output of the nozzle due to mechanical failure of the nozzle; due to the uneven cloth pressing by the roller, there are problems such as cloth wrinkles. The above problems can lead to the surface defects of printing products, such as pass channel, uneven ink output, ink leakage of nozzle, cloth wrinkles and so on. Affect the value of the final product [12,13]. Due to the blockage of the nozzle, uneven ink output of the nozzle and the deviation of the stepping motor, there are defects such as white silk, spots and folds in the fabric. See Table 1 for common defect types.

2.1.2 System hardware design

The hardware part is mainly composed of CCD camera, light source and image processing equipment. Among them, the CCD camera adopts MVGED501C-T color area array camera of a company, the camera resolution is 1,920 dpi × 1,080 dpi, and the pixel size is 3.45 μm × 3.45 μm; the lighting system adopts LED light source; the lens adopts MV-LD-25-5M-K lens [14]. The schematic diagram of visual platform control is shown in Figure 1.

Figure 1

Schematic diagram of vision platform.

The working principle of the detection system is: when the roller turns a certain angle, the encoder collects the image from the industrial camera; then the camera converts the collected image into digital image signal and sends it to the upper computer to analyze and process the image to obtain the defect information; finally, the detection system sends instructions to the controller response module according to the defect information to control the roller and inkjet printer [15].

2.2.1 Image preprocessing

As the collected printed fabric image is affected by equipment operation jitter and insufficient illumination in the factory, the image brightness is low, including noise and blurred details, which affects the subsequent operation. Therefore, the collected images need to be preprocessed. The main processing methods are image graying, image contrast enhancement and image sharpening.

The SURF algorithm is an improved algorithm based on SIFT. When extracting image feature points for feature matching, it has a certain degree of stability for images with rotation, scale transformation, and brightness interference. The core of the algorithm is to use the Hessian matrix to obtain the local maximum value of the image and refer to the concept of integral graph, which reduces the complexity of the calculation process and is one of the most commonly used methods for image feature matching.

Let I(x, y) be the position of a pixel in the image, σ is the scale of the Hessian matrix, and the Hessian matrix at I(x, y) is defined:

L _xx , L _xy , L _yy are the second-order partial derivatives at the pixel point I(x, y) after convolution by the second-order Gaussian filter function, so that the discriminant of each pixel point Hessian can be obtained.

According to the sign of det(H), each pixel of the image can be classified to determine whether the point is a candidate feature point.

Since the color area array camera used cannot directly obtain the gray image, and the color image is not suitable for direct and rapid detection, it is necessary to gray the color image. The steps are as follows [18]:

1. Calculate the RGB component of each pixel of the image;

2. Calculate weighted gray value 0.3 × B + 0.59 × G + 0.11 × R;

3. Assign the calculated gray value to each corresponding pixel.

Because the collected image has the problem of low brightness, it is necessary to adjust the image contrast to enhance the image. Adjusting image contrast belongs to pixel transformation point operation, and the formula is the following formula (5):

where g(i, j) is the adjusted pixel; f(i, j) is the pixel before adjustment; α is the gain coefficient, α > 0; β is the gain variable.

After the image is grayed and the contrast is adjusted, the image has been significantly improved, but there are some problems such as blurred details, so the image sharpening operation is needed. Image sharpening operations include gradient sharpening and Laplace sharpening. The gradient sharpening effect is good, but the noise resistance is poor. The Laplace method has simple calculation and good real-time performance. Therefore, the research group adopts Laplace operator to realize image sharpening. The main idea of the sharpening operator is to determine the value of the pixels in the image after traversing the Laplace 4.

2.2.2 Image registration

Feature point matching. Due to the placement of the camera and the vibration of the printing equipment, the image to be tested needs to be affine transformed to register the template image for the next defect detection operation. The author uses the improved surf algorithm to match the feature points, and combines the two-way uniqueness matching method with the single matching to eliminate the mismatched points; Then the least square method is used to fit the parameters of affine transformation for image registration. Surf algorithm uses the approximate image of Hessian matrix to construct the Hessian matrix of each pixel in the image. When the scale is σ, the Hessian matrix of pixel X = (x, y) is as follows (6):

where L _xx is the second derivative of filtered image g(σ) in x direction; L _xy is the second derivative of filtered image g(σ) in x and y directions; L _yy is the second derivative of filtered image g(σ) in y direction.

Compared with SIFT algorithm, surf algorithm introduces the concept of integral image and uses box filtering with changing size to replace image pyramid [19]. The response image of Hessian matrix is obtained by using box filtering and integral image of each size. In order to select the feature points in the image, the original image needs to be transformed, that is, the approximate value of Hessian matrix of each pixel in the original image is used to form the transformation image. The weight is introduced to reduce the deviation caused by approximation, and the discriminant is the following formula (7):

Judge whether the point is a point of interest according to the positive and negative results of Eq. (7). The size of the interest point is compared with the 26 points in its three-dimensional field in the neighborhood of different scale space. If the interest point is the point with the largest eigenvalue in the neighborhood, it is recognized as the feature point in the region.

Take the feature point determined in the previous step as the center, select the area with radius of 6 S (s is the scale value of the feature point), count the Haar wavelet responses of all points in the 60° sector in the area, and give weights according to their responses. The direction of the feature point takes the direction of the longest vector in the area. Take the square box of 20S * 20S near the feature point. The box direction is the main direction of the feature point. Divide the box into 4 * 4 sub regions. Each sub region counts the harr wavelet features of 5 * 5 pixels. In this way, each sub region has a vector with four-dimensional components. The formula is as follows (8):

where d _x , d _y , |d _x |, |d _y | represent the sum of horizontal direction and vertical direction and the absolute sum of horizontal direction and vertical direction respectively.

So far, each feature point has a 16 * 4 = 64 -dimensional description operator. The Euclidean distance between feature points is calculated to judge whether the matching is correct. The smaller the distance, the higher the similarity. When the Euclidean distance is less than the set threshold, it can be determined that the matching is successful [20].

Affine transformation. Although the feature points in the template image and defect image are matched, in practice, the two collected images cannot be fully registered. The transformation parameters need to be determined by matching point pairs to obtain two fully registered images. The schematic diagram of affine transformation is shown in Figure 3.

Figure 3

Schematic diagram of affine transformation.

The affine formula is as follows (9):

where (m, n) and (m′, n′) are the coordinates of pixels before and after image transformation; d _m and d _n are the translation amounts; a, b, c, d are the rotation and stretching parameters.

The matched feature point pair is obtained by SURF algorithm, and the least square method is selected to fit the matching points to obtain the above parameters. After determining the parameters, transform according to Eq. (9), and finally make the size, position and angle of template image and defect image completely consistent.

2.2.3 Defect detection

After two fully registered images are obtained, the image difference method is selected to obtain the defect location. The basic idea is to differentiate the two images, weaken the similar part of the image and highlight the changing part [21]. The absolute value of the difference between the gray values of pixels in the same coordinate is taken as the new gray value of the difference image. After the differential operation of the image, the gray value of the defect position will be relatively high, so it will appear white; for the defect free position, because the pixel values of the two images are the same, the gray value is zero and appears as black. Because the acquired image is often accompanied by noise and affected by affine parameters during image registration, the difference image often has point or linear “false defects.” After obtaining the above differential image with “false defect,” carry out image binarization operation and morphological processing to eliminate “false defect” and prominent defect. Morphological processing is used to operate the image information that needs to be highlighted or removed in binary images, including corrosion, expansion, open operation and close operation. Corrosion reduces the image; expansion causes the image to expand outward. The open operation is often used to eliminate the impurity and burr in the binary image. Closed operation is usually used to supplement the blank points in the image area.