Target recognition and detection system based on sensor and nonlinear machine vision fusion

3.1 Image acquisition based on machine vision

Machine vision is widely used in different fields of complex technologies. In simple applications (e.g., using a backlight to detect the profile of opaque objects), the imaging system can operate reliably with loose system parameters. However, in more demanding applications (such as detecting mirror surface defects), the main parameters must be set within a smaller tolerance range or use more expensive high-precision hardware for the imaging system to operate stably.

The number of major parameters involved in the imaging system increases with the complexity of imaging applications. Edmund Optics Tokyo Imaging Laboratory conducted research on this and proposed up to 20 precise adjustable parameters.

Image acquisition is the key link for automatic recognition of photoelectric sensor weld seam: only by acquiring a high-definition photoelectric sensor weld seam image can the target area of the image be detected more effectively and accurately, so as to improve the recognition accuracy of the photoelectric sensor weld seam. Machine vision technology is an image acquisition technology, which is simple in operation and of high quality. Therefore, the author uses machine vision technology to collect the weld image of photoelectric sensor, laying a foundation for the subsequent automatic recognition of photoelectric sensor weld [7]. The noise during the welding process, as well as the transmission and conversion of images, can affect the quality of the image. Therefore, in general, the image is first preprocessed by using certain filtering and image restoration techniques to denoise and repair the obtained image, thereby reducing the impact of various interference signals and enhancing the strength of the required feature signals. The commonly used preprocessing methods can generally be divided into two types. One is the frequency domain processing method, which involves first performing Fourier transform on image pixels, and then performing homomorphic filtering, frequency domain high (low) pass filtering, and other methods on the transformed complex pixel geometry. Another method is spatial domain processing, which generally uses methods such as neighborhood averaging, median filtering, and histogram correction to directly process image pixels. The high computational complexity of frequency domain processing methods results in poor real-time performance, so they are rarely used in machine vision systems with real-time requirements. Due to the advantage of reducing noise while not blurring edges, the median filtering method has been widely used among all existing spatial domain processing methods. Its principle is to replace the target pixel’s grayscale value with the median of neighboring grayscale values.

When collecting the weld image of the photoelectric sensor to be recognized by the camera charge coupled device (CCD), the light source should be considered to make the lighting effect in the best state so as to obtain a clear image. Therefore, according to the size and working requirements of the photoelectric sensor weld, the author combined the camera CCD and selected the XF-5MDT05X65 lens to collect the photoelectric sensor weld image to be recognized, in order to ensure that the gray value of the photoelectric sensor weld image is uniform [8].

The gray pixel value of photoelectric sensor weld visual distribution is set to d ( x , y ) , and the constructed photoelectric sensor weld image acquisition model is as follows:

where E is the image fractal coefficient and x i represents the pixel value.

3.2 Target area detection based on Gaussian model

In order to automatically identify the weld seam of photoelectric sensor, it is first necessary to detect the weld seam image. The common detection method is the Gaussian model, which is generally composed of two to six single Gaussian models. If the number of models is L, the weight is y, and the distribution is h, the expression of Gaussian model is as follows:

where Q represents the target data density, V represents the average value of Gaussian model, U represents threshold, ζ represents the variance value of Gaussian model, and x represents the target data.

According to the characteristics of Gaussian model, the target area detection of photoelectric sensor weld image is divided into the following steps:

1. Calculate the mean difference value of adjacent models and compare it with the threshold value. If the mean difference is less than the threshold value, use the following formula (3) to update the model, where λ is the parameter learning rate.

   (3) y = ( y L , y L + 1 ) max v = ( 1 − λ ) v L + 1 + v · λ ζ 2 = ( 1 − λ ) ζ L + 1 2 + ζ · λ .

2. The environment for automatic recognition of welds by photoelectric sensors is complex; in order to improve the recognition efficiency, the noise recognition factor is introduced and expressed by formula (4), where O represents the noise recognition factor, m is a function, and g is a model function.

   (4) mg ∏ L = 1 L y L L − O = m ( ( y , L ) / L ) .

3. After the noise recognition factor is added, use the following formula (5) to update the algorithm, discard the model with weight less than zero, and repeat the operation to suppress the impact of noise on the recognition results.

   (5) mg ∏ L = 1 L y L O + mg ∏ L − 1 L y L − O = m ( y , L ) μ ∑ L = 1 L y L − 1 + m σ / σ y L = 0 y L = y L + ∂ ( O L − y L ) − ∂ O O ,

   where μ represents the mean value of Gaussian model, σ represents the reference value, and ∂ represents the number of samples.

4. With the target area as the reference, select the area with the average variance strength greater than the assumed threshold as the key area, and use the following formula (6) to complete the detection of the target area of the weld image, where D is the target image in the key area and Q is the characteristic of the key area.

3.3 Feature extraction

The photoelectric sensor weld seam has characteristics such as instability, which requires feature extraction to reduce the data density; the specific process is as follows:

1. Using the Hessian matrix model J for feature point detection, the Hessian model can be described by the following formula (7), where x and y represent any point in the matrix, ζ represents the matrix scale distance, and M represents the Hessian derivative.

   (7) M xx ( x , ζ ) M xy ( x , ζ ) M xy ( x , ζ ) M xy ( x , ζ ) = J ( x , ζ ) .

2. In order to ensure the stability of the photoelectric sensor weld feature, it is necessary to determine a feature point that can be reused. First, take the previous feature point as the center of the circle, take ζ as the diameter, divide the feature area, calculate the conversion rate of x and y, and then predict the direction of the feature point [9,10].

3. The gradient direction is corrected by the direction of the feature point, and the gradient direction is calculated according to the following formula (8), where ϑ represents the direction of the feature point, Y stands for normalized feature vector, V represents vector dimension, and λ represents the ontology vector coefficient.

   (8) ϑ ( x Y ) − ϑ = ϑ ( x L ) Y = Y / | Y | y = ∑ L = 1 M ( y L − v ) · Y L / λ L · M L .

4. In order to reduce the operation time, the principal component analysis method is used to reduce the dimension of the target vector; first, use the following formula (9) to calculate the covariance of all sets of eigenvectors, and set D as the matrix model and U as the matrix dimension.

3.4 Automatic recognition model of quantum gate nonlinear neural network

Quantum is a representative unit of calculation; there can be two quantum states at the same time, so the automatic recognition model of quantum gate nonlinear neural network has excellent distinguishing function, strong accuracy, and obvious effect on automatic recognition. When the quantum gate nonlinear neural network is used to automatically identify the photoelectric sensor weld, the flow can be seen as shown in Figure 1, and the specific steps are as follows:

1. Input the pretreated photoelectric sensor weld characteristic data into the quantum gate neural network model to calculate the quantum state S, where σ is the probability amplitude and i is the point where the pixel is located, as shown in formula (10).

   (10) S ( σ ) = cos σ − sin σ sin σ cos σ S = sin σ i | 1 + cos σ i | 0 > .

2. Calculate the network error, delete the quantum that produces the network error, recalculate, and get the back propagation error information [11].

3. According to the gradient reduction rule, use the following formula (11) to update the values of ϑ and y. In formula (11), A represents the hidden range, B represents the probability amplitude of the model output, ϑ stands for hidden matrix, γ represents the model output matrix, and N represents the number of hidden matrices [12].

   (11) arcsin ∏ i = 1 n sin ( σ i + ϑ i ) = A arcsin ∏ i = 1 n in ( σ i + γ i ) s = B .

4. According to the optimal iterative method, output the data with the highest accuracy, and combine the gradient reduction algorithm to obtain the calculation results in the quantum neural network model, and then complete the automatic identification of the photoelectric sensor weld. The gradient reduction formula is as follows (12):

Figure 1

Flow of quantum gate nonlinear neural network model.

4.1 F1 value

F1 value is an indicator used to measure the performance of the automatic recognition method, which is very representative in statistics; its calculation formula is as follows:

where J z represents the recall rate, A c stands for accuracy, TP represents the correctly identified normal photoelectric sensor weld results, TN represents the correctly identified abnormal photoelectric sensor weld results, FP represents the incorrectly identified results, and FN represents the incorrectly identified abnormal results [16].

The judgment of the advantages and disadvantages of the photoelectric sensor weld seam recognition mainly depends on the identification accuracy, recall rate, and F1 value; the accuracy, recall rate, and F1 value of the proposed method, membrane wall recognition method, combined high- and low-frequency recognition method, and feature extraction recognition method are calculated, respectively. The recognition rate of different methods is shown in Table 2, the recall rate test result is shown in Figure 2, and the F1 value test result is shown in Figure 3 [17].

Figure 2

Recall rate test results.

Figure 3

F1 value test results.

According to the data in Table 2, with the increase of the number of experiments, the recognition rate of the proposed method is higher than that of the membrane wall method and the feature extraction recognition method; the recognition rate of the proposed method is relatively stable, without too much fluctuation, whereas the recognition rates of the membrane wall method and the feature extraction recognition method fluctuate greatly. With the increase in the number of experiments, the experimental results show that the proposed method has a high recognition rate for the automatic recognition of the photoelectric sensor weld [18].

According to Table 2, Figures 2 and 3, the author’s method has the highest value among the three test indicators, with the highest accuracy of 97%, the highest recall of 98%, and the highest F1 value of 94. The test index values of the author’s method are higher than those of the three methods. The reason is that the author uses Hessian matrix to extract the feature points of the photoelectric sensor weld target area, and then inputs the extracted feature points into the quantum gate neural network model to accurately obtain the automatic recognition results of the photoelectric sensor weld.

4.2 Floating-point operands

Floating-point operands are used to measure the complexity of the weld recognition method, F is used to represent. The smaller the F value is, the smaller the recognition complexity is. The larger the F value, the greater the recognition complexity. F can be calculated by the following formula (14):

In formula (14), I represents the height of the input data matrix, X represents the width of the input data matrix, D represents the channel coefficient of the data matrix, L represents matrix size, J represents the number of dimensions of the input model, and P represents the number of dimensions of the output matrix.

The test results of floating-point operands of the proposed method, combined high-low frequency recognition method, and feature extraction recognition method are shown in Figure 4.

Figure 4

Floating-point operation values of different methods.

The analysis of Figure 4 shows that the F values of the proposed method, combined high- and low-frequency recognition, and feature extraction recognition methods are different. No matter which data set, the F value of the proposed method is higher than that of the joint high- and low-frequency recognition and feature extraction recognition methods. Moreover, with the change of the quantity set, the F value of the proposed method does not change much, while the F value of the combined high- and low-frequency recognition and feature extraction recognition methods vary greatly, indicating that the recognition complexity of the proposed method is low.

4.3 Time side test

The proposed method, membrane wall recognition method, and feature extraction recognition method are used to compare and analyze the automatic recognition time of photoelectric sensor welds; the comparison results are shown in Figure 5 [19,20].

Figure 5

Weld seam identification time of photoelectric sensor in three methods.

According to Figure 5, the time consumed by the author’s method for automatic identification of photoelectric sensor welds is within 6 s, and the time consumed by film wall identification method for automatic identification of photoelectric sensor welds is within 20 s, and the time consumed by feature extraction and recognition method for automatic recognition of photoelectric sensor weld seam is within 22 s; the time consumed by the author’s method for automatic recognition of photoelectric sensor weld seam is the shortest and the recognition efficiency is the highest [21,22]. In traditional machine vision, vision sensors generally only collect and analyze two-dimensional images, cannot conduct multi-scale analysis of target objects, and are affected by environmental factors such as lighting, which limits their application. Aiming at the shortcomings of traditional machine vision, this article introduces nonlinear features and deep neural networks on the basis of traditional machine vision to realize multi-scale analysis and recognition of target objects. In order to extract the feature information of the target object effectively, the original image is filtered by Gaussian filter, and the edge information of the image is obtained by edge detection. Then the histogram statistical method is used to analyze the edge information. On this basis, nonlinear features are used for multi-scale analysis and identification.