MSVM Recognition Model for Dynamic Process Abnormal Pattern Based on Multi-Kernel Functions

2.1 Quality abnormal pattern

With the modern industries and service organizations developed, there exist a large number of dynamic data from the automatic manufacturing or continual production process. These data describes the status of quality in a current operating or processing. The study of quality abnormal pattern recognition for dynamic process is almost specific to diagnose several particular industrial processes and recognize control chart abnormal patterns[7-8]. It is believed that the dynamic process fluctuation involves normal and abnormal changes[12]. The normal changes illustrate the process running in its intended mode, while the abnormal changes will lead to the various quality problems of dynamic process. Both of them are called quality patterns, which can exhibit six common types of pattern: normal (NR), cycle (CC), increasing trend (IT), decreasing trend (DT), upward shift (US), and downward shift (DS), as shown in Fig.1.

Figure 1

Quality abnormal patterns

When the dynamic data fluctuates around the target value at random, the current process is under a quality normal pattern, which is shown as Figure 1(a). When the dynamic data appear a trend, shift or cycle, the process fault or product quality problem will be occurred, which can be called quality abnormal patterns. The trend includes two types of pattern, such as increasing trend and decreasing trend seen in Fig.1(b) and (d) respectively. The shift pattern contains upward shift shown as Fig.1(c) and downward shift shown as Fig.1(e). Therefore, in order to detect and diagnose the quality problems in productive process, it is necessary to recognize effectively the quality abnormal patterns of dynamic data.

2.2 MSVM recognition framework

A SVM performs two-class classification tasks by constructing optimal separating hyper planes. For six quality patterns, multi-SVM should be introduced to solve the multi-class classification problems. There are two widely used methods to extend binary SVM to multi-class classification. One is called the “one-against-all” method, and another is called “one-against-one” method[13]. Both of them will be used in the following recognition framework so as to achieve the high efficiency.

In order to recognize quality abnormal patterns effectively, a recognition framework based on MSVM will be set up. First, by means of the “one-against-all” method, a SVM₄ classifier for normal pattern and SVM₁, SVM₂, SVM₃ classifiers for the trend, shift and cycle three quality abnormal patterns are proposed respectively. Then, the SVM₅ and SVM₆ classifiers are established respectively to further recognize the increasing trend, decreasing trend, and upward shift, downward shift with “one-against-one” method. A MSVM framework for the recognition of quality abnormal patterns is shown as Figure 2. According to above framework, three steps will be done for the recognition of quality abnormal patterns of dynamic data as the following:

Figure 2

MSVM recognition framework

1. SVM₁, SVM₂, SVM₃ and SVM₄ classifiers are set up respectively to identify the trend, shift, cycle three abnormal patterns and a normal pattern of dynamic data.

2. After the SVM₁ classifier identifies current data as the trend abnormal pattern, a SVM₅ classifier will be applied to further distinguish the increasing trend and decreasing trend of dynamic data.

3. Similarly, a SVM₆ classifier will be used to further distinguish the upward shift and downward shift of dynamic data after the SVM₂ classifier identifies current data as the shift abnormal pattern.

2.3 Kernel function selection of MSVM

The principle of SVM performs classification tasks by constructing optimal separating hyperplane. In the linearly separable condition, suppose that (x_i, y_i), i = 1,2,··· , n, x ∈ R^d, is a training sample set of dynamic data, where y_i ∈ (1, –1) represented the category of a training sample. The purpose of learning and training for dynamic data sample is to find an optimal hyperplane which can classify the two types of abnormal patterns correctly. Thus, constructing this hyperplane can be represented the following optimization problem:

The above optimal hyperplane problem can be transformed to its dual problem by uses of Lagrnage multipliers. The optimal decision function is given by

where the ai∗ are optimal Lagrange multipliers, and b^∗ are classification thresholds, which the training sample corresponding to ai∗≠0 is a support vector. As the sample data from quality abnormal patterns in a dynamic process are all linearly inseparable, SVM can efficiently perform non-linear classification using kernel function, which can map these linearly inseparable sample data into high-dimensional feature spaces so as to solve linearly inseparable problems. Therefore, the problem of finding an optimal hyperplane can be transformed to another optimal problem as follows:

where ζ_i is a slack variable, ζ_i > 0, and c is a penalty parameter. The greater c is, the heavier punishment for misclassification will be. The classify function of SVM can be obtained by adding the kernel function k(x_i, x):

According to the above analysis, it should be chosen different kernel functions for different SVM classifiers. The recognition efficiency of a SVM model mainly depends on the selection of the kernel functions. It is essential to select a suitable kernel function for improving the recognition accuracy of a SVM classifier. Under the above MSVM recognition framework, it is crucial to select the kernel functions of MSVM so as to improve the recognition accuracy of quality abnormal patterns in a dynamic process. At present, the general requirements for selecting the kernel functions have not seen. Usually, the kernel functions should be chosen by the experience and simulation experiments[14]. There are three common kernel functions of SVM as shown as Table 1[15].

As the sample data from quality abnormal patterns in a dynamic process are linearly inseparable and their fluctuations are different, in order to choose some suitable kernel functions for constructing the MSVM recognition model, in this paper, the recognition efficiency of the above three different kernel functions for different quality patterns are compared by the following simulation experiment.

3.1 Generation of experiment data

The sample data of simulating dynamic process fluctuations, x(t), are generated by Monte-Carlo method, as shown as the formula (5):

where μ represents the target value of a dynamic process. To simplify the simulation data, suppose μ is zero. r(t) denotes the variations of random factors in a dynamic process, r(t) ∼ N(0,1), which represents the white noises in our experiment, and d(t) shows the abnormal fluctuations. When x(t) = r(t), the simulation data represents the quality normal pattern in a dynamic process. The five quality abnormal patterns including increasing trend, decreasing trend, upward shift, downward shift and cycle can be simulated by the formula (6) ∼ (10). (1) A increasing trend pattern can be simulated by:

(2) A decreasing trend pattern can be denoted by:

where k is the gradient of increasing or decreasing trend, and it’s values are taken from 0.2 to 0.5.

(3) The simulation data of an upward shift pattern are given by:

(4) A downward shift pattern can be described by:

where b is the position of an upward or a downward shift, b = (t – t₀), and t₀ denotes the time of an upward or a downward shift. Before the shift, b = 0, and after the shift, b = 1. s is the altitude of a shift, which is supposed in the range of 0.8 ∼ 2.

(5) The sample data of cycle pattern can be obtained by:

where T is a cycle period, and suppose it is in the range of 15 ∼ 30. A is a cycle amplitude, which is taken from 0.8 to 2.

By uses of the above formulas, the sample data of a normal pattern and five quality abnormal patterns in a dynamic process at time t, x(t), are generated respectively, which is supposed in the range of 0 ∼ 59. 60 groups of the data for each quality pattern are sampled to compare the recognition accuracies of MSVM with different kernel function. The former 20 groups are taken as training samples, and the other 40 groups as testing samples.

3.2 Analysis of experiment result

In order to compare the recognition accuracies of multi-SVM with different kernel functions in the MSVM framework, the value of a penalty factor should be set. The parameter c is known as the penalty factor of a SVM identifier. In the paper, the value of c is taken as 1.2 for the simulation experiment of SVM with linear and poly kernel functions by means of the Cherkassky’s empirical formula (11). For the SVM with RBF kernel function, the grid search method can be introduced to optimize its parameters.

where x̅ and σ_x are the mean and standard deviation of the training samples from a dynamic process respectively.

Based on the sample data generated by above Monte Carlo method, the recognition accuracies of SVM identifiers with linear, polynomial and RBF functions are tested by the simulation experiment. For the 60 groups of the data sampled by six quality patterns respectively, 20 groups of data will be merged into 120 groups as the training samples of SVM₁, SVM₂, SVM₃ and SVM₄, while 40 groups of the data can be combined as the 240 groups of testing samples of them. Each 20 groups of data from the increasing trend and decreasing trend pattern will be merged into 40 groups as the training samples of SVM₅. So do 80 groups as testing samples of SVM₅. Similarly, the training and testing samples of SVM₆ for shift pattern can be obtained respectively. The recognition accuracies (RA) of six SVMs with different kernel functions are compared by the simulation experiment, as shown as Table 2.

The experiment results indicate that the recognition accuracies of the MSVM classifiers with the different kernel functions are quite different. The RBF function has the higher accuracies for all the quality abnormal patterns than the others, which the average of RA has reached to 98.4%. The recognition accuracies of SVM₅ and SVM₆ with linear and polynomial functions for trend and shift patterns have achieved the highest accuracies 100%.