Face Recognition in Complex Unconstrained Environment with An Enhanced WWN Algorithm

2.1 The WWN-7 model

Where-What Networks model [14] is an embodiment of the developmental network model [31, 34], whose structure is shown in Figure 1. It is composed of three areas: X area, Y area and Z area.

Figure 1

Structure of the WWN-7 Model.

As the sensor, X area is the retina and the perception part of the agent, responsible for external information input. The entire Y area, as the “brain” of the agent, is enclosed in the skull. Y area is not under the direct supervision of the outside world and it realizes the information exchange between X area and Z area, just like a “bridge”. It is divided into four parts: Y₁, Y₂, Y₃ and Y₄, with different neuron numbers. The preprocessing area is responsible for calculating the pre-action energy of neurons in the learning process to determine which neuron can fire (be activated). Z area is the motor terminal of the agent and it is divided into three parts: LM (location-motor), SM (size-motor) and TM (type-motor). LM and TM simulate the dorsal pathway and ventral pathway in human visual system, respectively.

In the WWN-7 model, X area unidirectionally inputs the images to Y area, and this implies a bottom-up connection, while the connections between Y area and Z area are bidirectional. The top-down connections indicate that the motor concepts of Z area can guide the brain to learn. In Y area, its bottom-up connections feed forward the image understanding of the network (agent) to the motor terminal.

(1) Connecting modes among neurons

Three connecting modes exist in the WWN-7 model, i.e., the bottom-up connection, the top-down connection and lateral connection. The information transmissions from X area to Y area and from Y area to Z area belong to the bottom-up connections, while the transmission from Z area to Y area means the top-down connection. There exist two modes of the lateral connection among neurons in the same area, i.e., activation and inhibition. The lateral connections exist in Y and Z areas, as shown in Figure 2, which is a simplified connection diagram of the WWN-7.

Figure 2

Three Connection Modes in WWNs.

2.2 Learning Algorithm: Lobe Component Analysis

Lobe component analysis (LCA) [13] is used as the learning algorithm in the WWNs model. In terms of the individual development, individual difference will lead to different learning rates. Because of the different learning rates, just as we teach students in accordance with their aptitudes, appropriate mechanism of the network should develop according to different individuals and each neuron can achieve the best learning rate. So it is very important to design a reasonable mechanism of the learning rate on the basis of temporal optimality. WWNs model is a neural network to roughly simulate the visual processing pathway of the human being in this work. The learning process is not once and for all, but a cycle of progressive process. In the learning process, the new knowledge will be learned, and the knowledge that is not used will be forgotten gradually. The repetitive learning process can enhance the memory of the agent. Similarly, the learning rate of WWNs is designed to reflect these ideas which are embodied in the following weight updating formulas:

where ω₁(n_j) and ω₂(n_j) are the learning rate and retention rate, respectively, both depending on the firing age n_j of neuron j. The retention rate ω₁(n_j) consolidates the knowledge that the neurons have learned. In the learning rate ω₂(n_j), the amnesic mechanism μ(n) is introduced, where m=2, r=10000, t₁=10, t₂=30 (based on the literature [13, 33]). In this way, we can dynamically adjust the learning rate and the retention rate in the weight updating process. An appropriate retention factor is set to make the network learn fast in early development stage. When it comes to a certain degree and the neuron age is large enough, the learning rate tends to be a constant gradually. So this mechanism can keep learning new knowledge persistently, and forgetting the knowledge less used.

2.3 Synapse Maintenance Mechanism

In this paper, the faces in the complex backgrounds have different types, locations and sizes. Especially in the case of different receptive fields, the backgrounds have great impacts on the detection of the face features. Therefore, it is able to estimate the standard deviation of each pixel in the input from X area to Y area by using the synapse maintenance mechanism (SMM) [31]. In training process, these pixels whose standard deviations vary greatly (corresponding to the background) will be suppressed, which will reduce the influence on the calculation of the pre-action energy of the neuron in Y area.

Receptive fields in computer vision are generally regular geometry, while the object outlines in real world are arbitrary. Therefore, the receptive field often contains partial background. In WWN-7, receptive field of the neuron in Y area is also a regular square. Obviously, the background will interfere the recognition result of the foreground. So the SMM is introduced into the WWN model to solve this problem. The SMM is expected to recognize the foreground and background in the receptive field. After the foreground outline is sought out, the background can be easily suppressed. It works to estimate the standard deviation of each pixel in the receptive field, through restraining the pixel with large standard deviation, to decrease the influence of the unstable pixel on the pre-action energy of the neuron. So this mechanism reduces the influence of the background on the neuron activation.

Supposing the input of Y neuron is p = (p₁, p₂, . . . , p_d) and the value of a neuron in Y area is v = (v₁, v₂, . . . , v_d). For the neuron i (i=1, 2, . . . , d), the matching degree between v_i and p_i can be calculated as follows:

where σ_i is used to evaluate the matching degree of the input and the value of the neuron in Y area.

According to the matching degree σ_i, each neuron decides to extract or retract the synapse dynamically. In Hebbian algorithm, v_i is the good estimate of the amount of p_i and the expected value of p_i. If v_i is estimated well, σ_i can be seen as the standard deviation of the input pixel. So the brightness dispersion of the input pixel in the same position is estimated by σ_i. A higher σ_i indicates a higher dispersion degree, meanwhile, the input pixel is more unstable and the synapse should be retracted. Now a given threshold is set to determine the synapse state. But the synapse will extend or retract repeatedly when the σ_i approaches the threshold we set. So the smooth synaptogenetic factor f_i is added in the following formula:

where ε is an infinitesimal quantity, used to avoid the zero denominator. f_i will be normalized to prevent itself from being too large.

The synaptogenetic factor f_i reflects the connecting strength between the pixel and the corresponding neuron. Range of the synaptogenetic factor f_i is between 0 and 1. If f_i is equal to 1, the corresponding pixel is completely connected, while 0 means completely suppressed.

Synapse maintenance requires pruning the input p and the value v as follows:

Then p_iand value v_iare used to calculate the pre-action energy of the neuron in Y area. Interested readers are suggested to refer the [30] for more concrete description of the SMM.

2.4 Neuron Regenesis Mechanism

Hebbian learning changes the synaptic weights between neurons, while neuron regenesis mechanism makes the inactive neurons have the chance to fire (meaning “new birth, renewal”) and make new connections with other neurons. Firing frequency is used to evaluate the neuron activation degree. If the firing frequency is low, the corresponding neurons are regarded not active, i.e., few winning times in competition. Therefore, they are possibly suppressed by their surrounding active neurons, and can’t learn new knowledge about the foreground. If these low level neurons don’t have a chance to regenerate and learn new features, it is a great waste of the neuron resources. So the neuron regenesis mechanism is designed to regulate the neuron resources, hence promote the WWNs performance.

At time t, the firing frequency is defined as follows:

where f (t) is the firing frequency of the neuron, n(t) is the neuron age (fire times) and N(t) is the current running time of the network. Accordingly, at time t+1, the firing frequency can be defined:

And the current running time of the network increases one unit:

To sum up, the firing frequency of the neuron can be calculated in the recursive manner:

Then, the neuron firing frequency of one neuron should be compared with its neighbors to determine which neuron should be regenerated. In 3D space, one neuron has a total of six nearby neurons with the distance equalling to 1. If a neuron competes to fire after winning, it should be compared with other six neurons. Suppose the neuron A is the winner and the neuron B is one of the six nearby neurons. The following criterion is adopted to compare the firing frequencies of neurons.

Only the active neurons at high active level have the ability to suppress other neurons, which means that “age” (denoting its firing times) of the neuron A is already very high. In the expression (15), n₀ is the “age” threshold, and in this work, it is set to 40, i.e., n₀=40. In addition, the firing frequencies of the nearby neurons are also relatively low. In the paper, the firing frequencies of the nearby neurons should be less than 1/4 of that of the neuron A. If the criterion is met, the neuron B should die and then regenerate to learn new knowledge.

2.5 How the Y Neurons Learn the Object Features

Firings of all Y neurons go through top-k competition, and only the winners can fire and update their weights, i.e., object features they have learned. In the top-k competition, neuron active degree is evaluated by its pre-action energy. Only the former k neurons with the largest pre-action energy can fire. The pre-action energy is composed of four parts: action value corresponding to the X input (bottom-up input), and action values corresponding to LM, SM and TM inputs (top-down inputs). Only when the four action values are large (matchwell) can the neuron fire. The firing neuron can learn the corresponding feature of the input image, and connect with the corresponding neurons in LM, SM and TM (supervised by the external environment), in other words, connecting the features with the concepts. After that, if this neuron wants to fire frequently (age increasing, more mature, learning more stable feature), it means that the feature in X layer and the corresponding concepts in LM, SM and TM must happen simultaneously, i.e., the feature in X layer is relevant with these concepts. In other words, if some feature in X layer has almost no relevance with the concepts in LM, SM and TM, this feature is difficult to be learned. Even the neuron corresponding to this feature fires a few times occasionally, its firing frequency will be not high. Since the concept is irrelevant to the feature, it cannot ensure that the four pre-action values are high. These neurons should die and regenerate through the neuron regenerate mechanism, to learn new features. Therefore, most neurons in WWNs can learn the foreground features, while few neurons can learn the background. Since compared with the foreground, the background has almost no relevance with the concepts denoted by LM, SM and TM which are provided by the external teacher.

3.1 Map Library

Face pictures used in this paper are LFW Face Database, and the complex natural backgrounds used are selected randomly from the natural images whose dimension is 38×38 pixels, as shown in Figure 3.

Figure 3

Face Images and Parts of the Training Pictures.

Sizes of the foreground objects range from 24×24 pixels to 34×34 pixels, with a pixel increasement each time, so there are 11 kinds of sizes. Due to the different sizes, the foreground objects have different locations, so the foreground objects with smaller sizes have more locations. The location number ranges from 225 ((38-24+1)×(38-24+1)) to 25 ((38-34+1)×(38-34+1)).

Objects to be recognized are 5 different human faces and each face has 11 different sizes. So there are 55 types in total, and the face location numbers in the complex backgrounds change from 5×5 to 15×15, as shown in the Table 1. For the same face type, images with even sizes (24×24 pixels, 26×26 pixels, etc.) are used for training, while those with odd sizes (25×25 pixels, 27×27 pixels, etc.) are used for testing. There are 25 face images to be tested in experiment. For each face, it can appear all possible locations in the backgrounds. Thus the recognition rate can be calculated.

The network will be randomly assigned to a complex natural background first, then the foreground object is embedded in the background in the training phase. After the foreground objects at all locations have been trained, the WWN performs a whole train procedure.

3.2 Inputs and Outputs

Inputs from X to Y layer are introduced first. The connections between X and Y layer are local and the different receptive fields mean the different characteristic information. The network involves the receptive fields of 7×7 pixels, 11×11 pixels, 15×15 pixels and 19×19 pixels, corresponding to the four parts of Y layer, respectively. Taking the receptive field of 7×7 pixels as an example. The retina can only capture the square range of 7×7 pixels each time. Each receptive field starts from the top left corner of the image and moves down by a pixel each time. When it moves to the bottom of the image, the receptive field moves to right side by one pixel and starts to scan from the second column. This process will be carried out until the entire image has been input. The scanned data are stored in a sequential order, so that the inputs of the first part of the Y layer are obtained. Other receptive fields can replace the former one to repeat this process and the inputs of the remaining three parts of Y layer will be obtained according to the same steps.

There are some independent inputs from the three parts of Z layer to Y layer. Because of the five different human faces, input dimension of TM is 5×1. Due to the 11 different sizes of the face images, input dimension of SM is 11×1. Location number must contain all the possible locations, so the input dimension of LM is 225×1, as shown in Table 2.

Above sections illustrate that different receptive fields correspond to the different parts of Y layer. Taking the receptive field of 7×7 pixels as an example. There are 9 sub-layers in Y₁ and the input image dimension is 38×38 pixels. Each small scanning frame corresponds to a neuron, so there are (38-7+1)×(38-7+1)×9=32×32×9 neurons in Y₁ sub-layer. In the similar way, the neuron numbers in Y₂, Y₃ and Y₄ sub-layers can be obtained and showed in Table 1.

Y layer receives the inputs from X layer and the three kinds of inputs (type, size and location) from Z layer simultaneously. The pre-action energy of Y neuron indicates the matching degree between the input and its corresponding weight, which represents the memory of the neuron. The greater the pre-action energy is, the more consistent it is between the input and the memory information, and the easier it is for the neuron to fire. After the pre-action energy are obtained, neurons in the Y layer compete to get the output according to the top-k competition mechanism.

3.3 Calculation and Distribution of Neurons

Before calculating the pre-action energy, neuron input should be normalized. Variable input and weight represent the input and connection weight between layers, respectively, while epsilon represents a small positive number. The normalization process can be depicted as follows:

1. Search the minimum and maximum of each row to form a vector min and max, respectively;

2. Calculate the difference between the maximum and the minimum. To avoid the zero denominator, a small positive number epsilon is introduced: diff=max-min+epsilon.

3. Execute the normalization process: input = (input-min)./diff; weight = (weight-min)./diff.

After normalizing the inputs and weights, they are used to calculate the pre-action energy of the Y neurons in the next step.

Pre-action energy determines whether the neurons can fire. If it is high, the neuron will become the winner in the top-k competition and will fire, and updates its weight and age through the Hebbian learning mechanism.

Split the formula 1 into three parts and calculation of the pre-action energy of the Y neuron can be depicted as follows:

where r_b and r_t are the pre-action value of the bottom-up connection and top-down connection, respectively. Dot above the vector represents the normalization of the vector. Subscript b and t represent the bottom-up input and top-down input, respectively. α refers to the ratio of r_t to r. When the network is in training state, α=0.5. When the network is in testing state, α=0 and the pre-action energy of Y neuron is only determined by the bottom-up input. r_t is made up of three parts: Z_TM, Z_LM and Z_SM, and each of them accounts for 1/3.

Then the achieved pre-action energy values are ranked and the neurons ranked in the former k (k can be selected according to the network situation, in this work, k=1) are activated. Energy value of the activated neuron is set to 1 and those of the rest neurons are all set to zero.

3.4 Visualization of the Internal Weights

After the network is trained 20 times, Figure 4 visualizes the bottom-up weights of partial neurons in the four sub-layers of Y layer. Each small box in Figure 4 represents the bottom-upweight of a Y neuron and the feature that the neuron has learned. Each box is separated by a white line and the dimension represents the size of the corresponding receptive field. The black box represents the dormant neuron which is not activated and is in the initial state. In the small receptive field (i.e., 7×7) of Y₁, the faces with different sizes have more similar characteristics, so more characteristics are cumulative. In this receptive field, most features that the neurons have learned are about the object margin orientation. While in the largest receptive field (i.e., 19×19) of Y₄,

Figure 4

Visualization of the Weight Value from X Layer to Y Layer.

local or global features learned can be mainly identified. Figure 4 indicates that most neurons have learned the face characteristics.

Furthermore, the receptive field of Y₄ is visualized in Figure 5. Sub-figures a, b, and c represent the top-down weights of the type, location and size, respectively. The weights are normalized and they are separated by white lines. Weight dimension of single neuron in sub-figure a and b is 5×1 and 11×1, respectively, which cannot deform into a square. So the box with full zeros is set up and the weight of a single neuron is placed into it to display.Weight dimension of a single neuron in sub-figure c is 225×1,which can deform into a square with 15×15. The white spots in different positions represent the different activated neurons.

Figure 5

Visualization of the Weight Value from Z Layer to Y Layer.

Visualizations of the synaptogenic factors, and standard deviations of the synapse matching are shown in Figure 6 and each box in the figure represents a neuron. In sub-figure a, foreground contour has been automatically formed by the synapse maintenance mechanism. In sub-figure b, standard deviations of the synapse matching are constantly updated in accordance with the Hebbian learning. Black box represents the dormant neuron which is not activated.

Figure 6

Visualizations of the Synaptogenic Factors and the Standard Deviations.

3.5 Effect Demonstration of Neuron Regenesis

WWN-7 is “skull-closed”, with the autonomous developmental ability, so the network internal resource can be regulated dynamically in the network learning. Here, resource dynamic regulation denotes that the connection among neurons can be built up or cut off dynamically.WWN-7 has the autonomous developmental ability to strengthen the right knowledge learned and weaken the false cognition, through continuous learning.

Figure 7 visualizes the bottom-up weights of Y neurons corresponding to the receptive field with 19×19 pixels, in different training epoches. One training epoch means that WWN has learned all training foreground images in all possible locations in the complex background. The top left sub-image (epoch=0) shows the initial state of the network, and there are no connections among the neurons, which means that the current WWN has strong ability to learn new knowledge. Just like a newborn baby, it has strong plasticity. The top right sub-image (epoch=1) displays the object characteristics that Y neurons have learned after the first network training epoch. We can see that some of the current characteristics are obscure, and it is difficult to distinguish them. In other words, the current WWN is not sure whether the characteristics learned is useful and should be kept. The bottom right sub-image (epoch=5) illustrates the weights after 5 training epochs, it is clear that partial characteristics are very obvious and a few neurons have regenerated to learn new features. Through the learning, the WWN has retained partial useful features, and the neuron regenesis mechanism has begun to regulate the neuron resource. With the network learning (epoch becomes larger), more and more neurons enter learning state and learn the object features. The bottom left sub-image (epoch=10) shows the weights after 10 network training epochs, we can see that most features are very obvious, and the neurons have been activated efficiently to learn knowledge, and the suppressed neurons have been reactivated to learn new features. Just like the human brain can produce new neurons to learn new knowledge, through the neuron regenesis mechanism, the WWN can reactivate the suppressed neurons to regenerate and learn new knowledge. So in the limited resource space (i.e., fixed neuron number), WWN can carry out efficient and accurate learning.

Figure 7

Visualization of the weight value from X layer to Y layer under different epochs.

Furthermore, to display the effect of the neuron regenesis, the content in red box in Figure 8 is amplified. The sub-figure (a) shows that, after one training epoch (i.e., epoch=1), the features that the neurons in the right top corner learned contain parts of foreground features and partial backgrounds. Since the background has little relation with the TM, LM and SM in Z layer, the fire frequencies of these neurons are not high. Through the neuron regenesis mechanism, these neurons will die and regenerate to learn the foreground. Sub-figure (b) displays that after 5 training epochs (i.e., epoch=5), these regenerating neurons have learned new face characteristics which contain most part of the foreground and little background.

Figure 8

The first neuron regenesis case.

Sub-figures (a) and (b) in Figure 9 illustrate that, after 1 training epoch and 5 training epoches, the neurons in the corresponding red box are in the learning state, but the foreground features they learned have not changed, and several neighbouring neurons learn the same or similar foreground characteristics. Sub-figure (c) shows that after 10 epochs, face features that the neurons in corresponding location learned have changed, and the neighbouring neurons have begun to learn different face features which means these neurons have experienced the procedure from dying to regenerating, and learned new foreground features.

Figure 9

The second neuron regenesis case.

Figures 8 and 9 illustrate the dynamic regulation of the neuron regenesis mechanism for the network internal resource: it not only makes the neurons that have learned the useless background information to regenerate and learn new foreground features (denotes in Figure 8), but also let the neurons with low fire frequencies in the neighbouring neurons learned the same or similar foreground features to regenerate and learn other foreground features, thus more foreground features can be learned (denotes in Figure 9). Therefore, the neuron regenesis mechanism is a good supplement to the Hebbian learning.

3.6 Performance Evaluation

Since the WWN-7 can learn three types of features: type, location and size, therefore, performance of WWN-7 can be evaluated from the recognition results of these three aspects. The recognition rate of the type is the proportion of the test images which are correctly identified. The recognition error of the location is the Euclidean distance between the recognition location and the real location. Recognition error of the size is the Euclidean distance between the recognition dimension and the real dimension. Recognition error of the location and size is the average of all the recognition errors of the test images. The performance of WWN-7 can be discussed from three situations: WWN-7 without the synapse maintenance and neuron regenesis, WWN-7 with the synapse maintenance only, and WWN-7 with both synapse maintenance and neuron regenesis. The diagrams of the recognition effect are shown in Figure 10.

Figure 10

Face recognition rates with WWN-7 in three cases. (a) Total recognition rate, (b) location error (unit: pixel), (c) dimension error (unit: pixel).

Figure 10(a) shows that the recognition rate of the WWN-7 without the synapse maintenance and neuron regenesis mechanism can be 98.1%,while that of the WWN-7 with these two biological mechanisms can reach 99.6%. Performance of the WWN-7 with these two mechanisms is better than that of only with the synapse maintenance. But during epoch=4 and epoch=9, performance of the former is worse than that of the latter, resulting from the fact that during the early time of the neuron regenesis, the neurons that have learned part of foreground characteristics and partial backgrounds will die. And the connections between these neurons are cut off totally. Their ages are set to 1 again, and their weights return to initial states. Before these neurons learn the new features and set up stable connections, they are not conducive to the performance of WWN-7. But with the training progress, these neurons learn more stable foreground features quickly, so its total performance is better.

Figure 10(b) displays that location error of the WWN-7 with these two mechanisms is the smallest among the three cases, i.e., 1 pixel. In face images, the foreground objects (faces) have little difference, while the backgrounds are randomly added from natural images which bring great difference. With the synapse maintenance mechanism, the background interference can be suppressed effectively, so it is a great help to identify the types and detect the locations. Similarly, Figure 10(c) illustrates that the size (or dimension) error of the WWN-7 with these two mechanisms is the smallest among the three cases which is close to 1 pixel. As previously mentioned, in training phase, face images with even size are trained, while in testing phase, odd size are adopted. There exists one pixel difference between the even size (e.g., 24×24 pixels) and the corresponding odd size (e.g., 25×25 pixels), so the best result of the dimension error is close to 1 pixel.

To sum up,WWN-7 with the synapse maintenance significantly improves the face recognition effect, compared with that of without the synapse maintenance mechanism. If it is added with the neuron regenesis mechanism, its recognition performance can be further promoted but the promotion is not very obvious.

3.7 Performance Comparison

To demonstrate the performance of WWN-7 on face recognition further, 3 classical methods: PCA+third-order nearest neighbor algorithm [12], the BP neural network [38], robust sparse representation algorithm [7] are employed to compare the effect with the WWN-7 model. In above experiment, performance of WWN-7 is evaluated from three indexes: the recognition rate of types, the identification error of locations and the sizes. However, these classical methods can only classify the face images, while the location and size of the face can’t be identified. Therefore, in this comparative experiment, only the recognition rates of the type are evaluated. The recognition results are provided in Table 2 to Table 6. The PCA dimension and the feature dimension of sparse representation in Table 2 and Table 4 are the dimensions of feature vector after extracting features.

In the comparative experiments, for the BP network, the neuron numbers of input layer, hidden layer and output layer are 5, 10 and 5, respectively. Each person has 11 face images, 6 images are used to train the network and 5 images are used to test (the training set and test set are the same as those used in the WWN-7, respectively). Hidden layer neurons use ‘tansig’ transfer function, and output layer neurons adopt ‘purelin’ transfer function. Network train function employs the ‘traingdx’, and its weights and thresholds are automatically initialized. Network training performance function adopts mean squared error (mse),when the mse=0.001, the network stops. According to the Table 2 and 4, PCA dimension adopts 140, and the feature dimension of sparse representation uses 150. To avoid the random error, the same recognition procedure is done 30 times with the 6 methods, thus to calculate the average, standard deviation, minimal and maximal value of the recognition rate. The final results are shown in Table 6.

Table 6 displays that the recognition rates of the former three classical methods are lower than those of the WWN-7, which result from two reasons: the first reason is the interference of the different backgrounds, and the second one is the location and size variation of the foreground. Large standard deviations imply that the former three methods cannot well adapt the changes of location and size of the foreground in complex background. Due to its biomimetic mechanism, WWN-7 with the synapse maintenance mechanism can seek out the foreground outline to recognize the foreground and the parts of the background in the receptive field. Moreover, the neuron regenesis mechanism can activate the suppressed neurons in the competition to regenerate and learn new foreground features, so the recognition rate of WWN-7 with the two mechanisms is higher than other approaches. These comparative experiments indicate that WWN-7 can well realize the recognition and detection of the types of human faces in complex backgrounds.

Finally, we compared the location error and size error between our work and the standard WWN-7 [35]. The results are provided in Table 7 and 8. From the Table 7 and 8, we can see that the improved WWN-7 model including the synapse maintenance mechanism and the neuron regenesis mechanism can effectively promote the recognition rate of the location and size of the foreground in complex backgrounds. For these two mechanisms, the synapse maintenance mechanism contributes to the performance improvements more than the neuron regenesis mechanism.

3.8 Computational Complexity

The previous part compares the performance of different methods on the face recognition, in this part, we theoretically compare the computational complexity of theses methods.

Since the WWN is a typical feed-forward network, and the computational complexity of common feed-forward network is O(n4) [9], where n denotes the number of neurons in each layer. Considering the number of layers in WWN is very small, so we can neglect its influence on the computation complexity, thus we can achieve the computational complexity of WWN, i.e., O(n³). Similarly, we can get the computational complexity of BP neural network, i.e., O(n³). While that of the PCA and robust sparse representation algorithm is O(n), where n represents the data dimension.

Based on the above discussion, we can approximately assess the computational complexity of the three approaches. For the same learning task, WWN and BP neural network have the highest computation complexity, which means a longer operation time. While PCA and robust sparse representation algorithm has the lower one, so their computation time is relatively little. Although WWN has the highest recognition rate among the three methods, how to design better network architecture to enhance its computation speed is an important research direction.