Application of portrait recognition system for emergency evacuation in mass emergencies

2.1 Application of portrait recognition

In police work, with the wide application of high-definition monitoring (Figure 1), the level of portrait recognition has been gradually improved so that it can play a good role in more fields and provide reliable support for target recognition, monitoring, positioning, etc. [15]. The specific application scope includes:

1. recognition of identification photo: all law enforcement departments can identify the authenticity of a suspect’s certificate to understand the real identity of the holder;

2. surveillance and arrest: the police can monitor and arrest escaping criminals in some important places such as airport, station, bank, etc. to improve working efficiency;

3. security verification: in some important places, the image recognition system can store the images of people in and out to improve security;

4. emergency handling: in the face of mass emergencies, the police can quickly understand the details of the trapped people.

Figure 1

High-definition monitoring in the command center of a public security bureau.

This paper mainly studied the application of the portrait recognition system for emergency evacuation in mass emergencies. If there is a mass emergency, it is often necessary for police officers to organize emergency evacuation to avoid further development of the situation. In an emergency, the scene often has poor signal and difficult evacuation for various reasons. Therefore, in the process of evacuation, it needs a high-level portrait recognition system to analyze the images in the video surveillance and recognize and detect portraits to help the police to confirm the identity of the trapped people and evacuate in time. The system can ensure the efficiency and safety of police work and improve people’s satisfaction.

In order to achieve a high level of evacuation and rescue and ensure the timely rescue of the trapped people and the personal security of the rescuers, we need a high-level portrait recognition system to achieve the confirmation of personnel information by detecting and identifying portrait targets in video surveillance.

2.2 Overall structure of the system

The portrait recognition system needs to capture the monitored image through accessing the high-definition monitoring and send it to the back-end server through the network for real-time detection and recognition of the portrait to confirm the identity of the portrait. In the system designed in this study, the router is connected with the external network, and the high-definition monitoring equipment and server are connected with the router. The server can receive the video image data transmitted by the monitoring in real-time through the IP address and can also respond to the request of the external network to ensure that the system can be accessed in any scene through the Internet. In emergency evacuation in mass emergencies, the places that need emergency evacuation are monitored through high-definition monitoring, and the identity of the people who need to be evacuated can be quickly obtained through portrait monitoring and identification after getting the images transmitted by the system.

The overall structure of the system mainly includes the following parts:

1. Image preprocessing: in order to facilitate the follow-up work of portrait recognition, the collected video image needs to be preprocessed. In this study, the portrait is geometrically normalized using the bilinear interpolation method [16] to correct the portrait at different distances. Then, the image is denoised by the Wiener filtering algorithm [17]. Finally, the contrast of the portrait is improved by histogram equalization [18] to make the difference between different features more significant.

2. Portrait detection: portrait detection means determining the position and size of the portrait in an image and circling it with a rectangular box to provide services for the subsequent portrait feature extraction and recognition. In this study, the SSD algorithm [19] is used. Based on deep learning, it is a one-stage general object detection algorithm proposed in BCCV2016 and has a good detection performance for objects of different sizes.

3. Portrait recognition: portrait recognition refers to extracting features after detecting the location of the portrait and comparing the features with the features in the database to obtain the recognition result. This study designed an improved feature extraction method combining PCA with LDA and used SVM as a classifier to realize portrait recognition.

2.3 Portrait detection algorithm

The SSD algorithm takes VGG16 as the basic model and adds a new convolution layer to obtain more feature maps. The SSD algorithm takes images in a size of 300 × 300 as inputs, replaces FC6 and FC7 in VGG16 with a convolution layer, and adds a recognition layer after Conv7 to perform down-sampling. In the Conv4_3 layer, there is a classifier layer for extracting the feature map. Every feature map is used as the input of the detection layer to predict the target. Then, the default boxes are defined in the detection layer to complete the prediction of the target. In the system designed in this paper, six feature maps of different scales are used.

In the aspect of the loss function, the SSD algorithm needs two operations: one is to classify the target, and the other is to regress the frame of the candidate box. The loss function can be written as:

where N refers to the number of positive samples of a priori box, x i j p ∈ { 1 , 0 } refers to the indication parameter ( x i j p = 1 means the ith priori box is matched with the j-th ground truth and the class is p), c refers to the predictive value of the class confidence, l refers to the predictive position value of the bounding box corresponding to the priori box, g refers to the position parameters of ground truth, L _conf(x, c) refers to the classification loss function (softmax), α refers to a weight, and L _loc(x, l, g) refers to a regression loss function (smooth).

2.4 Portrait recognition algorithm

The PCA method is also known as K-L transform [20], which takes the minimum mean square error as the criterion and projects the features of a high-level space into a low dimensional space to ensure that the reduced data can effectively represent the original data, so as to improve the operation speed. However, this method can only reconstruct the original samples and cannot achieve a good classification of samples. Therefore, this paper improves the traditional PCA method and combines it with the LDA method [21].

In order to reduce the influence of uneven illumination on portrait recognition, first, PCA is improved based on standard deviation and local mean. First, the dark area in the portrait is determined. It is assumed that the global mean of the portrait is E _g and the local mean value is E _s.

and k ₀ is a constant smaller than 1; it means this part is dark and needs enhancement. If the area to be enhanced is

where

σ g is the global standard deviation, and k ₁ and k ₂ are constants smaller than 1, then the formula of portrait enhancement can be written as:

where δ refers to the amplification gray coefficient, η and y are stretching contrast coefficients, and σ s is the local standard deviation.

After portrait enhancement, the PCA + LDA method is used for feature extraction. It is assumed that the training sample is

where m is the number of pixels and N is the number of images, then the training sample matrix can be written as:

where C stands for the number of classes.

All the images in the sample are transformed into column vectors, and two values are calculated, i.e., the intra-class average image of every kind of image:

and the population average image:

Then, the two difference images are calculated:

and

The covariance matrix of the difference is calculated:

The nonzero feature value τ of Φ and the corresponding feature vector V are calculated, and the arrangement is

Then, the accumulative contribution rate is calculated:

The first r feature vectors are selected as the subspace of PCA:

The input image, x, μ _i , and μ are projected to the subspace of PCA, then

and

The three values in LDA are calculated:

intra-class scatter matrix:

population intra-class scatter matrix:

inter-class scatter matrix:

According to Fisher criterion, the best projection space is obtained:

According to the Lagrange method,

The first p feature vectors are selected as the subspace of LDA:

The feature spaces of PCA and LDA are fused:

For any column vector x, in the fused feature space, its projection relationship is written as:

In portrait recognition, the training image and the test image are projected into the fused feature space, respectively, and classified by the SVM classifier to obtain the result of portrait recognition.

3.1 Experimental environment and data set

The system test environment was Intel®Core(TM)i5 CPU M540@2.53 GHz. The size of the memory was 2.00 G. The operating system was Windows7. It was implemented in Microsoft Visual C++ 6.0 integrated development environment. Intel open-source visual library OpenCv2.2 was used.

The system was applied in a shopping mall. The mall was equipped with high-definition cameras for monitoring. The collected videos were processed. The obtained images were used for human image recognition. The obtained monitoring images were sorted and processed manually, and the names of people were added manually. Finally, 12,015 images in a JPG format of 250 × 250 were obtained, involving 4,952 people.

The data set used in the portrait detection experiment included one thousand images randomly selected from the 12,015 images. The 1,000 original portraits were taken as positive samples; then, the 1,000 images were randomly cut to produce 1,000 non-portrait samples as negative samples. There were 2,000 images for the portrait detection experiment.

The data set used in the portrait recognition experiment included 6,000 pairs of portraits randomly selected from the portrait detection data set, including 3,000 pairs of positive samples, i.e., matched portraits, labeled as 1, and 3,000 pairs of negative samples, i.e., non-matched portraits, labeled as 0.

3.2 Portrait test results

The performance of the SSD algorithm used in this study was compared with the Adaboost algorithm [22] by the established data set, and the results are shown in Table 1.

It was seen from Table 1 that the missing detection rate of the Adaboost algorithm was 5.43%, the missing detection rate of the SSD algorithm was 0.78%, which was 4.65% lower than the Adaboost algorithm; the false detection rate of the Adaboost algorithm was 3.56%, and the false detection rate of the SSD algorithm was 2.89%, which was 0.67% lower than the Adaboost algorithm. It showed that the SSD algorithm had a better performance in portrait detection and could accurately detect the portrait in the image, which provides a reference for subsequent portrait recognition.

3.3 Portrait recognition results

The performance of PCA, LDA, PCA + LDA, and improved PCA + LDA algorithms was compared to verify the effectiveness of the algorithm designed in this study. The classifier was SVM in the four algorithms. The recognition rates of different algorithms are shown in Figure 2.

Figure 2

Comparison of recognition rate between algorithms.

It was seen from Figure 2 that when the PCA algorithm was used for portrait recognition, the recognition rate of the algorithm was only 79.3%; when the LDA algorithm was used, the recognition rate was 80.1%, which was not high; when PCA and LDA algorithms were combined, the recognition rate was 88.6%, which was significantly higher than the former two algorithms; the improved PCA + LDA algorithm had a recognition rate of 95.8%, which was 7.2% higher than the PCA + LDA algorithm. The above results showed that the algorithm was further improved after improving the PCA algorithm.

The performance of different algorithms was analyzed through the receiver operating characteristic (ROC) curve. The larger the area under the ROC curve was, the higher the reliability of the algorithm was. The results are shown in Figure 3.

Figure 3

The ROC curve of different algorithms.

It was seen from Figure 3 that the ROC curve of the improved PCA + LDA algorithm was the closest to the top left corner and had the largest area under the curve, followed by the PCA + LDA algorithm and the PCA algorithm. The results showed that the improved PCA + LDA algorithm had the highest algorithm and best performance among the four algorithms.

It was seen from Figure 4 that the recognition time of PCA or LDA alone was 934 and 876 ms, respectively; after combining PCA and LDA algorithms, the recognition time was reduced to 543 ms, which was much better than the former two algorithms; after further improving the PCA algorithm, the recognition time was 465 ms, which was 14.36% shorter than the PCA + LDA algorithm. In practical application, the improved PCA + LDA algorithm had a higher response speed and a better performance in emergency evacuation in mass emergencies.

Figure 4

Comparison of recognition time between algorithms.