Automatic target recognition method for multitemporal remote sensing image

1 Introduction

With the development of the remote sensing technology, the spectrum and the spatial and temporal resolution of remote sensing imaging have been improved continuously, which makes the image data of remote sensing sensors to be collected and transmitted to the ground rapidly. How to identify an object from the massive remote sensing image is a challenging and an urgent problem. It is important research the visual function of human being and the process of human recognition and the physiological mechanism of human identification. The reason why a human being can quickly and accurately identify the designated target in the remote sensing image is because a human being has the necessary knowledge and the ability to acquire new knowledge through knowledge reasoning. The foundation of the use of knowledge and ability is the perfect visual perception system formed by human evolution over a long time. Therefore, it is of great value to research how to apply knowledge reasoning and visual mechanism to object recognition of the remote sensing image [1].

Currently, the resolution of the remote sensing image is becoming higher, and information is more abundant. It is possible to realize the automatic recognition of targets such as road, building, and airport. The widely used automatic target recognition technology plays an important role in civil navigation and precision attack of military targets. In recent years, many researchers have done a lot of research in this field. But so far, the technology of automatic information extraction and target identification from the remote sensing image is still not perfect and mature. Most automatic target recognition technologies usually can identify certain targets and do not have universally applicable values.

In this article, considering some typical remote sensing targets as an example, we research on configurable target segmentation and recognition. The object-oriented segmentation and recognition method is introduced. The main idea is as follows. The image is segmented into object primitives, and the algorithm of object segmentation and recognition is abstract and independent. Then, according to the appropriate criterion, the object segmentation method is used to obtain the target to be recognized. Automatic recognition of multiclass targets is achieved by using the feature configurable recognition method. The main research features of this article are as follows. First, in the aspect of object segmentation, the configurable object-oriented image segmentation method is studied. According to the characteristics of the target, the object recognition preprocessing method is introduced, and the configuration result of target segmentation is given. Second, in the aspect of feature extraction, the feature extraction algorithm based on the high-resolution remote sensing target is studied. The target features of spectrum, texture, and geometry are deeply mined, which provides sufficient candidate features for feature configurable target recognition [2]. Finally, in target recognition, the feature configurable target recognition technology is studied. A feature configuration criterion based on the separability measure is proposed. The classifier is used to learn the features that have been configured and determine the classification threshold and weight coefficient of each feature, so as to achieve the identification parameter configuration. The proposed recognition algorithm is compared with the support vector machine recognition algorithm. Experimental results verify that after feature configuration, the accuracy of target recognition is improved.

2 Automatic target segmentation of multitemporal remote sensing image

Automatic image segmentation is the technology and the process of automatically dividing the image into several specific and unique regions and proposing interested objects. It is a key step from image processing to image analysis. As an ancient and classic research topic in image processing, segmentation is an indispensable part of remote sensing image processing. Segmentation plays an important role in the computer vision system, which is a bridge between low-level vision processing and high-level vision processing. Therefore, how to effectively segment and extract useful information is the key point of the vision-based remote sensing image processing research. A deep research on target segmentation and extraction of visible light remote sensing images is the most important part [3].

The segmentation of the remote sensing image is to classify the pixels in the image by using some rules or algorithms according to the spectral luminance, spatial structure, or other feature information of different bands. The ultimate goal of the classification is to realize the recognition of targets of interest. In the process of recognition, the features of the target are needed to be extracted, which are the contour or inner area that represents the target of interest.

Segmentation is the basic operation of image processing. In remote sensing image processing, to achieve the recognition of some target of interest, the region of interest is usually needed to be understood and localized. The localization of the target of interest refers to separating the target from the background by the proper segmentation operation. The remote sensing image is first segmented into a region containing the corresponding visual meaning, which is data preparation for recognition. However, due to the limitation of image acquisition conditions, the segmentation and the extraction of the target of interest in different application scenarios often cannot be properly controlled. Therefore, the segmentation is still a very difficult and complex problem. In the practical application, there are still many aspects that need further research and breakthrough.

Image segmentation is the division of the optimal region and the most significant part of the image. Mastering the overall contour information can ensure the accurate position of the image boundary and improve its accuracy, so as to achieve variable level set segmentation. If the level set function changes with time, then its expression is u(x, y, t), which can make E[u] smaller gradually, leading to the significance of the area within the closed curve increasing gradually. Therefore, the auxiliary variable t is introduced based on u(x, y), then, the following equation is obtained.

The level set function is near the zero level set, and the preprocessing can make the complex image surface smooth and thus the final convergence result reach the edge of the target object.

According to the proposed variational level set method, the differential equation of the level set function can be obtained by minimizing the energy functional of the level function. The expression of calculation dE/dt can be obtained as follows:

In this equation, c ₁ = G _i /N _i , c ₂ = G _o /N _o , λ ₁, λ _2, λ ₃ are the weight parameters of the significant term, which are the parameters related to the evolution curve and the total gray value of the evolution curve.

It can effectively segment the visual saliency image in the saliency map. The level set of variation without initialization can result in the smooth image, a good segmentation effect, and the fast convergence speed. Image segmentation based on visual saliency can improve the edge details of the image, and hence, the texture clarity is high. As a result, more segmentation information is obtained, and the target object is segmented quickly and accurately.

In this section, the method of object segmentation in the remote sensing image is used. The object blocks are obtained by these segmentation methods. As the processing unit of the subsequent target detection, these object blocks provide a feasible preprocessing technology for the object-oriented object segmentation method. For the different characteristics of several typical targets in the remote sensing image, the object segmentation technology proposed in this section meets the preprocessing of multiclass typical remote sensing target segmentation [4]. This method mainly uses the edge smoothing technology and the clustering segmentation technology, which are used for preprocessing objects with strong visual sense in the object-oriented segmentation.

Figures 1 and 2 show the mean shift smoothing result for a high-resolution remote sensing image under different kernel bandwidth parameters. In Figures 1 and 2, remote sensing images contain rich remote sensing target information, including airports, waters, woodlands, and residential areas. Since these remote sensing data are mosaicked with two remote sensing images, the obvious block effect can be seen.

Figure 1

Original image.

Figure 2

Segmented image.

Figures 1 and 2 show that, with the increase of the bandwidth of the kernel function, the greater the smoothness of the image, the more blurred the color resolution. In the original image, with a variety of bandwidth parameters, after a different degree of Mean Shift smoothness, the edge information of the region with strong significance in the image can be still retained well. The block effect boundary can still be seen clearly after smoothing. The boundaries of these block effects also fully verify that the mean shift smoothing method is a good edge-preserving smoothing technique. Then, each pixel in the image is iterated through the mean shift vector and converges to the appropriate color value. The segmented image has a good smoothing effect. All kinds of remote sensing objects, such as airport, water, woodland, and residential area, converge to their respective color values, which lays a good foundation for subsequent object recognition [5].

3 Automatic target extraction of multitemporal remote sensing image

One of the basic steps of target detection and recognition of the multitemporal remote sensing image is the selection and the extraction of target features, which affects the accuracy of target recognition to a large extent. Remote sensing target recognition usually needs to take into account the features of the spectrum, the shape, the texture, and the spatial relation to obtain more ideal detection and recognition results. There are also great differences in the features used in different target recognitions. For example, the feature of a typical parallel line is used in airport target detection. The circular feature and the spatial distribution feature of the oil tank are used in the oil depot group recognition. The features of texture and spectrum are used in water extraction [6].

The airport target detection can be characterized by a typical parallel line. The identification of the oil depot group uses the circular feature and the spatial distribution characteristics of the oil tank. The water extraction should use the texture and spectral features. Because the original satellite remote sensing image is generally poor in quality, which can be easily distorted by the illumination and climate, in the process of target detection and recognition, and the shape feature is more important than other features because of its better stability.

For the recognition of the linear targets of the airport and port and the blob target of the oil depot and aircraft, the fast realization algorithms of the linear feature extraction, the linear structure feature extraction, the circular feature extraction, and the regional geometric feature extraction are used in this article, which improves the extraction speed of the typical shape features and lays the foundation of the improvement of the subsequent detection and recognition.

Figure 3 shows the linear feature extraction.

Figure 3

Linear feature extraction.

The artificial targets in the remote sensing image (such as bridges, buildings, and so on) usually contain parallel and vertical lines, resulting in more U-shaped structure, or semi-rectangular-shaped structure, which are powerful evidence for the existence of artificial targets. The U-shaped structure can be seen as two parallel lines and a vertical line (the vertical line is the common side of the two L-shaped structures) or the combination of two relatively distributed L-shaped structures. The U-shaped structure is composed of two L-shaped structures with sharing one side. It should be noted that the other two sides must be on the same side of the public side. In Figure 3, the two L-shaped structures do not share a straight line, but there is one line segment approximately collinear, that is, the straight-line area. However, these two lines should be combined into a straight line (dotted line in Figure 3) [7].

4 Automatic target processing of multitemporal remote sensing image

The goal of remote sensing image processing is to enhance and extract geoscience information needed for the target of interest. There are many differences in the digital processing methods of remote sensing images because of different goals of geoscience, different working areas, different types of images, and the different information to be extracted. However, as a whole process, the working methods of the remote sensing image have their own characteristics and follow certain objective laws. This section analyzes the process of the remote sensing image [8].

The factors that affect the selection and the scheme of remote sensing image processing are very complex. Currently, there is no general remote sensing image processing flow for reference. Therefore, a new general process of remote sensing image processing is introduced in this paper, as shown in Figure 4.

Figure 4

Automatic target processing of multitemporal remote sensing image.

5 Automatic target recognition of multitemporal remote sensing image

In the process of target recognition of high-resolution remote sensing images, it is often necessary to divide the target to be identified into some pattern class according to certain criteria. The formulation of these criteria is based on the analysis and learning of the target samples, and the process of analyzing and learning samples is the process of extracting and configuring the sample features. With the target feature extraction methods, a large number of sample features can be obtained from training samples. These sample features contain enough class information. Then, a classifier can be designed to achieve the correct classification. However, it is difficult to determine what features contain rich classes of information and what features do not contain abundant class information. Because all kinds of objects have different attributes, one of the features makes class information more uncertain. To improve the recognition accuracy, the feature information is always extracted to the maximum extent, which directly causes the high dimension of the sample features, that is, the problem of dimension disaster often occurs in pattern recognition [12]. The dimension disaster is also often accompanied by the rapid enhancement of the complexity of image processing algorithms, the decline in processing speed, and the overfitting phenomenon in the sample learning process due to a large number of unrelated and redundant features. It eventually leads to a decline in the correct rate of target recognition. Therefore, a feature description method suitable for target recognition process can not only improve the accuracy of target recognition but also reduce the complexity of the image processing algorithm and the time required for sample training. It provides a certain technical basis for the real-time and intelligent of target recognition [13].

Although there are many target feature extraction methods and the feature dimension can be very high, the feature dimension of the target recognition is not high in the actual target recognition process, and the feature dimension cannot determine the correct rate of target recognition. The purpose of feature configuration is to find out the proper and effective features from the original features of the target. In this section, how to measure the importance of feature is researched to provide a basis for feature allocation and separability feature configuration criteria.

According to the theory of pattern recognition, the law of feature distribution is reflected by the probability density distribution. Usually, there are some of the following cases for the probability distribution in one dimension of the two types of samples [14].

In Figure 5–7, through two classes of C1 and C2, and three possible features of feature A, feature B, and feature C, three possible cases of the probability distribution of sample features are given. These figures show that the probability distribution of different classes on feature A presents a partially separable case. This is because a small part of the regions overlaps in the two classes of probability distributions. For feature B, the overlap region of the two classes of probability distributions is very large, which will cause a significant increase in the error rate of target recognition. The probability distribution of feature B shows a weakly separable feature [15]. Feature C shows an ideal probability distribution. There is no coincidence between the probability distributions of the two classes C1 and C2 on the feature attribute. Therefore, this feature is very suitable for target recognition and classification. The probability distribution of feature C shows a fully separable feature.

Figure 5

Partial separable recognition (feature A recognition).

Figure 6

Weakly separable recognition (feature B recognition).

Figure 7

Fully separable recognition (feature C recognition).

According to the theory of multitemporal remote sensing image recognition, the remote sensing image is actually a two-dimensional matrix composed of pixels. Each pixel in the remote sensing image contains spatial information and color information. Spatial information is the coordinate vector of pixels in the remote sensing image matrix, expressed as A ^S. Color information is the vector of the color value of each band, denoted as A ^C. In the case of a gray image, the color information is the one-dimensional vector. In the case of a color image, color information is a three-dimensional vector. If it is a multispectrum image, the color information is the p-dimensional vector and the p is the number of bands of color information. Each pixel is regarded as a sample point of mean shift. Each sample point x is a p + two-dimensional vector A = (A ^s, A ^r) composed of spatial and color information [16]. The kernel function W h s , h r is used for the estimation of the distribution of x. W h s , h r is given by

where A is a normalization constant and h r s is the bandwidth of the kernel function, which control the degree of spatial smoothing and the resolution of color values, respectively.

Assume h _s and h _r represent the pixels of the original image and the image after mean shift smoothing. Then mean shift smoothing algorithm is described as follows.

1. Initialize iterative variable W = 1 and let W h s = A S and W h r = A r .

2. Calculate W h s , W h r by using equation (1) and the mean shift algorithm.

3. Judge whether A _S and A _r are convergent. If it is not convergent, return to step (2). If it is convergent, record the value after convergence. Save the result of mean shift smoothness and assign a value.

The separability of features represents the quality of feature in the target recognition process, which provides a basis for feature configuration. How to determine the separability of features is a problem that needs to be solved. From the minimum error rate of Bayesian decision, the Bhattacharyya distance is an effective measure for class separability. Assume that the feature values of the samples satisfy the normal Gaussian distribution, and the Bhattacharyya distance can be expressed as follows:

where (R _n , C _ps) are the mean and variance of some feature distribution of this class, respectively. In the Bhattacharyya distance equation, the first item is the difference between the mean values of the feature distribution of the class, and the second is the difference within the variable covariance matrix of the feature distribution. From equation (2), it can be seen that when the mean of the two classes is equal, the first part of equation (2) is 0, and when the variance of the two classes is equal, the second part of equation (2) is also 0. Therefore, the smaller the Bhattacharyya distance value, the worse the separability of the feature for the two classes. The larger the Bhattacharyya distance value, the better the separability of the feature [17].

In remote sensing image processing, automatic recognition of important topographical objects is always a hot research topic. Remote sensing images contain rich target information, which can be used to identify specific targets. In the traditional remote sensing recognition method of the topographical object, the algorithm is the center of the whole target recognition process. Specific tasks use specific algorithms, and specific algorithms use specific knowledge. The knowledge of the target is solidified in a specific algorithm program. The form of knowledge is scattered, isolated, and fixed. This traditional method is for specific target recognition task, so the target recognition effect is better. But the flexibility is low, and the knowledge reusability of algorithm is weak. For the better use of knowledge, the algorithm of different tasks and the reusability of related knowledge are needed to be enhanced, the whole target recognition method is considered with knowledge as the center.

If the target information is considered at a higher general level, it will be found that the targets of different classes often share the same or similar information. If we extract the target information and make it more applicable, it will undoubtedly be beneficial to the unified identification of multiple targets. The result obtained with the abstraction and systematization of target information is general knowledge. For example, we analyzed the three classes of targets, such as airport, bridge, and road, and found that their features are closely related to the straight line. Therefore, the knowledge related to straight lines (parallel, intersecting, collinear, etc.) can be used to accomplish these three classes of target recognition methods [18]. As we know, the ability of people to identify objects increases with the increase of knowledge, and the guiding effect of knowledge on human recognition is obvious. Inspired by this, we can consider the centralized storage of knowledge to make it a key module of the remote sensing topographical object recognition system and provide the unified knowledge support for various target recognition tasks. For the expansion of the system, it is no longer just relying on adding new complex algorithm modules, but adding key knowledge and a small number of necessary algorithm modules. This method is called as the knowledge-centric remote sensing topographical object recognition method.

The traditional method and the multitemporal target recognition method are very different in terms of the system structure. The knowledge-centric remote sensing topographical object recognition method is characterized by relatively independent knowledge and specific algorithm. The core of target recognition is knowledge extraction, representation, and reasoning. Knowledge is usually stored in a knowledge base for the unified management and use. This method has high flexibility and strong reusability of knowledge. The function expansion of the target recognition system is realized by adding or modifying knowledge. By better applying knowledge, the processing effect of this method can be improved and perfected continuously. Figure 8 shows the flow of the multitemporal remote sensing target recognition method.

Figure 8

Flow of multitemporal remote sensing target recognition

There are three stages in the process of multitemporal remote sensing target recognition. (1) The original image is obtained for preprocessing. (2) Extraction of image feature points. (3) Identification, analysis, and judgment. In the three stages, the image preprocessing is important, which is directly related to the development of the next two stages. In this article, with the characteristics of TMS320VC5402 (Temperature Measurement Society), we focus on the preprocessing algorithm and digital signal processing (DSP) implementation. It includes extremum filtering, smoothing filtering, Laplasse sharpening, iterative binaryzation of remote sensing technology, and CCS5000 (Integrated development environment) simulation implementation of the algorithm in the DSP development platform CCS2.2.

The purpose of recognition preprocessing is to make the image clearer and the edges more obvious, so as to extract the feature points of the image for recognition. In this article, extremum filtering and improved smoothing filter are used to remove noise and make the image undistorted. Laplasse sharpening is used to enhance the image and highlight the edge information and provide convenience for iterative binaryzation of adaptive threshold [19].

Because the amplitude of a remote sensing image is generally large, a feature vector of the input remote sensing image is calculated directly according to the aforementioned process, which reflects the overall feature of the image. A large remote sensing image may differ greatly in the local texture features. If the feature vector reflects only the overall feature, the local important texture information will be ignored. Therefore, the local texture information is needed to be integrated. Specifically, before recognition, N selected subimages should be selected from the original remote sensing image from the left to right, from the top to the bottom, with a certain position interval. The 17-dimensional feature vectors of each subimage are extracted, and the mean of the feature vectors of these subimages is used as the original input of the total feature vector of the remote sensing image to represent the original input remote sensing image [20].

6 Experimental research

To verify the practical application effect of the proposed recognition method, a comparison with the traditional recognition method is carried out.

6.3 Experimental results and analysis

In this article, the ship image samples and airport image samples are taken as examples, and the support vector machine classifier is used to compare the recognition accuracy. In the process of support vector machine (SVM) classifier recognition, 31 features are selected for classifier learning. Figures 9 and 10 are airport image samples and ship image samples, respectively.

Figure 9

Airport sample image.

Figure 10

Ship sample image.

In this article, the new method and the traditional method are used for multitemporal remote sensing image target recognition of ship and airport samples. The recognition accuracy of the two methods is compared, and the comparison results are shown in Figure 11.

Figure 11

Comparison results of recognition accuracy.

According to Figure 11, the target recognition accuracy of this method is higher than that of the traditional method, up to 90%. Because this method uses the linear feature extraction method to extract the segmented image, including image contour analysis, image preprocessing, image feature analysis, the region of interest location, image enhancement processing, recognition processing, and result output, which improves the accuracy of image recognition. Therefore, this method has some advantages in the shape and texture features of airport recognition.

To further verify the effectiveness of this method, the target recognition time of this method and the traditional method are compared and analyzed, and the comparison results are shown in Figure 12.

Figure 12

Comparison results of recognition time.

According to Figure 12, the target recognition time of the multitemporal remote sensing image in this method is within 25 s, while that of the traditional method is within 70 s. The target recognition time of the multitemporal remote sensing image in this method is shorter than that of the traditional method, because the complexity of the algorithm in this paper is lower, the recognition time is reduced.

7 Conclusions

The automatic target recognition technology has always been a hot spot and difficulty in the field of remote sensing image processing. Therefore, the research of the widely used remote sensing image automatic target recognition technology is of great significance for promoting the real-time and intelligence of remote sensing information processing. In this article, the automatic target recognition method of multitemporal remote sensing images is researched. The research work includes a brief overview of automatic target recognition technology. The image segmentation technology of three kinds of object primitives is researched, which provides a candidate preprocessing method for target image segmentation. The detailed process of three object segmentation technologies is given. An object-oriented framework for remote sensing image segmentation is proposed. From the aspect of target saliency and homogeneity, the configuration of target segmentation methods is researched. The remote sensing target segmentation methods and experimental results of two typical targets of airport and ship are presented. From the three aspects of spectrum, texture, and geometry, the feature extraction method for high-resolution remote sensing target is researched in detail, and a detailed extraction algorithm is given.

Although most of the problems encountered in the automatic target recognition of multitemporal remote sensing image are systematically researched in this article and a configurable segmentation and recognition method is proposed, many difficulties still need to be solved in automatic target recognition. In addition, the proposed algorithm also has many shortcomings, which need further research and solution.