The simulation approach to the interpretation of archival aerial photographs

1 Introduction

Aerial photographs obtained in the last century are a valuable source of information on the Earth’s surface. Their high value is attributed to the fact that frequently these are the sole materials with a high degree of detail which contain credible information on terrain coverage and use. Because of this, archival aerial photographs are frequently used in scientific research projects around the world. Numerous scientific papers have been published where, based on archival aerial photographs, problems have been raised concerning, among other things, landscape and climate changes, anthropogenic impact upon such changes, as well as the restoration of the environment at given moment in time [1, 2, 3, 4, 5, 6, 7, 8, 9, 10].

In this article, the authors pay special attention to the important role of archival aerial photographs in the process of modernisation of the register of land plots and buildings. In the Republic of Poland, the boundaries of over 23% of cadastral districts have been set out based on photomaps [11, 12]. The application of technology based on a 1:5000 scale photomap resulted in the very low precision of 3.0 m of setting out the coordinates of boundary points of registered land plots. Such precision is insufficient to meet the requirements of regulations presently in place [13, 14]. Because of this, it has become necessary to develop a technology for the re-determination of location of land plot boundary points, ensuring the required precision of setting out the coordinates of such points. That would not require exorbitant financial outlays as in the case of full upgrade of the land and buildings register. One must note here that the majority of field details once utilised as a horizontal grid when establishing the land and buildings register are no longer in existence, land plot boundaries have not been stabilized with landmarks, and decades of agricultural activities may have caused the boundaries to be moved or even erased. Therefore, the sole appropriate solution is to utilise the archival aerial photographs with simultaneous application of sophisticated digital photogrammetry methods [15]. The research carried out within the scope referred to above confirmed the possibility of setting out the initial and final survey lines with sufficient precision to allow the utilisation of data included in the aerial mosaics, upon which the register had been established. The precision of setting out the co-ordinates of survey line footholds depends upon the photogrammetric measurement method adopted and upon Poland the geometric, spectral and radiometric quality of scanned aerial photographs. In the case of manual measurements on stereo models of aerial photographs, such precision is associated with the precision of the spatial orientation of photographs, and hence with the precision of tie points measurements. Furthermore, the precision of correlative measurement of tie points depends upon the spectral and radiometric quality of archival aerial photographs. The better the quality of imagery, the higher is the precision of tie point measurements.

Unfortunately, the monochromatic nature of archival aerial photographs makes their interpretation considerably more difficult. That bears a negative impact upon both qualitative and quantitative analysis of photographed objects, as well as upon photogrammetric measurements. Colour imagery data with the same spatial resolution as monochromatic data provide a higher number of parameters characterising photographed objects, thus allowing a more credible interpretation thereof.

Given the need to improve the spectral quality of such data, the authors developed the simulation methodology of new spectral bands with spatial resolution equal to that of archival aerial photographs [16, 17, 18, 19, 20]. The developed methodology concerns the process of adding the spectral information to archival aerial images based on the multispectral bands simulation from archival satellite imagery using pan-sharpening method. The applied methodology [17] was used in this research and was presented in Figure 2. Simulation is a process that enables the generation of new spectral bands based on imaging data from other sources available for the area under analysis. Bands simulated with the application of this method contain information on actual spectral characteristics of the objects photographed. Simulation allows the reconstruction of spectral information for data with spectral resolution insufficient for the performance of certain research [21, 22, 23, 24, 25, 26, 27]. This is of utmost importance concerning archival aerial photographs [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 28, 29, 30], for which no multispectral data has been recorded.

The objective of this research is a preliminary assessment of the interpretative capacity of colour images acquired as a result of simulation of new spectral bands with high spatial resolution for archival aerial photographs. In itself, the simulation process of new spectral bands for archival aerial photographs represents an innovative approach. To date, research efforts focused mainly on simulating new spectral bands for contemporary digital data, acquired with the application of sensors with known parameters. The application of simulated contemporary data bands refers mainly to the possibility of foreseeing technical problems prior to building a sensor, increasing the data transmission rate between satellite and a ground-level receiver [31], or reducing current costs of environmental research while simultaneously acquiring as much information as possible on the objects under research [23, 25, 32, 33]. No research was found about the aspect of adding spectral information to archival aerial images. The problem of archival data processing was discussed in the literature and concerns enhancing the radiometric quality of these data [34, 35, 36]. The authors proved that the simulation of spectral channels works not only for current image data but also for archival aerial images. The authors, additionally, propose a new application for simulated spectral bands that may significantly influence the upgrade process of the land and buildings register based on archival data. In the process of photographic interpretation of arable and afforested land and analysis of the boundaries and utilisation thereof, the application of simulated spectral bands for archival aerial photographs may contribute to a considerable improvement in the precision of setting out the coordinates of boundary points of land plot boundaries. The application of the simulation process allows the generation of images in higher spectral quality as compared to the original aerial photograph while preserving its degree of detail. That means that the enhanced image with simulated spectral bands is a resource with an increased number of parameters characterising a given object, and hence differentiating it against the background. It also contributes to considerable improvement of photo-interpretation capacity, relating mainly to the aspect of upgrade of the land and buildings register and to set out the locations of starting and ending points of survey lines. This article presents the results of preliminary research into the impact of simulated, new spectral bands upon the interpretation capacity of photographed arable and afforested land. In the paper, the authors attach particular attention to the possibilities of detecting individual objects, as well as to the identification and differentiation of such objects within the boundaries of the same type of land cover or land use.

2 Data and Methodology

The research was carried out based on contemporary data: panchromatic and multispectral aerial photography and multispectral satellite imagery. Since no aerial multispectral material has been physically obtained for the archival aerial data for which the simulation methodology was proposed, it has been necessary to base the research on contemporary data to compare the enhanced imagery with the original multispectral aerial material.

Panchromatic and multispectral aerial photographs at 0.30 m spatial resolution, provided by MGGP Aero, have been used as well as multispectral satellite imagery at 10 m spatial resolution, acquired from the Sentinel-2 satellite (data downloaded free-of-charge from https://scihub.copernicus.eu) (see Figure 1). The selection of data for tests was aimed at the acquisition of images with possible similar registration times and lack of cloud cover. Aerial photographs were obtained with DMC II 230 camera on 20 August 2016. Satellite imagery was acquired on 28 August 2016. The research area includes an area of Michalowo municipality, situated in the Podlaskie Voivodeship in the Republic of Poland. The images present urban and rural areas, characterised with diverse forms of coverage and use, including but not limited to, forests, meadows, cultivated fields, uncovered soil and anthropogenic facilities. The research covers an area where no changes were obtained during the period from 20 August 2016 to 28 August 2016.

Figure 1

Aerial photographs (a,b) and satellite imagery (c) in natural colour composition.

Figure 2

Flowchart of data processing and analysis procedures.

The research methodology is presented in the form of a flowchart (see Figure 2). Aerial and satellite imagery were subjected to geometric correction to reduce errors associated with the differences in the geometry of such data. The data were also subjected to orthorectification, inclusive of the georeferencing process (UTM/WGS84) and then co-registration. For the orthorectification of aerial images, 12 control and ten checkpoints were used, while for Sentinel imagery, a digital elevation model (DEM) obtained from shuttle radar topography mission (SRTM) and a satellite geometric model was applied. Next, the satellite image was subjected to the process of radiometric correction based on producer’s data, as well as to atmospheric correction [37]. For process input data, algorithms implemented in ERDAS IMAGINE and ENVI programs were used.

In subsequent stages, aerial and satellite data were fused using the pan-sharpening method. In previous tests, the Brovey method was applied. The Brovey method allowed the achievement of higher spatial and spectral quality parameters as compared to other pan-sharpening methods [17]. Panchromatic aerial photographs represent high spatial resolution data (0.30 m, one band only), whereas multispectral satellite imagery represents high spectral resolution (10 m, several bands). Landsat-5 satellite data may be utilised solely as multispectral material for archival aerial photographs. Unfortunately, no data from the Landsat platform is available for the area under research within the time limit in proximity to the registration period of contemporary aerial photographs, hence data from the Sentinel-2 platform was used in the research. In subsequent stages, visual analysis of enhanced images was conducted in terms of detection and identification of individual photographed objects. For this purpose, the sharpened image was compared to the original multispectral aerial image. The degree of colour preservation was analysed for individual objects in relation to their surroundings. The presence of artefacts and the degree of preservation of details were also analysed.

Visual assessment was supported by the application of the overall accuracy index (oa) (formula 1), error of commission (ec) (formula 2), and error of omission (eo) (formula 3) [18], frequently utilized in the assessment of images following classification [38, 39, 40, 41].

The parameter p_ii refers to the number of correctly assigned pixels, whereas p_ij means the number of samples (pixels) analysed.

The error of commission (Formula 2) is calculated based on the value of lines in the error matrix. The error consists of the ratio of the number of pixels incorrectly classified to the number of all pixels. The error of omission (Formula 3) is calculated based on the value of columns of error matrix and represents the pixels that, according to reference data, should be assigned to a given class but have been erroneously assigned to other classes.

The indices were calculated for each land cover class analysed, such as forests, meadows, mowed meadows, stubble, maize and ploughed fields. Samples for evaluation of simulation results were distributed in areas where no changes were observed due to the time difference in obtaining aerial and satellite data. Reference samples assigned to a specific type of land cover were selected based on the original multispectral aerial image. Next, based on the visual interpretation of the original aerial panchromatic image and the enhanced image, it was verified if these sample-pixels could be assigned to the same classes as the reference pixels. There is no one-size-fits-all way to arrange samples to assess classification results [41]. Concerning article [42], the observer can arrange the samples depending on the type of input data. In this work, samples were selected in places with the expected low accuracy, i.e. close to the borders of different classes, because in such places the most significant spectral distortions occur due to the vast difference in the spatial resolution of the input data. In order to ensure the independence of the evaluation results, the measurements were made twelve times by each of the ten observers with no visual defects.

3 Results

The result of the processing of panchromatic aerial and satellite data was enhanced, 3-band images with a spatial resolution of 0.30 m (see Figure 3 (c)).

Figure 3

Panchromatic aerial photograph (a), satellite imagery in natural colours (b), enhanced image (c) and aerial photograph in natural colours (d).

The resultant enhanced images are colour images which also contain spectral information concerning the land cover elements photographed. In detailed environmental research, enhanced images serving as the source of reliable spectral characteristics of photographed objects provide a considerable advantage over colour images produced using a traditional colouring process. The method applied here allows the preservation of the high degree of detail of original panchromatic aerial photographs, thus bearing a positive impact upon the identification process of arable and afforested land (see Figure 4).

Figure 4

Comparison of agricultural land on a panchromatic aerial photograph (a, c) and enhanced images acquired from simulation of new spectral bands (b, d).

The distortion of colours is clearly visible as compared to the original multispectral images. However, according to the literature [23], images closely resembling natural colouring are not always the best source of the spectral information of photographed objects.

A comparative analysis of panchromatic and enhanced images was performed. For each of six classes, a minimum of 500 samples (pixels) were selected [43], located in sites marked with expected low precision of the fusion process – most of the samples were selected near the boundaries of different types of land cover (see Figure 5). Samples assigned to a given class based on visual assessment of aerial multispectral imagery were checked for their capacity to be assigned to the same class based on the interpretation of the panchromatic image and sharpened image. Measurements made by all observers were averaged and presented in Table 1.

Figure 5

Fragment of image with distribution of samples participating in precision assessment of the pan-sharpening process: original multispectral aerial photograph (a), panchromatic image (b), enhanced image (c) (red colour was used to mark meadows, orange – maize, yellow – mowed meadows, pink – forests, green – stubble, blue – ploughed fields).

Analysis of values displayed in Table 1 indicated increased values of the oa index, calculated based on enhanced images in relation to the original monochromaticimages. Higher values were attained for each terrain coverage class analysed. The study also indicated a reduction in the values of commission and omission errors for each class. For enhanced data, the overall classification precision stood at 71%, whereas for monochromatic data it was 66%. Different values of parameters were acquired depending upon the type of land cover. The highest value of oa index was acquired for mowed meadows (83% for enhanced data), and the lowest for ploughed fields (58% for enhanced data). The highest values of errors were obtained for stubble and ploughed field classes.

In the case of integration of archival aerial photographs with archival satellite imagery (see Figure 6) one must take into consideration the fact that the precision of this process, and hence its photo interpretation capacity, may be lower as compared to contemporary data [17]. Firstly, it may be caused by more significant spatial differences between input data. Spatial resolution for Landsat-5 (30 m) data is one-third of that for Sentinel-2 (10 m) data. Secondly, simulation results may be influenced by the lack of available parameters of aerial camera and photosensitive material, as well as the ageing process of the latter [39, 40].

Figure 6

Two sets of archival data from 1987: “black and white” aerial images (a,d), multispectral Landsat-5 (b,e) images, images enhanced by simulation method (c,f) [17].

4 Summary and Conclusions

In this paper, the authors conducted a preliminary analysis of images simulated by the integration of aerial photographs with high spatial resolution with satellite imagery with high spectral resolution. Based on visual assessment, the authors concluded that satellite images are highly useful in the context of simulating new spectral bands for aerial photos.

Visual analysis indicated increased interpretation capacities of simulated imagery as compared to panchromatic imagery. Colour images may be better interpreted by the human eye than monochromatic images. Taking into consideration all samples used in the analysis, the oa index was found to increase by 5%. The precision of the fusion process was noted to depend upon the type of objects falling within the scope of interest.

Simulated high-spatial bands will make it possible to improve the spectral quality of archival aerial photographs, and hence to improve the interpretation capacity thereof, based not only on the colours acquired but also on extremely significant, reliable spectral characteristics of objects. Despite colour distortion, the colour images acquired from the simulation renders it possible to more effectively interpret the boundaries of arable and afforested land, the detection of which on “black and white” photographs is considerably more difficult.

Limitations: The basic limitation of this method is the availability of satellite imagery registered during a similar period and for the same area as the archival aerial photographs. It must also be taken into consideration that the precision of the simulation process for archival data will probably be lower as compared to contemporary data, on which the research was carried out. Reduced interpretative capabilities shall bear an impact mainly upon lower resolution of input data (spatial, spectral, radiometric), the process of scanning aerial photographs, as well as lack of information on the parameters of an aerial camera and photosensitive material.

Further research questions: Research into the impact of new, simulated bands with the high spatial resolution for archival aerial photographs upon the interpretation of such data shall be expanded by adding new sets of aerial and satellite data. The impact of spatial, spectral and radiometric resolution of input data upon the simulation process will be investigated. Additional terrain coverage classes will be included in the assessment of interpretative capabilities. It is assumed that the accuracy of the photo interpretation process will increase through the use of ground truth as reference data instead of pixels of multispectral aerial images. Therefore further analysis will be carried out based on in-situ data.