Drones applications for smart cities: Monitoring palm trees and street lights

1.1 Smart city and drone applications

A smart sustainable city is an innovative city that uses information and communication technologies and other means to improve the quality of life, the efficiency of urban operation and services, and competitiveness, while ensuring that it meets the needs of present and future generations concerning economic, social, and environmental aspects. The drone technology was frequently used for military purposes including combat operations either by a human operator or flies in an autonomous manner through the software-driven flight plan [1]. Moreover, with the improvement of innovation and sophistication that occurred over a while, this small equipment has seen widespread applications including commercial and civilian activities such as surveillance, road cracks detection, wildfire management, traffic monitoring, estimation of wind, crisis management, search operations, education, agriculture, remote sensing, rely for ad hoc networks, and civil security [2,3,4,5,6,7,8,9,10,11,12,13]. In recent years, the most emerging area of drone technology promotes the smart city research [14].

The modern era provides better security for communication networks and data; therefore, the combination of sensors with detection and defense mechanism can be utilized to protect drone-based data transmission from cyberattacks, resulting in a secure drone communication method [1]. Drone management systems such as infrastructure monitoring and package delivery services were used for urban developments and smart city transportation [15,16], traffic management [17,18], border patrol mission management [19], wind estimation and airspeed calibration [20], agricultural monitoring for preventing field damage and yielding better crop quality [21], vegetation cover assessment and crop monitoring [22], and rescue operations, surveillance, and emergency responses [23]. Moreover, the safe distance model and the Markov model were proposed to facilitate the drone techniques for transportation management systems in smart cities, where the study by Dung (2019) revealed that the Markov model outperforms the safe distance model [16].

Federal aviation administration expects that 30,000 drones will be flying over the United States in less than 20 years [24]. Therefore, drones in smart cities are expected to have multiple platforms, executing missions at the same time, necessitating a safe and secure environment to support drone operation, and prompting governments to enforce safe security standards and restrict the deployment of inadequate cybersecurity solutions in real-world scenarios. Dung and Rohacs discussed two types of drone-following models: one that focuses on maintaining a safe distance based on the relative velocity (which is based on determining the drone acceleration based on variations in speeds and gaps between the provided drone and its leading drone), and the other that is based on the stochastic diffusion process of speed decision [24]. The simulation results demonstrate that the velocities of the following drones change at almost the same rate as the leading one [24]. Another study examines the real-time performance of monitoring and controlling drones via the Internet using a cloud-based approach [25]. This study primarily uses cloud-based devices and services for processing, storage, and web access, and examines the real-time performance of monitoring and controlling drones via the Internet (4 G D-com Viettel) [25].

A multidimensional framework was presented by ref. [14] for various smart city applications comprising eight core features such as technology, policy, organization, people communities, economy, built infrastructure, natural environment, and governance. These features have been declared important and critical while investigating the context of smart cities. The technological intervention plays a key role in making cities smarter and improving the quality of the living standard. The main avenues for drone applications in smart cities are package delivery, traffic monitoring, policing, drone taxis, ambulance drones, pollution control drones, firefighting, and rescue operations. These applications of drone technology for smart city applications are discussed in detail in the light of past research. Since 2012, Google X has been executing a project involving a drone delivery program called “Project Wing” [26,27]. The “Project Wing” team developed the software with the intent to manage many drones at the same time during their flights and to start an air delivery program by the end of 2020. The UPS (United Parcel Services) conducted a successful auto drone delivery test in the State of Florida from the top of a company electric van [28]. On the other hand, another postal service Dalsey Hillblom Lynn (DHL) exposed their drone delivery pilot program for delivering express and most urgent items [29]. Under the DHL program, the first drone-based automatic deliveries were made to the German Island of Juist in 2014. Continuing their services, DHL made more than 100 successful deliveries in early 2016, by using its Parcelcopter 3.0 drone in the Bavarian Alps. To provide legal cover to their drone delivery program, Amazon launched a mega competition called “Prime Air” by earning a patent from the U.S. Patent and Trademark Office for dropping packages from drones into consumers via parachutes [30,31,32]. A Danish consulting firm, COWI, is the first that used drones for monitoring and analyzing the traffic flow in Denmark.

Kanistras et al. [33] have investigated unmanned aerial vehicle (UAV)-based traffic monitoring and management systems, which have been demonstrated to be a feasible and less time-consuming alternative to real-time traffic monitoring and management. Roads and Transport Authority (RTA) in Dubai, UAE, also planned for deploying drones for monitoring traffic and accidents in Dubai by the end of 2017 [34,35]. As a result, RTA accomplished more than 300 inspections of heavy vehicles and trucks using drones and AI in 2020, reducing inspectors’ exposure to climbing vehicles to assess top areas and verify technical concerns. A French company, Elistair Tethered, deployed a drone called “DataFromSky” traffic analyzer to manage real-time traffic flow and process information about various vehicle types [36].

Drones can also be a very useful device for high-risk police missions including tracing the suspect of various sorts. The United Kingdom has the first exclusive police drone unit called “Devon and Cornwall Police Forces” [37]. This unit started its initial testing in 2015 with Dorset Police, which has now dedicated full-time operators. More than 25% of the 43 forces across England have deployed drone technology for criminal investigation [38]. On the other hand, Chinese Police have around 1,000 drones for providing support in locating opium lands and tracking suspects [39]. A drone was also deployed by the Madison Police in Wisconsin, USA, to locate and track a suicidal person on a loose near the Hawk’s Landing Golf Club [40]. A Chinese drone taxi called Ehang-184 made successful flights in Dubai City during the test runs in 2017 [41]. The smart city environment can be revolutionized through this innovative passenger-carrying drone technology. Dubai City is expected to start offering commercial air taxi services in 2022 [39,41]. Singapore and Las Vegas in Nevada, USA, are expecting to deploy autonomous drone taxis for commercial public use by the year 2030 [42].

Moreover, drones can also play a vital role in saving human life in many instances of critical consideration. The researchers at the Delft University in the Netherlands developed an ambulance drone with the capability of arriving at the target spot in 1 minute after being launched [43,44]. Also, two Russian institutes of high repute, Russian Medical Company Altomedica and Moscow Technological Institute, collaborated to bring forth an advanced ambulance drone, which is also capable of delivering medicines and biomaterials in emergencies [45].

There are several successful drone application examples where the usage of drones made a significant effect. The Chinese Aviation Industry Corporation launched a Para-foil drone to control pollution caused by traffic, house heating, and industry in major city centers [46]. The tech professionals in Hong Kong are developing a “parasitic” drone that will have the capabilities of perching on neon billboards, absorb pollutants through a carbon-absorbent polymer paint to produce fuel, and grow plants [47]. The collaborators of Lamont-Doherty Earth Observatory and Hilpert are working on developing a research platform for a quadcopter drone for measuring pollution from industrial smokestacks [48]. This parasitic drone will be capable of rising to a height of 400 feet (121.92 m) to collect air samples for a follow-up analysis in a laboratory. Drones have also been utilized in firefighting, and search and rescue operations. Dubai’s Emergency Management Organization acquired 15 quadcopters to patrol high-risk locations [49]. The Dubai Civil Defense is also planning to deploy three drones for firefighting and rescue operations. A specialized firefighting drone, Knight Hawk Dutch developed by Dutch Geoborn, contains heat sensors and navigation systems [49]. A drone was used by the Greater Manchester Fire and Rescue service to help firefighters tackle the high flaming blaze in the CWS building near Victoria Railway Station in Manchester City Centre [50]. The New York Firefighting Department also showed an interest in including drones for monitoring dangerous flame and providing assistance in enhancing rescue operations [51].

1.2 Drone applications: palm trees and street lighting

Date palm trees are a major crop throughout the MENA region, accounting for nearly 90% of the worldwide date fruit production [52]. Date palm trees are also utilized in a variety of commercial products (e.g., cosmetics, building materials, and paper), and therefore, surveying them is crucial for predicting production and managing plantations. Traditionally, palm tree mapping was done manually, which resulted in inaccuracies, increased processing time, and required human intervention. Thus, surveying date palm trees is critical for estimating production volumes and managing plantations. Remote sensing, including satellite and UAV, has recently revealed many potential solutions. For example, UAV applications were utilized for smart detection approach for date palm plants, demonstrating an efficient way for agricultural monitoring. The YOLO-V5 model was used as an automated plantation management system in detecting and locating date palm trees of various sizes in congested, overlapping surroundings and sparsely distributed places with a mean average precision of 92.34% [52].

The UAE is one of the leading MENA countries in terms of date palm trees production with 16,757,940 trees in 2016 [53], with over 200 varieties, 68 of which are commercially important [54]. The UAE has reached 100% self-sufficiency in date production [55], exporting 275862.901 tons of dates to 110 countries worth 160215.460 US$ in 2016 [56]. The UAE, on the other hand, faces significant date palm production challenges, including a shortage of qualified and trained personnel in diverse date palm activities, such as harvesting and processing methods, water management, and proper pest and disease research studies [55,57]. Also, the red palm weevil (RPW) is considered a major pest of the date palm in UAE, causing damage to the palm trees [53,57,58,59]. As a result, proper monitoring is crucial for recognizing and enhancing the health of palm trees. Thus, the first purpose of this research is to investigate the use of drones and image processing techniques for monitoring and analyzing healthy date palms from the influence of RPW.

The article titled “A Low-Cost UAV Based Application for Identify and Mapping a Geothermal Feature in Ie Jue Manifestation, Seulawah Volcano, Indonesia” used a DJI Phantom 4 Quadcopter equipped with the FLIR One and MAPIR Survey3 sensors to map the heat signatures of a geothermal feature, and it also stimulated normalized difference vegetation index (NDVI) data from the MAPIR sensor, which was capable of showing vegetation contrast with high resolution [60].

Another study for the Černouček site in the Czech Republic focused on testing and examining the UAV DJI Phantom 4, with the results indicating that the UAV DJI Phantom 4 can be utilized for accurate monitoring applications [61].

Street lighting systems are essential for a smart living since they provide safety and ensure security for drivers, cyclists, and pedestrians in urban areas while also improving the quality of life [62]. According to Loreta et al. and World Health Organization around 1.3 million people die in road crashes each year globally [63,64], which equates to around 3,287 deaths per day [63], making high-quality lighting essential. Moreover, the global number of street lights is estimated to reach 352 million by 2025 [65]; thus, the ability to inspect such infrastructure will be critical. Lighting audits are expensive and time consuming since personnel must manually collect GPS locations and identify the light status. Therefore, developing a technology that will autonomously audit cities’ street lighting infrastructure such as employing a drone-based platform, where data on lighting infrastructure will be retrieved faster and less expensively from collected imagery. Moreover, the street lighting network is one of the most well-known amenities utilized all over the world to ensure social security and safety to all civilians by providing generous lighting conditions in both congested and remote areas at night time. The street lighting network is an essential part of the urban areas because it improves the quality of life by providing comfort, safety, and security for drivers, cyclists, and pedestrians. Thus, monitoring the street light system is fundamental for ensuring that each mechanized street lamp is repaired and maintained to the standard level. Therefore, the second drone application of this study focused on a street lighting monitoring system with image processing.

Overall, this research looks into the potential benefits of utilizing technology in the city to improve the monitoring of palm trees and street lighting in the assigned area. Section 1.1 focused on the utilization of drones and AgiSoft software to identify and assess the health of palm trees using stimulating NDVI analysis models, and Section 1.2 covered the use of drones to monitor the street lighting quality, which are detailed in Section 2.

2.1 Drone application to identify the health of palm trees

In the first application, the UAV captured pictures of the palm trees at the selected site on the UAEU campus in Al Ain, UAE. Then these images are transferred to the central computer system for further analysis. The flowchart presented in Figure 2 shows the stepwise development of orthomosaic models. Traditional methods for palm tree studies largely relay on collecting images and creating an NDVI map. While in this study, the camera was calibrated to detect and rectify lens distortions. Moreover, two scenarios were applied including:

1. First scenario: the wavelength correction comes first, followed by orthomosaic correction.

2. Second scenario: orthomosaic correction comes first, followed by the wavelength correction.

Figure 2

Flowchart methodology for developing orthomosaic models.

It was found that there are pixel anomalies in the second model.

The AgiSoft software is used to stimulate NDVI models that show the health of palm trees. In the process of photosynthesis, palm trees absorb sun radiation as a source of energy. Leaf cells emit more solar radiation in the NIR than in the visible spectral region. Hence, green plants appear relatively bright in the NIR. The pigment in plant leaves, chlorophyll, strongly absorbed visible light from 0.4 to 0.7 µm for use in photosynthesis. The cell structure of the leaves strongly reflects NIR light from 0.7 to 1.1 µm. The more the leaves plants have, the more these wavelengths of light affected.

NDVI = (NIR − VIS)/(NIR + VIS),

VIS and NIR stand for the spectral reflectance measurements acquired in the visible and near-infrared regions, respectively. Drone sensors capturing light in the 600–800 nm range will be the most effective in providing the data for NDVI. The NDVI images are processed in AgiSoft software and analyzed for a range of indicators of crop status and plant disease. The value range of an NDVI is −1 to 1. Table 6 illustrates that NDVI values from −1 to 0 correspond to dead plants; between 0 and 0.3 correspond to areas with sparse vegetation; moderate vegetation tends to vary between 0.3 and 0.6; anything above 0.6 indicates the highest possible density of green leaves [66]. By comparing these NDVI ranges for the selected region, palm model 1 and palm model 2 are calculated and these ranges are compared to normal values as defined in Table 6.

Figures 3 and 4 illustrate the main flowcharts of the stepwise method; all of the steps depicted in these flowcharts have been implemented in this research project.

Figure 3

Flowchart of the methodology for monitoring the palm trees.

Figure 4

Flowchart of methodology for monitoring the street light network in MATLAB.

2.2 Drone application for street light monitoring

As the second application of the study, the drone camera captured images of road lights in UAEU in Al Ain City. These image data are easily transferred to the computer through USB utilizing the pulse width modulation signal and then analyzed in MATLAB software. In MATLAB software, the transferred images are in RGB format, and they are converted into gray-scale images. After that, gray-scale images are converted into binary pictures, and the light sources in the image are displayed conspicuously. Noise is the result of errors in the digital image acquisition process that result in pixel values. To lessen the effect of noise and small light sources in the image, MATLAB filtration techniques such as averaging or Gaussian were used. Then, to allow the comparison of the results and eliminate noise without reducing the clarity of the digital picture, a median filter was applied, which is a special form of order-statistic filtering. In addition, MATLAB predetermined functions are utilized for the conversion of negative images, and bright spots are extracted from the processed images, and their specific location was identified using the Global Positioning System. These processed images are then examined to determine the current status of the street lighting network.

3.1 Palm trees image processing and modeling

The first section of the application involves tracking and analyzing the health of palm trees at a designated site on the UAEU campus in Al Ain city, UAE. The photos acquired by the drone were processed with AgiSoft software to build NDVI models that depict the health of palm trees. The natural breaks classification method was used in the ArcGIS to compute break values for the NDVI map. These are then used to find palm values. The natural breaks classification method is used to identify 10 classes to produce the optimal solution models (Figure 5).

Figure 5

Extraction of palm values’ classes using natural breaks in ArcGIS.

Two NDVI models were extracted and developed to ensure the quality of data. The palm values were calculated using these models, which are reported and presented in Figure 5. Model 1 palms values range from 0.315165441 to 0.540839461, whereas model 2 palms values range from 0.299601716 to 0.48636642. Two NDVI models were extracted and developed based on the statistical computations, which are presented in Figure 6.

Figure 6

NDVI maps were extracted for the palm trees case at UAEU. (a) NDVI Map: model 1 and (b) NDVI Map: model 2.

Two models of NDVI maps are stimulated. While developing model 1, AgiSoft was deployed in stage 1 for orthomosaic and geometry correction. In the second stage, wavelength correction was performed in the Map Camera Control software, which was then followed by computation and the final output of an NDVI map. The development of model 2 started with wavelength correction, followed by the geometry correction using AgiSoft software. The next step was the computation for generating NDVI images, and the final deliverable was an NDVI map (orthomosaic using AgiSoft). As the outputs of the two models are compared, model 1 seems statistically better than model 2. This is due to the fact that the NDVI map generated by model 2 still has a distortion after stitching as shown in Figure 7. Both models were used to develop the reclassification maps for palm trees as shown in Figure 8.

Figure 7

NDVI models – palm trees classification model 1 and model 2.

Figure 8

Reclassification map for model 1 and model 2.

Table 7 illustrates the simulated NDVI ranges for the selected green site at UAEU campus, palm tree model 1 and palm model 2 are calculated as 0.315165441–0.540839461, and 0.299601716–0.48636642, respectively. These values belong to the NDVI range of 0.4–0.6, indicating that the selected area has a moderate vegetation status and that date palm trees do not have any major health issues. Future research is needed to validate the generalization of the study’s findings.

3.2 Street light image processing

In the second application, the drone camera was used to capture street light images. These digital images were then transferred to a central computer, which processed the model and determined if the street light was in good or terrible functioning order. Figure 9 shows real-world images of UAEU streetlights that were imported to MATLAB software, where the RGB picture was converted to a gray-scale image, and then filtering was applied to the gray-scale image to decrease the noise level. The gray-scale image was then transformed into a binary image, which allowed light sources to be displayed more clearly, as shown in Figure 10.

Figure 9

UAEU street light pictures received from the drone.

Figure 10

Input image is converted into gray-scale and binary image.

The next step produces a negative image using MATLAB predetermined functions. The result of this operation in an image matrix is assigned a binary 1 and binary 0 to differentiate the bright spots for selected street lighting processed images, as shown in Figure 11. This study also presents a regular monitoring method of the street lighting network by using the drone technology and enhances the image processing in MATLAB software. Table 8 depicts that the present status of lights is OFF in G image. Therefore, the location of this image was determined using ArcGIS Pro software as shown in Figure 12 to be the UAEU female campus, Al Ain, x:55.679472, y:24.196573.

Figure 11

The filtered and processed street light image in MATLAB software.

Figure 12

The location of OFF condition light in ArcGIS Pro software.

The photos taken by the drone camera were sent to the central computer, where they were processed to generate the black spots that indicate that the light is turned on. Moreover, it helps to identify the present status of the street lighting system in Al Ain Street and to detect the location of each street lamp from the inbuilt Global Positioning System enabled by drone technology. Therefore, the present status of the street lighting system can be identified by using an effective scientific monitoring system.

The study’s major limitations are based on the risk associated with the use of drones, which include limited drone flying time, drone falls from a great height due to drone damage caused by battery complaints and weather condition (low air temperature, precipitation), and crashing with obstacles such as trees or buildings. Moreover, legal approvals for drone flying over the selected areas are very time-consuming process. Emerging/changing technology that must be integrated with the current drone to avoid errors. Also, some of the study challenges include locating farms with a variety of tree heights, comparing the pixel distortions requires going back to RGB aerial photos, adding basic information such as altitude and coordinates to the captured images, and the impact of high temperatures on the iPhone devices on which the applications were installed.

In the next phases of this project, data with high accuracy can be explored, and the reliability of drone technology can be improved by using hyperspectral sensors. In future investigations, the use of thermal drones and hyperspectral sensors in the case of streetlight monitoring will give high accuracy data, and the road security system can be improved with proper management techniques. Future research on palm and street light analysis could include remote sensing and geographic information system mapping and visualization. Furthermore, the future research might concentrate on developing a web-based application that can provide real-time updates on the status of agricultural farm and street light networks.