Path reader and intelligent lane navigator by autonomous vehicle

1 Introduction

Globally, nearly 3,287 people die daily due to car accidents. One of the major reasons behind this is the driver sleeping in the car or trying to stop the car when it is at a very high speed. Just as the industrial revolution freed humanity from physical drudgery, artificial intelligence (AI) has the potential to free humans from mental drudgery.

To reduce the number of accidents that occur on a daily basis, it is critical to reduce the amount of human error; it will be extremely fascinating if all we have to do is fit our destination into our schedule and keep working until we reach our goal without making any mental or physical mistakes.

The use of a self-driving car can not only prevent accidents, but also provide self-relief for minor daily activities.

The Internet of Things (IoT) is a network of standard items such as motorized vehicles, the Internet, televisions, and other contraptions that are specifically linked together, enabling new types of correspondence between things and people as well as between things themselves. Building the IoT has advanced in recent years, adding another estimation to the universe of data and correspondence movement estimations.

Home automation or Smart Homes can be portrayed as presentation of improvement inside the home condition to give settlement, solace, security, and essential capacity to its inhabitants. In addition, figuring out how to improve home condition can give broadened singular satisfaction. With the presentation of the IoT, the examination and use of home mechanization are getting progressively standard.

Self-driving cars can help persons who are unable to drive on their own due to infirmities such as blindness. According to studies, a driver’s mistake is cited as a reason in 94% of crashes, and self-driving vehicles can help eliminate driver error. Because it does not require rest like people and can operate constantly for hours, it can increase traffic congestion, save fuel, and reduce greenhouse gas emissions.

Reduced travel time: Travel by a car should be safe whether the car is going slowly or rapidly. Higher speeds are likely to be possible, because computers will eliminate human error as a cause of accidents. Less expensive insurance: If car insurance companies join the car movement, your rates could decrease significantly. Risk allocation depends more on the vehicle than the driver, so you can expect insurance premiums to go down. Redirecting our emergency services’ efforts and resources will allow us to redirect our emergency services to where they are needed.

In order to further approach with the process, the structural diagram will show the entire basic procedure of the progress of the study.

The Figure 1 contains the basic flow information where the data are gathered from the environment and then these data are further processed and resulted in the different maps of path observer, then the suitable path is chosen, which taking the consideration of the lane tracking and then the control is there while moving in the lane through reading of the environment.

Figure 1

Flow diagram of the proposed process.

2 Literature review

We can better examine and grasp new research projects if we have a broader view of the algorithms that can be deployed alongside existing ones.

How is technology helping transform the world? There have been great changes in the technology of self-driving and automatic cars since the 1920s when the first ever radio-controlled vehicle/car was introduced. In the years following to this, many automatic electric cars were seen on the roads that were powered by embedded circuits, and by 1960 automatic cars which had same electronic guiding system came in the picture. In the 1980s, vision-guided autonomous vehicles (AVs), which were a great achievement of the technology at that time, were introduced. The similar or slightly modified forms of this technology are still being used today [1].

We can increase people’s trust in driverless cars. Although public trust is crucial to widespread adoption, this is the main obstacle. The goal of this study is to determine which variables are most important in increasing the use of driverless cars. The study found that a vehicle’s ability to meet performance expectations and its reliability were important adoption determinants, according to quantitative research. The major concerns that were raised had to do with privacy, such as location, security, and the like [2].

Robotic platforms allow for incremental development of manual processes, and path planning has become a critical area even if the environment inside and outside the building is unknown. Our challenge is to invent ways to make the algorithms as intelligent or pre-established as possible, and to arrive at our destination in the most efficient manner. One of the most significant issues in this field is finding a path that is free of static and dynamic obstacles. In this article, a methodology is proposed to cover the critical points and reach the initial key point in a dynamic environment with the implementation details of the robotic platform. The primary computation is taking place inside the Raspberry Pi B + module, and other modules include compass, wheel encoders, and ultrasonic sensors [3].

Convolution neural networks (CNNs) are used to create a self-driving automobile based on monocular vision. The authors of this research sought to develop a method for modeling raw input photos to a specific steering angle predicted by the CNN. CNN was trained using data acquired from the vehicular road/platform Raspberry Pi 3 and a front connected camera, as well as photos of the road and time-synchronized steering wheel angle gained through manual driving. Whether road markers were present or not, the speed reached was 5–8 km/h [4].

CNN maps raw images that have been taken from a front attached camera on the vehicle directly to the steering wheel. This system works very well on traffic-filled roads with or without markings on the road. The only training data provided were human steering angles which were then used to predict the particular angle at which the car should be steered. This system has smaller networks and better performance as minimal number of processing steps are required and this performs better as components can self-optimize [5].

Efficient and extremely compact CNNs were generated in their study, which makes use of a novel sparse connection topology. Because of the sparseness of inter layer filter dependencies, this results in a significant reduction in processing power consumption as well as a reduction in the number of parameters required, without sacrificing CNN accuracy. The article’s findings indicated that the system’s accuracy was greater than that of CNN’s cutting-edge architecture. When compared to previous models, the model required 40% less parameters and was 31% faster on the CPU, while preserving greater or similar efficiency [6].

By comparing different models of CNN while implementing them on a self-driving car, they test which model is the best and proves to be the most efficient in a simulated environment. The CNN has been trained by the manually obtained data by driving a car and using previously obtained data from end-to-end deep learning techniques. When training is done, the CNN is tested in the driving simulator by checking its ability to reduce the distance traveled by the car to go to the center, heading error, and root mean square error. The conclusion drawn was that adding long-short term memory layers in CNN produced better steering of the car which took into account the previously predicted value by CNN and not just the new predicted value or a single instance [7].

Level 2 automatic cars are implemented by the authors by taking the inputs from the front-facing camera on the vehicle and feeding them as steering inputs. The network requires minimal human intervention as maximum variable features are learnt from the camera inputs themselves. The data set used is from NVidia and Udacity, and when the CNN is given real inputs it can adapt to real environment driving given a controlled environment. The setup consists of an ultrasonic sensor that will detect obstacles and an red green blue depth camera working at 10 HZ which outputs a steering angle [8].

OShea and Nash [9] have described the various Artificial Neural Networks (ANNs) and their types, most significantly CNN. CNNs are mostly used to solve difficult image-driven tasks that require pattern recognition. These have precise and simple architecture and are easy to implement; this study gave great insight into ANN and especially CNN.

Unlike typical cars [10], self-driving cars can park anywhere. Instead, they can drive, fly, or cruise (circle around). Vehicles are enticed to work together to clog roads. According to San Francisco’s downtown data, self-driving cars might roughly treble the number of vehicles entering, leaving, and inside cities. Planned travels extend due to parking and cruising. Parking subsidies may have the unintended consequence of worsening congestion. According to the study’s conclusions, the introduction of congestion pricing in cities in the near future will be heavily reliant on AVs. Congestion pricing should incorporate a time-based penalty as well as a distance- or energy-based fee to internalize various externalities associated with driving.

Vehicle speed, eye-gazing, and hand gestures [11] all reveal a driver’s purpose and attentiveness. The appearance and behavior of a car indicate to passengers whether the driver is likely to pay attention to the road. This research aims to enable passengers to comprehend and express their autonomous car awareness and intent to pedestrians, which might be difficult if explicit interfaces are avoided. The idea of an AV’s mission and awareness to pedestrians was conducted. Four user interface prototypes were designed and tested on Segway and cars. It is possible to taste, touch, smell, and hear things out in the environment and combine the senses to do so.

Deep learning-based vehicle [12] control systems are becoming more common. Before building a vehicle controller, engineers must rigorously test it under various driving conditions. Recent improvements in deep learning algorithms promise to solve challenging non-linear control problems and transfer knowledge from earlier events to new situations. These significant advances have gotten little attention. This study uncovers current and valuable information on intelligent transportation systems, which is vital for the field’s future. Control and perception are interwoven in this research.

Modern autonomous [13] driving systems rely on historical mapping. Although prevalent in cities, precise maps are difficult to develop, preserve, and transmit. Rural areas have high turnover, making exact mapping challenging. A self-driving automobile was tested in the countryside to ensure its functionality. The car uses its local sensing system to detect its road conditions. This system calculates a car’s distance and speeds through recursive residual filtering and odometer, allowing it to navigate complex road networks easily.

This AI product features [14] should assist in minimizing traffic congestion, road accidents, and social exclusion. Future human transportation will have AI-powered drivers. Despite its apparent benefits, people are still wary about driverless cars. People’s trust in machines may help build autonomous systems. This study assesses the acceptance of autonomous technologies. That is, future studies should examine user trust and approval. Changes to the roadway and subsurface infrastructure impacted traffic, community attitudes and concerns, potential transferable behaviors and requests, other business models, and strategy. Malaysian law enforcement agencies must identify critical elements to investigate AV manipulators’ conspiracy claims appropriately.

A family of nonlinear [15] under-actuated systems was found to be soluble. The vehicle’s lateral dynamic control system incorporates the usage of forwarding and backward controls. Even if the findings of theoretical studies on AV lateral control can be applied to multiple circumstances, the results can still be used in other applications.

In the study, the performance of the closed-loop system was compared to that of a typical human driver.

AVs will completely [16] revolutionize ground transportation. In the future, new cars that can judge and drive themselves are expected to replace traditional cars. Sensors help self-driving cars sense and comprehend their surroundings, whereas 5G allows them to sense and comprehend distant environments. Local perception, like human perception, can be helpful for short-range vehicle control. Despite the fact that people’s perspectives have broadened, they can still prepare for the future and drive with greater caution while adhering to a set of norms (safety, energy management, traffic optimization, comfort). Faults can emerge as a result of background noise, ambient circumstances, or manufacturing problems, regardless of how well an electronic sensor has previously worked. The most practical solution to the shortcomings of individual sensors is for them to be integrated. The goal of this research is to talk about performance optimization for local automated driving systems in automobiles.

Table 1 contains the multifunctionality of the AV and the strategic issues and challenges which can really be taken as the premium objective to work in this article are as follows:

1. An AV with a specific algorithm should contain more amount of parameters for validation and testing.

2. The involvement of the Deep Learning and AI model can be enriched more to make the system more agile and updated.

3. It should contain more fusion-based approaches for innovative vehicle systems.

4. Effectiveness and accuracy in the fusion approaches can be increased.

3 Problem formulation

As mentioned in the literature and through the different challenges we received from the survey, to do the automatic navigation, we need to have specific road measures. These measures are kept in consideration; one by one we discussed the standards and their factor which could affect the data processing. The following observation should be taken as reference.

1. We need to verify all the road conditions and the feasibility of the data receiving and processing through the system.

2. Then, by considering all the parameters, we need to design the mode of solution.

   Some factors that we will consider in optimization of fuel-efficient routes are as follows:

3. International Roughness Index (IRI) is pavement roughness. The roughness parameter in Table 2 is calculated by the vertical oscillations of the vehicle chassis per road section (generally 100 m). Its unit is mm/m. This table describes the roughness factor for the different surface parameter where it ranges up to 5.6 where it contains the different characteristics and satisfies all the conditions.

Taking the shortest route and avoiding (Table 3) congested overcrowded paths should be followed since they might result in increased fuel consumption. Congestion is graded on a scale of 1 to 5. In this table, there is a congestion factor for several conditions of measurable factor that contain distinct features; if the ranges vary within the range, the choice factor for choosing the road can be deviated or chosen.

We can save money by taking a route with less traffic lights (Table 4), as opposed to spending a few minutes standing in line to change signals and wasting fuel. This can be classified on a scale of 1–5. These are the most important factors to consider while determining the best driving parameters for safety. It also defines the conditions that are beneficial for moving from one node to another by optimizing and selecting the best conditions for the journey.

The path has fewer potholes. Potholes on the road can be categorized on a scale of 1 to 5 (Table 5). Potholes are also a key element in determining whether or not to travel on a road, and these conditions may be taken to avoid catastrophic accidents and to offer a smooth driving experience for visitors who travel on that route. Potholes have distinct circumstances in the road where it disrupts the smooth driving element in the road and these are also different factors which are required for determining the road conditions and evaluating the distance.

3.1 Designing and development

First, calculate the optimized distance by consulting the effecting pavment condition (EPV) from equation 1. These factors helps to analyze the proper navigation. We will convert the required map into a graph where each place on potholes, the map will be depicted as the node on the graph. This helps to detect the potholes in accurate manner.

A car has to enter the area or the place where it wants to start the journey and set the destination area. If the car has to go from one place to another, then all the routes possible according to the map are depicted in the form of Figure 2 as shown. What is being added with each distance here is the factor effecting pavment condition (FEPV) value that will help us find the best fuel-efficient path in the final route.

(1) Fepv = D p + ( Pr + Cf + TL + Rf ) / 4 D p = Distance, P r = Pavement roughness, C f = Congestion factor, T l = Traffice light factor, R f = Road factor .

Figure 2

These two figures contain the frame for the region of interest and actual region of interest calculated.

After this we have to design an algorithm for the aforementioned problem, and this project also aims to choose the best optimized algorithm which will be used to find the shortest path between two points entered by the user (Table 6).

Table 6

Finding out the navigation path through the proposed approach

Algorithm	Finding out the navigation path through the proposed approach
Input: Enter the Distance Nodes values
Output: Path Navigator and observed value
Begin
If nij = 0 if i = j Dij = 0 length (ni,nj) Cij = 0 otherwise NULL
for K = 0 to A-1
for J = 0 to A-1
nij(K + 1) = min(nij(k)),Epv(nij(k) + nij(k) + dij(k)
End
for
End
for
End

4 Methodology

First, we need to do the setup of the car as shown in Figure 2 with the required hardware requirement. Take all the four motors and connect jumping wires with them. Since our H bridge can only handle only two motors at a time, connect two motors with each other at one time. Assemble all the motors in the plastic plates. Remember to cross-couple to let the motors move in the same direction. open source computer vision library stores the image in the form of the BGR color format; however, we need to change it to RGB color format which is important to adjust the settings of the view we have. We will use a setup function for our camera to stabilize, then we will take region of interest around these four corners.

We will take a sample, region of interest as shown in Figure 3. For this we will define the region we want identify for lane to get focused by camera to move car in forward direction. In the implementation process, we will convert our RGB image into gray scale to get the clear vision through camera.

Figure 3

Calculation of the right and left position of the lane and the gray scale image.

We define the threshold manually, initially setting a specific value and creating a histogram for all values above this threshold. These are converted into white pixels, while all remaining values become black pixels. The next step, involves identifying all edges and corners within the lane using the Canny edge detection technique. This process facilitates easier object identification for our autonomous car.

Prior to image processing, we convert our RGB image in Figure 4 into gray-scale for easier manipulation. We define the threshold manually by setting a specific value. Those greater than the threshold are turned into white pixels, while all remaining values are turned into black pixels.

Figure 4

Calculation of the center of the lane and calibrating lane center with frame center.

The next step is finding all the edges and corners coming in the lane so that it can help our car identify objects easily, with the help of canny edge detection. Canny edge detection basically detects the sudden change in the image gradient. For getting canny edges, we will apply sobel operator on our threshold image.

In sobel operator, suppose Gx is an image pixel where each pixel contains the horizontal derivative and Gy is an image pixel where each pixel contains the vertical derivative, then G = sqrt(Gx ² + Gy ²) where G represents the image gradient. Then we will find the exact position of the lane, i.e., right position and the left position. The next step is finding the left position and the right position of our lane where our autonomous car will move in during its journey.

Green lines depict the lane finder. In the next step, we will find the lane center using the left lane position and the right lane position. The blue color in Figure 5 shows the lane center.

In the next step, we will calibrate our lane center with frame center (Figure 5). The green line depicts the lane center and blue line depicts the frame center; we will shift frame center towards the left so that it can calibrate with lane center. In the next step, we will move our autonomous car in different directions and check the difference between lane center and frame center as a result.

Figure 5

Difference between lane center and frame center.

In the following stage, we will use CNNs (Figure 6). CNNs are a type of neural network that have proven to be particularly effective in picture recognition and categorization. The CNN design categorizes and is primarily utilized for character recognition jobs, such as Classification, Convolution, Filters, Non-Linearity (ReLU) Activation Function, Pooling, or Sub Sampling (Fully Connected Layer).

Figure 6

CNN model.

Since we are dealing with CNN, we will deal here with conv2D, MaxPool2D layer which is present in Keras, and after we have imported the sequential model we will first add some convolutions layer. We will first add convolution layer with 32 filters for the first two convolution layers having 5x5 kernel matrix filter, which can be involved on original image for extracting the important feature from image.

The kernel matrix is applied on complete image matrix. We have now incorporated a down-sampling filter, specifically Max2D, which reduces the image’s dimensions. This process effectively shrinks the size of the image, simplifying further manipulation and analysis.

Next, we must decide upon the pooling size. It is critical to select the pooling dimension as well. Also, we are using convolution and pooling in this layer to allow our model to learn more information.

Next, we will add two more convolution layers, with 64 filters and down sampling, at the conclusion. To get things started, after we have finished pooling, we will go on to the dropout layer, which is a regularization approach that randomly sets the weights of a section of the nodes in the layer to zero. The last step involves dealing with the feature that causes certain nodes to randomly disconnect from the network. This necessitates the remaining network to reach a distributed solution.

When it comes to increasing generalization and controlling over fitting, this strategy works well. (ReLu) is an abbreviation for maximal activation function (0,x). The rectifier activation function is used to introduce non-linearity into the system.

The flatten layer is used to convert the final feature mappings into a single 1D vector representation. It will be necessary to flatten the layers once they have been convolution and max pooled in order to use completely connected layers. Essentially, it combines into the convolution layer all of the previously trained local properties.

As an alternative to digging deeper, we built an ANN classifier based on the properties of the previous layer. The final layer produces a distribution of the likelihood of each class, which is displayed on the screen.

5 Result and analysis

The process has been thoroughly evaluated the crucial aspects of the path by testing various parameters. While updating the results, these parameters were analyzed in relation to the condition of the pavement.

Table 7 contains the optimal path efficiency considering the EPV factor in the various dimension of pavement level, and this table contains the level which defines the five constraints from 0 to 1. This table defines the optimized path by checking the notation of the noise value of the road, smoothness, roughness, localization, and finally, the EPV factor.

Figure 7 contains the detail analysis for the optimal path relation with the EPV and payment condition and describes the feasible nature for the optimal decision-making by the system.

Figure 7

Optimal path by consideration of EPV factor and pavement level with different parameters.

Table 8 contains the analysis of the congestion condition considering the EPV factor, and there was feasible observation in the levels 5 and 4 where the driving condition is best by considering the multifactor analysis (Figure 8).

Figure 8

Analysis of the congestion condition.

Table 9 analyzes the traffic condition and pothole condition and contains the major feature balance for the optimal detection of the path to identify the feasibility for the driving. The table describes the conditions and reading obtained during the testing phase in the road.

Figure 9 describes the functional analysis for the optimal travel path for deciding the final route on the basis of pothole condition and the traffic light condition by using the EPV factor.

Figure 9

Analysis of pothole condition and traffic light condition.

The figure discusses the setup model, and this model is tested in the different road structure and domain. And it shows the promised observation in the strategic road condition.

Table 10 contains the comparison of the existing system in terms of the following parameters: sensor fusion, perception, localization, mapping, and efficiency. The proposed system shows better competency in deciding the path with the existing system.

6 Conclusion

Based on the findings from the system analysis, our team attempted to compare the results based on sensors, perception, and localization. We suggested that the system have a multifunction choice for choosing the pathway, taking the EPV component into account slightly to ensure it includes the most prevalent path. With this approach, we were able to identify the most efficient approach to take while discussing in a hostile environment. This was an effective way to determine the most effective course of action as it takes into account multiple factors such as sensors, perception, and localization. Furthermore, it further strengthens our team’s commitment to providing maximum efficiency with minimum effort.