Transition from machine intelligence to knowledge intelligence: A multi-agent simulation approach to technology transfer

2.1 Building virtual environment

Road traffic accident is one of the major safety hazards around the world, and automated driving technology is expected to significantly reduce traffic accidents through route planning and intelligent cooperation [20,21]. In addition, the energy consumption of cars is very large, and autonomous driving technology can improve energy efficiency, reduce emissions, and increase fuel efficiency, thereby contributing to the sustainability of future transportation systems.

The emergence of autonomous driving is to solve traffic problems, such as traffic safety, congestion, sustainability, and efficiency, and improve the performance and quality of the transportation system. Through rational planning of the driving environment, it has a positive impact on urban life and social development. In order to carry out intelligent analysis of automatic driving effectively, it is necessary to build a virtual environment to provide all aspects of data information for the automatic driving system.

It can build a highly realistic virtual road environment, including road maps, road markings, and road obstacles. Real-world geographic information data can be obtained from OpenStreetMap, an open source mapping data. Map data can be preprocessed, and the obtained real-world geographic data can be applied to virtual environment through data cleaning, coordinate system transformation, and road classification. Some of the real-world geographic information data in OpenStreetMap is shown in Table 1.

Table 1 describes the part of the geographic information data, showing the road type, longitude, latitude, road name, speed limit, number of lanes, intersection type, and lane markings in detail. Interstate 405 has four lanes, with lanes marked as solid lines and intersections as roundabouts. All kinds of road environment can be simulated accurately through real geographic information data.

Since there may be noise data in the collected real-world geographic data, the missing values and outliers are processed by data cleaning. There may be data missing in the imported geographic data; the missing data needs to be deleted; the correctness of the data with large or small anomalies should be verified and manually corrected. When the abnormal value cannot be explained or is unreasonable, the abnormal value should be excluded. In order to facilitate geographic data analysis, the original geographic data is converted to the Shapefile format supported by the virtual environment.

The virtual environment uses the Cartesian coordinate system, while the real geographic data uses the latitude and longitude coordinate system, considering the curvature and projection of the earth, and converts the geographic data in the data set to the Cartesian coordinate system. Let the longitude and latitude coordinates of a point in real geography be (b, c), and the points in the converted Cartesian coordinate system be (x, y, z). The formulas of coordinate system transformation are expressed as

where R represents the radius of the earth.

The Cartesian coordinate system provides a unified coordinate representation, and converting geographic data into the Cartesian coordinate system can achieve the consistency of geographic data from different sources and formats. This helps to simplify the data processing and analysis process, allowing different types of data to be compared and integrated in the same coordinate system.

Weather simulators can be used to model various weather conditions, including simulating sunny, rainy, snowy and foggy days, and the shape of the sky, raindrops, and snowflakes can be changed through the parameter settings of the weather simulator. Simulating different weather conditions can change the friction on the road and the performance of the vehicle, so as to accurately analyze the automatic driving in the real environment.

Vehicle movement is also influenced by the surrounding dynamics and the complexity of the environment, through the introduction of virtual motor vehicles, pedestrians, non-motor vehicles, and traffic lights. It creates a real vehicle scene by adding traffic density and complex routes. The virtual vehicles introduced have intelligent agent functions and can follow traffic rules, and the simulated traffic lights are also consistent with those in the real world.

Classification of virtual environments, subdivision of virtual environments, and labeling is conducive to the subsequent training of autonomous driving models. The classification of virtual environments is shown in Table 2.

In Table 2, virtual environment classification is described. Six kinds of environments are refined to a total of 18 types, and each type of virtual environment is labeled. In the actual autonomous driving environment, six kinds of environments are combined to form a complex virtual environment. The virtual environment settings can help the system to comprehensively learn and adapt to various complex road scenes, and improve the robustness and safety of the auto drive system under different conditions.

2.2 Sensing the environment

With the increasing number of vehicles, in order to reduce traffic congestion, optimize traffic fluency, and improve road utilization rate, it is necessary to effectively coordinate vehicle actions through automatic driving system [22,23]. The autonomous driving system needs to accurately perceive the surrounding environment, make quick decisions according to the surrounding environment, and optimize driving efficiency under the premise of following traffic rules.

Machine intelligence is a computer system that enables machines to perceive, learn, understand, and make decisions and perform tasks by simulating human intelligent behavior [24,25]. The application of machine intelligence to the autonomous driving system can help the car intelligently perceive the road environment and carry out task planning.

In the process of automatic driving, the car obtains the surrounding road environment information through the camera, and the image including road signs, vehicles, and pedestrians can be collected through the camera. It is the basis of automatic driving perception to detect the features of the collected images and identify the object information in the images.

Inception is a deep convolutional neural network that processes input images in parallel by using multiple convolutional cores of different sizes to capture image features at different scales [26,27]. Inception model contains parallel 1 × 1 convolution, 3 × 3 convolution, 5 × 5 convolution, and 3 × 3 pooling, which reduces the feature dimension through multiple convolution kernels of different sizes.

The inception model adopts a multi-scale convolution design, which effectively captures different scales and complex features in images by using convolution kernels of different sizes in parallel. This architecture gives inception models an advantage in processing complex road environment images and can improve the accuracy of environmental perception.

The purpose of using multiple convolution kernels of different sizes in the inception model is to capture features of different scales in the input image. This multi-scale convolution strategy helps the model to understand and learn image information more comprehensively, improving the model’s perception ability of structures and patterns at different scales.

The inception model can effectively capture multi-scale features in input images and improve the model’s representation ability for complex images through parallel operations with convolution kernels of different sizes. Although the inception model reduces the number of parameters through dimensionality reduction operations, the model still has a certain computational complexity due to parallel operations and concatenation, especially in deep cases.

The activation function used in the inception model is

where ReLU ( x ) is a piecewise function. ReLU ( x ) has a value of 0 when x ≤ 0 is used.

The calculation of the ReLU function is very simple, just checking whether the input is greater than zero, so the calculation speed is fast. When the input is negative, the output of ReLU is zero, which may cause certain neurons to never be activated during training, known as the “death ReLU” problem.

The global maximum pooling layer is used to focus on the most obvious feature information in the image features. The formula for global maximum pooling is as follows:

where ( x ij ) r represents the information of the rth feature graph, and K r represents the most significant information of the rth feature graph.

After sensing the road environment, it is necessary to understand and analyze the surrounding information; integrate the road signs, vehicles, pedestrians, and other information collected by sensors; and comprehensively analyze the road environment data through the backpropagation neural network model. Let the perceived road environment data set be A = ( a 1 , a 2 , ⋯ , a n ) , and the connection weight between data set A and neurons be W = ( w 1 , w 2 , ⋯ , w n ) . The comprehensive analysis of the road environment can be expressed as

where a i represents the ith perceived road environment information.

During model training, the backpropagation neural network adjusts the weights of different neurons through errors, so that the errors reach an acceptable range [28,29]. Through forward data analysis and reverse transmission of errors, the ability of automatic driving to understand the road environment is improved. The errors are expressed as

where y 1 j and y 2 j represent the expected and actual output of the jth output neuron, respectively.

After comprehensive analysis of the perceived road environment information, the automatic driving system makes decisions based on the surrounding environment, deciding to accelerate, slow down, brake, turn, and path planning. The automatic driving system uses the RRT algorithm to create a tree, which contains the starting node as the root node, randomly generates a node, expands the node closest to the new node in the tree, and adds it if the path is feasible. In the RRT algorithm, the distance between two points A and B is expressed as

In the automatic driving system of machine intelligence, the road environment information can be sensed and intelligently understood. Intelligent decisions can be made based on real-time road environment conditions, and the vehicle can be physically controlled to achieve safe, efficient, and autonomous driving.

2.3 Knowledge representation and knowledge base

The automatic driving system of traditional machine intelligence still has certain technical defects, which makes it difficult to make accurate decisions in the face of complex road environment. Technology transfer is the transfer of a certain technology, knowledge, or innovation from one field or organization to another for wider application or commercialization [30,31]. Through technology transfer, the knowledge about vehicle driving can be applied to the automatic driving system, thus improving the performance of the automatic driving.

Autonomous driving should follow the traffic rules, and the safety of driving can be optimized through previous traffic accident cases. The knowledge base is determined to be traffic regulations manual, road sign database, and traffic accident case database. Using natural language processing technology to extract text data information from knowledge base, the extracted text data can be converted into the first-order logic, which is conducive to machine understanding.

This study converts the text information of Interstate 405’s speed limit of 105 km/h into SpeedLimit(Interstate 405, 105 km/h). The information in the knowledge base can be periodically updated to ensure that the information in the knowledge base is up-to-date. Domain knowledge related to automatic driving is associated by building a knowledge graph.

The inception model is used to process perceived environmental images. The inception model can extract advanced features of images for understanding information such as traffic signs and road conditions in the environment. The knowledge of traffic regulations, road signs, and traffic accident information obtained from the data is expressed. The information is mapped to a form that the model can understand, and a knowledge graph is constructed.

2.4 Reinforcement learning and knowledge integration

In order to comprehensively relate the knowledge related to automatic driving, it is necessary to integrate the relevant automatic driving knowledge. This article creates an interface that enables the agent to quickly query information in the knowledge base. This interface can also accept the query and return all knowledge related to the query content.

The state of the environment around the car can be understood and expressed into a form that the machine can understand, including vehicle speed, surrounding vehicle position, road sign information, etc. Specific actions can be performed based on information about the environment around the car and related knowledge.

Setting the action space for an automated driving agent allows people to define all possible agent actions. The actions of the agent include acceleration, braking, steering, changing lanes, keeping lanes, stopping, avoiding obstacles, emergency braking, changing lane signals, turning on the wiper, and adjusting the lights.

Autonomous driving is the process of continuous learning, improvement, and optimization of driving performance. Based on the data driven on the road, the learning ability of the autonomous driving can be enhanced by setting a reward function. In autonomous driving, the reward is usually based on the agent’s behavior, whether it follows traffic rules, whether it conflicts with other vehicles, whether it maintains a proper distance, and so on. It can positively reward performance that improves driving performance and penalize performance that decreases it.

The reward mechanism takes into account driving safety, driving efficiency, compliance with traffic rules and other directions, and safety behaviors such as avoiding collisions, not speeding, and not running red lights can obtain positive rewards. Positive rewards can be earned for following traffic rules, maintaining flow, and making efficient lane changes. Parking, yielding to pedestrians, and not driving the wrong way can get a positive reward. Incentives can be used to encourage agents to adopt safe behaviors and improve the safety of automated driving.

The design idea of the reward mechanism focuses on the safety, efficiency, and behavior conforming to traffic rules of the auto drive system. The goal of designing a reward mechanism is to guide autonomous driving agents to adopt safe, efficient, and compliant behaviors through positive rewards and punishments.

The current environmental state and available actions of the autopilot are input into the DQN model, so that the output layer of the DQN network is consistent with the available action dimensions, and thus, the Q value that may occur for each action is output.

The mean squared error is selected as the loss function of the DQN model to analyze the difference between the predicted Q value and the target Q value [32,33]. The mean squared error formula is expressed as

where n represents the total number of samples, y i represents the target Q value for sample i, and y ˆ i represents the predicted Q value for sample i.

In this study, Adam optimizer is selected for DQN model training to optimize the automatic driving system’s comprehensive analysis ability of driving-related knowledge and driving environment, and gradually realize the transformation from machine intelligence to knowledge intelligence.

2.5 Autonomous driving multi-agent model

The autonomous driving multi-agent model is a computational model used to simulate and optimize the driving of autonomous vehicles in multi-vehicle traffic environments, and comprehensively analyze the road environment and related driving knowledge. Vehicles can operate safely and efficiently in complex traffic scenarios through the collaboration of multiple agents.

The autonomous driving multi-agent model includes several virtual autonomous vehicle agents, each of which has the ability of perception, understanding, and decision-making. The autonomous driving system in the real world can be simulated through the cooperation of various agents, and the agents share information and cooperate through communication protocols [34,35].

3.1 Setting traffic situation

The multi-agent of autonomous driving with knowledge fusion can quickly perceive the surrounding environment through multiple intelligent agents and combine the relevant driving knowledge to make feasible driving behaviors. In order to effectively analyze the knowledge fusion of autonomous driving multi-agents, the experiment simulates various traffic situations in virtual environment and reflects the performance of autonomous driving multi-agents through simulated traffic situations.

Because there are too many traffic scenarios that can be set in the virtual environment, the labels of traffic scenario P1 are initialized as A1, B1, C1, D1, E1, and F1. In order to set up different traffic situations, only one of the environmental contents of road environment, traffic environment, terrain environment, weather environment, traffic sign environment, and intersection environment is changed, so as to generate different types of situations.

On the basis of traffic situation P1, the road environment is changed into expressway, rural road, and mountain road, and the corresponding traffic situation is PA2, PA3, and PA4, respectively. PB2 and PB3 can be generated by changing the traffic environment to free circulation road and construction area on the basis of traffic situation P1. By analogy, traffic situations can also be generated: PC2, PD2, PD3, PD4, PE2, PF2, and PF3.

The situational information about the road environment set in the virtual environment is shown in Table 3.

Table 3 describes the situational information about the road environment, showing the specific content in P1, PA2, PA3, and PA4 scenarios. By observing the performance of automatic driving in different situations, the influence of knowledge intelligence transformation on automatic driving performance can be well evaluated.

3.2 Evaluation of automatic driving performance

In order to effectively evaluate the performance of the knowledge fusion autonomous multi-agent system, it is necessary to compare the knowledge fusion autonomous multi-agent system with the traditional machine intelligence autonomous driving system. Traditional machine intelligence autonomous driving systems use backpropagation neural network models to comprehensively analyze road environment data to determine the movement of the car.

Knowledge fusion of automatic driving multi-agent system introduces a variety of driving knowledge on the basis of traditional machine intelligence, integrating knowledge of traffic regulations, road signs, and traffic accident cases. Through the reward mechanism and DQN model training, it improves the comprehensive analysis ability of the automatic driving system to the surrounding environment.

In order to comprehensively analyze the performance of knowledge-fused autonomous driving multi-agents, this article sets up a variety of driving scenarios. By observing the safety, efficiency, feasibility, environmental perception accuracy, and resource utilization of autonomous driving, it can judge the performance of autonomous driving multi-agents in simulated driving situations.

The first requirement of autonomous driving is safety, and the number of accidents per 100,000 km and the number of violations per 100,000 km of autonomous driving are recorded in the simulated environment P1. Vehicle accidents are identified as vehicle collisions, including vehicle-to-vehicle collisions, vehicle-to-guardrail collisions, and vehicle-to-pedestrian collisions. The violation of the vehicle is reflected in the violation of traffic rules, including running red lights and not driving at the specified speed.

Traffic intersections are complex driving environments. In this study, three traffic intersection environments, P1, PF2, and PF3, were set up to record the average driving speed of vehicles in the simulated environment. Different weather conditions would also affect the efficiency of driving. In this study, virtual environments of P1, PD2, PD3, and PD4 were set to record the average driving speed of autonomous vehicles.

The realization of the automatic driving system is based on the perception of the external environment, and the accurate identification of traffic signs and road obstacles can provide information for vehicle driving. The traditional machine intelligent automatic driving can perceive the environment through inception model, while the knowledge-integrated automatic driving establishes a traffic knowledge base on the basis of inception model, and comprehensively realizes the environment through knowledge representation and image recognition.

Automatic driving environment perception can be divided into four categories according to the results of predictive recognition and actual results: TP indicates that the environment is recognized as a positive example and the actual is also a positive example. FP indicates that the environment is identified as a positive example but is actually a negative example. FN indicates that the environment is identified as a negative example but is actually a positive example. TN indicates that the environment is identified as a negative example and the actual example is also negative.

Environmental perception performance can be comprehensively evaluated by accuracy, precision, recall, and F1 value, and the accuracy formula is expressed as

where N represents the total number of samples, N = TP + FP + FN + TN .

The precision rate measures how much of the model’s predicted positive class of samples is actually positive class of samples, namely, the accuracy of the predicted positive examples. The formula of precision rate is expressed as

The recall rate measures the proportion of the number of positive class samples correctly predicted by the model to the total number of actual positive class samples, i.e., the model’s ability to identify positive examples. The formula of recall rate is expressed as

The F1 value is determined by the results of formulas (11) and (12), and the F1 value is expressed as

In order to more intuitively reflect the environment perception ability of machine intelligent automatic driving and knowledge fusion intelligent driving, receiver operating characteristic (ROC) curve is drawn to evaluate the environment perception performance of different automatic driving systems.

In order to fully analyze the performance of the knowledge fusion autonomous driving system, the resource utilization of the autonomous driving system in the simulated environment is measured. CPU is a key component for performing various computing tasks in an autonomous driving system, and CPU utilization represents the extent to which the CPU’s computing power is used to run an autonomous driving application. GPUs are commonly used in autonomous driving systems for image processing, deep learning inference, and computationally intensive tasks. GPU utilization represents the extent to which it is used when running an autonomous driving application. The simulation environment of automatic driving is set up: P1, PA2, PA3, PA4, PB2, PB3, PC2, PD2, PD3, PD4, PE2, PF2, and PF3.

4.1 Security

The simulated environment P1 is composed of urban road, congested road, flat road, sunny road, clearly marked road, and intersection. The safety performance of the autonomous vehicle is recorded in P1 environment, and the number of accidents per 100,000 km and the number of violations per 100,000 km are shown in Figure 1.

Figure 1

Security results: (a) safety of automated driving with traditional machine intelligence and (b) safety of knowledge-integrated autonomous driving.

In Figure 1(a), the safety of automated driving with traditional machine intelligence is described. In this study, the number of accidents and violations were measured in 100,000 km in P1 environment, and the horizontal axis indicates that a total of 10 sets of tests were conducted. Traditional machine intelligent autonomous vehicles have been involved in driving accidents and violations in 100,000 km. The average number of accidents of traditional machine intelligence autonomous driving in 100,000 km was 3, and the average number of violations in 100,000 km was 4.3. The safety of knowledge fusion autonomous driving is described in Figure 1(b), and 10 sets of tests were also conducted. In the 100,000-km P1 virtual environment test, the number of accidents and violations of knowledge fusion autonomous driving was smaller than that of traditional machine intelligence autonomous driving. The average number of accidents in 100,000 km was 1.4, and the average number of violations in 100,000 km was 1.8. When perceiving the surrounding environment, the driving knowledge base can be combined with the knowledge base content and the road environment analysis results to make driving decisions together. Knowledge fusion of automatic driving can effectively reduce the number of accidents and violations, thereby improving the safety of automatic driving.

4.2 Driving efficiency

Autonomous driving is autonomous navigation and driving without human intervention, and efficient driving planning can improve road capacity and reduce traffic congestion. This study analyzes the driving efficiency of automatic driving in different traffic intersections and different weather conditions.

4.2.1 Traffic efficiency at different traffic intersections

The intersection environment in the virtual environment includes intersections (P1), roundabout (PF2), and T-intersections (PF3), which use the average speed of the autonomous vehicle to reflect the driving efficiency. The driving efficiency results of different traffic intersections are shown in Figure 2.

Figure 2

Traffic efficiency at different traffic intersections: (a) driving efficiency of traditional automatic driving at different traffic intersections and (b) driving efficiency of knowledge fusion automatic driving at different traffic intersections.

In Figure 2(a), the driving efficiency of traditional machine intelligence autonomous driving at different traffic intersections is described. This study sets up three kinds of intersection environment: intersection, roundabout, and T-intersection. Traditional machine intelligent automatic driving in P1, PF2, and PF3 environment are all urban roads, and the driving speed is not very high. The driving speed of different intersection environments is different, which is mainly due to the different traffic complexity of different types of intersection environments. The average driving speed of the 10th group in P1, PF2, and PF3 environments was 46.7, 49.5, and 55.4 km/h, respectively. Figure 2(b) describes the driving efficiency of knowledge-integrated intelligent automatic driving at different traffic intersections, and the driving speed of knowledge-integrated automatic driving in three traffic intersections is higher than that of traditional automatic driving. This is mainly knowledge fusion automatic driving that can integrate a variety of driving knowledge, so as to make more intelligent driving decisions and improve driving speed. The average driving speed of group 10 in P1, PF2, and PF3 environments was 53.9, 60.0, and 69.5 km/h, respectively. Therefore, knowledge fusion automatic driving can improve driving efficiency at different types of traffic intersections.

4.2.2 Driving efficiency under different weather conditions

Virtual environments P1, PD2, PD3, and PD4 are set, which are sunny roads, rainy roads, snow and ice roads, and foggy roads, respectively. The driving efficiency results of different weather environments are shown in Figure 3.

Figure 3

Driving efficiency under different weather conditions: (a) driving efficiency of traditional automatic driving in weather environment and (b) driving efficiency of knowledge fusion automatic driving in weather environment.

Figure 3(a) describes the driving efficiency of traditional machine intelligence autonomous driving in different weather environments. In this study, the driving speed of sunny road, rainy road, snow, and ice road and foggy road are calculated. Road friction is different in different weather conditions, and self-driving cars need to adjust their speed according to weather conditions to ensure safety. The driving speed is the highest in sunny days, which is mainly due to the wide driving field of vision in sunny days and the high friction coefficient of the ground. The average driving speed of the 10th group in P1, PD2, PD3, and PD4 environments was 52.0, 40.0, 26.5, and 28.5 km/h, respectively. Figure 3(b) describes the driving efficiency of knowledge fusion automatic driving in different weather environments. The driving speed of knowledge fusion automatic driving is generally higher than that of traditional automatic driving. The average driving speed of group 10 in P1, PD2, PD3, and PD4 environments was 62.8, 42.8, 31.5 and 35.0 km/h, respectively. Therefore, knowledge fusion automatic driving can improve driving efficiency under different weather conditions.

4.3 Environment awareness performance

The environmental perception accuracy rate, precision rate, recall rate, and F1 value of traditional machine intelligence automatic driving and knowledge fusion automatic driving are shown in Table 4.

Table 4 describes the environmental perception performance of traditional machine intelligence automatic driving and knowledge fusion automatic driving. The accuracy rate, precision rate, recall rate, and F1 value of knowledge fusion automatic driving are all higher than those of traditional machine intelligence automatic driving. This is mainly due to the knowledge fusion of autonomous driving that can conduct comprehensive analysis of traffic regulations, road signs, and traffic accident-related information. The environmental perception accuracy of traditional machine intelligence autonomous driving and knowledge fusion autonomous driving was 87.2 and 96.8%, respectively. The shift from machine intelligence to knowledge intelligence can effectively improve the environment perception performance of autonomous driving systems.

In order to intuitively analyze the environmental perception effects of different automatic driving systems, the ROC curve is shown in Figure 4.

Figure 4

ROC curve of environment perception.

Figure 4 describes the ROC curve of environmental perception. The horizontal axis represents the false-positive case rate, while the vertical axis represents the true-positive case rate. The larger the area under the curve, the better the performance of environmental perception. It is obvious that the area under the ROC curve of knowledge fusion automatic driving is larger than that of traditional machine intelligent automatic driving. Both knowledge fusion autonomous driving and traditional machine intelligent autonomous driving have ROC curves that exceed random classifiers, but knowledge fusion autonomous driving has better performance. The ROC curve of knowledge fusion automatic driving is closer to the point (0, 1), which means that knowledge fusion automatic driving has better environmental perception performance and can more accurately identify traffic signs and road obstacles in complex driving environments.

4.4 Resource utilization result

The resource utilization of the knowledge fusion automatic driving system is analyzed, and the utilization of CPU and GPU is shown in Figure 5.

Figure 5

Resource utilization: (a) CPU utilization and (b) GPU utilization.

In Figure 5(a), CPU utilization is described, and 13 virtual environments are represented on the horizontal axis. Average CPU utilization and peak CPU utilization of 13 virtual environments are counted. The CPU utilization of knowledge fusion autonomous driving system is different in different virtual driving environments, which is mainly due to the computational complexity of different virtual environments. The average CPU utilization of knowledge fusion autonomous driving in 13 virtual environments was 34.7%, and the average peak CPU utilization was 65.1%. In Figure 5(b), GPU utilization is described, and the GPU utilization of the knowledge fusion automatic driving system is very good. It can make good use of GPU resources, so as to effectively improve the computing efficiency of automatic driving. The average GPU utilization of knowledge fusion autonomous driving in 13 virtual environments was 75.9%, and the average peak GPU utilization was 96.1%. Application knowledge fusion of automatic driving can make good use of CPU resources and GPU resources.