Author: Li Tianyi, Huawei cloud EI expert
【 Abstract 】 Unmanned driving is through the automatic driving system, partially or completely replace the human driver, driving the car safely. The automobile automatic driving system is a complex system of hardware and software which covers multiple functional modules and various technologies. This article will explore the autonomous driving issue based on 5G technology.
Unmanned systems are increasingly expected to take the place of humans in some activities. Unmanned systems have permeated every aspect of human activity, from robots that help people sweep the ground automatically to drones that coordinate situational awareness on the battlefield. Driverless cars, for one, are in demand for a wide range of products, from port freight to passenger car driving. In recent years, with the promotion of demand and the popularity of artificial intelligence, many technological breakthroughs have been made in the field of driverless vehicles, which at the same time attract more investment and the investment of scientific and technological forces, making it a vigorous and emerging technological field [1-4].
Driverless vehicles are safely driven by autonomous driving systems that partially or completely replace human drivers. The automobile automatic driving system is a complex system of hardware and software which covers multiple functional modules and various technologies. Before the large-scale rise of machine learning, big data and artificial intelligence, autonomous driving systems were similar to other robotic systems, with the overall solution largely relying on traditional optimization techniques. As artificial intelligence and machine learning have made major breakthroughs in the fields of computer vision, natural language processing and intelligent decision making, academic and industrial circles have gradually begun to explore artificial intelligence and machine learning based on each module of unmanned vehicle system, and some achievements have been made. As a solution to replace human driving, unmanned driving system contains a lot of understanding of human driving habits and behaviors behind its design ideas and solutions. Now, unmanned driving has become one of the most promising applications of artificial intelligence [5-7].
1. Work related to automatic driving
Autonomous driving is a systemically complex effort, typically built on top of a conventional car. Below refer to gm’s Cruise self-driving cars introduces the hardware system architecture [8], as shown in figure 1, other companies are similar, can see clearly, self-driving hardware system mainly includes five parts: sensing module, autopilot computer, power supply module, communication module, execution and brake modules.
Figure 1 Automatic driving hardware system
Fig.1 Automatic driving hardware system
1.1 Perception module
As an important way to perceive the surrounding environment, the sensing module in the automatic driving hardware system has strict requirements in terms of specifications and performance, which is not available in traditional vehicles. The main function of the perception module is to replace the eyes and ears of the driver in the traditional driving car, as well as the driving experience. It usually consists of a camera, lidar, millimeter-wave radar, and GNSS/IMU.
Cameras are equivalent to the eyes of human drivers and are mainly used to obtain image information. They can be used to identify objects such as pedestrians, cars, trees, traffic lights and traffic signs for positioning purposes. Laser radar, in contrast, by receiving the reflection data, can obtain more abundant and accurate information, such as the target distance, azimuth, altitude, speed, attitude, and even shape parameters, such as to target detection, tracking and recognition and more accurate positioning, the principle of three dimensional range is determined by measuring the time difference and phase difference of the laser signal distance, The Angle is measured by horizontal rotation scanning, and a two-dimensional polar coordinate system is established according to the two data. Then, the height information of the third dimension is obtained by obtaining the signals of different pitching angles. The visual information obtained from the data obtained by laser after the processing of identifying, classifying and labeling different colors is shown in Figure 2.
Fig. 2 Schematic diagram of information acquisition by lidar
Fig.2 Schematic diagram of lidar information acquisition
The millimeter-wave radar operates at 24GHz and 77GHz, and can be used to identify obstacles and range by acquiring reflection data. Compared with other mainstream radars, millimeter-wave radar has better performance, which is not affected by the shape and color of the target object and atmospheric turbulence, and has a good stable detection performance and good environmental adaptability. For daily driving may encounter bad weather has a very good fault tolerance, by the weather and the change of the outside environment is little, in practical application, for rain and snow weather, dust, sunshine have a strong adaptation. In addition, the Doppler frequency shift is large and the measurement accuracy of relative velocity is improved, which is very suitable for the functions of high-precision positioning and identification of automatic driving and plays a great role in promoting the engineering application of automatic driving [9]. GNSS/IMU combination is used to obtain global location information in real time.
1.2 Automatic driving computer
Automatic driving computing is responsible for computing processing related to automatic driving, which generally consists of five parts: CPU, GPU, memory, hard disk storage space and hardware interface. There are also specialized processors designed to speed up calculations.
Among them, according to the performance characteristics of CPU, general, it is mainly used for processing logic judgment, process control and planning, the GPU used in parallel computing, complete the mass a large amount of calculation of the same kind of data, such as object recognition, classification, processing, etc., is generally used as deep learning deployment, there is a special processor of alternatives, Special optimization design is made for the matrix operation to improve the computing efficiency and obtain a higher energy efficiency ratio, such as Huawei’s ascension series processor. Memory is used for processing large amounts of data and loading high-precision maps, which has a great impact on the overall running speed of the system. The hard disk storage space is used to store high-precision maps and related application data. Rich hardware interfaces, such as serial port, CAN, Ethernet, USB, etc., are used for various sensor connections.
1.3 Execution and braking modules
Execution and braking are crucial to driving safety. With the continuous development of autonomous driving technology, implementation and braking systems have made great progress. The execution system receives the specific execution instructions of the vehicle made by the automatic driving control module according to the fusion decision, and controls the execution of the vehicle power (throttle and gear), chassis (steering and braking) and electronic and electrical systems, so as to realize the speed and direction control of the automatic driving. In contrast, the traditional automobile chassis braking system is hydraulic and pneumatic braking. In order to achieve the stability of the body frame and extend the intelligent driving function, the wire control movement will be the long-term development trend of automobile braking technology, and the wire control movement can deeply integrate the intelligent driving function module. This trend is similar to the gradual conversion of flight control systems from hydraulic to fly-by-wire systems in aviation.
1.4 Automatic driving software system architecture
On the whole, the hardware system of automatic driving is added and upgraded on the basis of the standard configuration of traditional vehicle hardware to better meet the needs of automatic driving.
In contrast, the matching software system is a brand-new design, equivalent to the human brain, which is responsible for making safe and reasonable decisions on the premise of integrating various sensor information and prior knowledge. By function, it is mainly divided into four modules: positioning, perception, planning and control. The positioning module is the foundation, and the contents of each module are shown in Figure 3.
Figure 3 Automatic driving software system architecture
Fig.3 Architecture of automatic driving software system
Is dependent on high precision positioning solution map of auxiliary, develop environmental awareness, path planning based on orientation information can be driving behavior decision-making and content such as vehicle motion control and path planning, behavior decision-making and interconnected, motor control loops and the previous output can be used as an input condition after. In short, the control module can take the action of decision planning as input, calculate the corner that should be executed and the throttle that should be controlled.
1.5 positioning
Positioning includes the positioning of the vehicle itself in the surrounding environment and the positioning of the surrounding objects, which is an important premise and information for decision-making. In order to meet the demand of automatic driving, the current minimum requirement of automatic driving positioning accuracy is 10cm. Generally speaking, it is difficult to achieve high-precision positioning through a single solution, mainly through multi-sensor and high-precision map fusion. The mainstream solutions are GNSS, IMU, lidar, camera and high-precision map fusion. The Global Navigation Satellite System (GNSS) mainly provides rough absolute position (latitude and longitude), and then more accurate positioning can be obtained according to the collected lidar data and camera data of the environment and high-precision map matching. IMU (Inertial Measurement Unit) Inertial devices provide acceleration and angular velocity in the equation of state (prediction) of state estimation algorithm.
The block diagram of the positioning scheme adopted by the autonomous driving team from Baidu is shown in Figure 4. This is a common and effective localization algorithm architecture at present. The research difficulty of location algorithm is that some minor processing and changes may cause a large accuracy gap. Therefore, researchers continue to make breakthroughs in positioning algorithms.
Fig. 4 Implementation block diagram of positioning algorithm
Fig.4 Block diagram of localization algorithm implementation
1.6 High-precision map
In the positioning scheme, the high-precision map plays an important role. High-precision map is to obtain road information data through high-precision laser radar, camera, GNSS and other sensors. Generally speaking, the more the number of sensors, the more comprehensive the information coverage, the higher the accuracy, the more accurate the high-precision map. In the case of automatic driving, it can be expressed as computer language and stored in the hard disk of the automatic driving computer. In the process of driving, high-precision positioning can be obtained through real-time comparison with high-precision map [10].
High-precision maps usually need to be established in advance, and there are a lot of classification problems in the establishment process, and the cost is huge, most of which is in the sensor system. Due to the huge data collected, various formats and many dimensions, artificial intelligence algorithm is usually used for data processing. High precision map mainly includes lane longitude and latitude, lane width, curvature, elevation, lane intersection position, width, curvature, signal lamp position, number of crossings, sign position and meaning, etc. At present, it is mainly processed by deep learning method. In the field of computer vision, Convolution Neural Network (CNN) is used to solve the problem well.
Fig. 5 Schematic diagram of neural network
Fig.5 Schematic diagram of neural network
Convolutional neural networks usually consist of one or more convolutional layers and full connection layers. The calculations performed by the convolutional layer include convolution operations and pooling operations. Convolution calculation is to obtain convoluted data through inner product (element by element multiplication and summation) operation of different window data and filtering matrix (a set of fixed weights). The pool calculation divides the data into blocks, and each data block is represented by the maximum value or the average value. The specific operation diagram is shown in Fig. 6.
Fig. 6 Schematic diagram of convolution (left) and pooling (right) operations
Fig.6 Schematic diagram of convolution (left) and pooling (right) operations
Another characteristic of the convolutional neural network algorithm is weight sharing, that is, for each point on an image, the weight of the convolutional operation at a certain layer is the same, and the parameters trained by the convolutional neural network are transformed into the training filter matrix (convolutional kernel), and the number of parameters is greatly reduced. Convolutional neural network is to obtain geometric information features in different directions through multiple convolutional layers. By extracting these features, the correlation of input data is obtained, and the training complexity is reduced by considering these correlations. This method has a good application in image and speech processing.
1.7 Perception and planning
Perception and planning go hand in hand and complement each other. Especially for online environmental perception, the main challenge for online real-time identification and classification of collected data is that the input data is dynamic and uncertain, with strong suddency and contingency, which is a great challenge.
The planning problem is to make decision on the motion sequence according to the perceived dynamic environment and the forecast of the moving body. Automatic driving requires rapid and accurate decision making in complex environments. It can be assumed that the test of planning through an extremely complex intersection is the key issue to reflect the intelligence level of automatic driving. The traditional path planning algorithm is not suitable for the complex dynamic environment because of the high time complexity. However, reinforcement learning is a good method to solve the sequential decision problem. At present, there are good simulations to solve the problem of autonomous driving planning. Reinforcement learning is a branch of machine learning on the same level as supervised learning and unsupervised learning. It comes from animal learning psychology and can be traced back to Pavlov’s conditional reflex experiment, in which the learning effect is constantly improved through feedback and evaluation of effectiveness [11].
1.8 control
The task of control is to digest the output track points of the upper motion planning module and convert them into specific control signals of the throttle, brake and steering wheel of the vehicle through a series of dynamics calculations, so as to control the vehicle as much as possible to actually execute these track points. This problem is generally translated into finding steering wheel Angle control (vehicle lateral control) and speed control (vehicle longitudinal control) that satisfy the dynamic attitude limit of the vehicle. The classical PID control algorithm can be used to control these state quantities, but it is strongly dependent on the model and has large errors. Intelligent control algorithms, such as fuzzy control, neural network control, etc., are also widely studied and applied in automatic driving control. Among them, neural network control uses neural network to regard the control problem as a pattern recognition problem, and the recognized pattern is mapped to the “change” signal of the “behavior” signal, which has achieved a good effect. It can even train the controller to obtain the control algorithm by using the data of the driver’s control process, which has a broad application prospect [12].
2 5G enlightenment and opportunities
Automatic driving is divided into six levels, as shown in Figure 7. At present, Level 3 can be reached when it is really applied in real life, which is mainly based on data fusion of sensors at the local end of the car to make decisions. There are certain limitations, which limit the development to a higher Level. Local sensors such as cameras and laser radars are limited by visual range, environment and other factors, and cannot achieve high safety. For example, when a car is traveling at a speed of 130 km/h, cameras and radars cannot safely detect a stop over 120 meters ahead, which will trigger an emergency brake, which is obviously unacceptable. How to make up for this deficiency and allow autonomous driving to look further is a challenge for researchers.
Fig. 7 Automatic driving level division diagram
Fig.7 Automatic driving classification chart
2.1 Autonomous driving scheme V2X under 5G
Currently, the industry is using C-V2X, a cell-based technology that supports not only existing LTE-V2X applications, but also 5G V2X applications, as shown in Figure 8. Based on the strong 3GPP ecosystem and continuous and perfect cellular network coverage, it is expected to significantly reduce the deployment cost of autonomous driving and Internet of Vehicles, and is likely to become the first successful vertical industry application scenario in the 5G era [13].
Figure 8 V2X evolution
Fig.8 V2X evolution
Unlike sensors such as lidar, V2X can be seen as a solution to a wireless sensor system that allows vehicles to share information with each other through communication channels, detect potential hazards, and expand autonomous driving perception, further enhancing the safety and comfort of autonomous driving. For example, if there is a broken car parked on the curve ahead of the road, but it is in the curve, it is difficult to detect the local sensors based on the car, and V2X can share the information over the network, and transmit the information of the broken car to the moving car.
5G, as an important communication carrier, is of great significance. In particular, the short-delay feature makes real-time communication no longer out of reach. It is highly reliable and low-delay service, which is very suitable for the field of automatic driving. In order to achieve the autonomous driving L5, high reliability and low delay are required. Due to the variety of sensors in the Internet of Things, MMTC (Mass Machine Terminal Communication) may also be needed. URRLC and MMTC are the final landing standards of 5G standard. Unlike the Internet’s “best effort” method of data transmission, 5G offers a consistent guarantee of low latency and high speed services, which is significant for driverless vehicles with high security requirements, such as the ability to guarantee network communication while driving in congested areas.
With the separation of control surface and data surface of 5G core network, NFV makes network deployment more flexible and enables distributed computing edge deployment. More data computing and storage will sink from the core to the edge and be closer to the data, thus reducing the delay and network load and improving data security and privacy.
It is worth mentioning that C-V2X can communicate with each other using the PC5 interface even in the absence of 4G/5G network coverage.
2.2 Opportunities brought by 5G
It will also be of great help to the high-precision map, the core technology of unmanned driving. By accurately describing the surrounding static environment, the high-precision map extends the sensing range of the sensor, and helps the car to understand the location, the surrounding environmental conditions, how to conduct the next operation and other positioning decisions at a more detailed scale.
High-resolution maps have more layers and more information than traditional navigational maps. In terms of the number of layers, it contains more data such as road levels, traffic facilities, etc. In terms of layer quality, high-precision maps are more finely delineated on each layer, allowing for centimeter-level navigation. High-precision maps provide special road information for autonomous driving. In addition to the road shape, direction and lane information of traditional navigation maps, they also contain information such as lane divider type, traffic signs, speed limit and some three-dimensional information of road geometry, such as curves and slopes. Huge data sets are a challenge in storage, processing and transmission, especially in transmission and update, because high-precision maps need to be updated in real time. Information collected by sensors and cameras can interact with the cloud through communication means, which can make maps more intelligent. Path planning based on intelligent map information makes traffic more efficient, which is made possible by the high bandwidth and low latency characteristics of 5G.
On the other hand, high-precision maps have a huge amount of data, reaching the Gbit/km level or above, and updating them in as little time as possible, which requires super-fast bandwidth support, which can only be provided by 5G networks.
Finally, the high-precision map can provide information of specific objects (moving pedestrians and cars) beyond the radar and visual detection range, as well as traffic lights and speed limit requirements. Based on this information, obstacle avoidance planning (including vehicle-vehicle game, vehicle-person game, etc.) can basically ensure the safety of unmanned vehicles on open roads. Will not occur any form of active collision and traffic violations, etc. The communication delay of this part of the content requires MS level, which urgently needs the support of 5G.
The sensors and cameras of the vehicle are equivalent to human eyes, the logical reasoning and decision-making of the automatic driving system are equivalent to human brain, and the motion control operation of the automatic driving system is equivalent to human hands and feet. The real-time communication of these information requires ultra-high speed transmission, ultra-high reliability and ultra-low delay. Compared with 4G network, 5G transmission rate increases 100 times, the peak transmission rate reaches 10Gbit/s, and the end-to-end delay reaches ms level. In practical applications, when a vehicle encounters an emergency on the road, the response speed is crucial. Only ultra-high speed and ultra-low delay such as 5G can meet the requirements of unmanned driving.
2.3 Development Status
In the 2019 World New Energy Vehicle Conference, Huawei’s 5G + C-V2X vehicle communication technology was highly recognized, and based on this, it developed the world’s first 5G vehicle communication module MH5000, which adopts self-developed Balong 5000 chip, single core and multi-mode, and supports SA/NSA dual network. With a maximum downlink peak rate of 2Gbps and a maximum upline peak rate of 230Mbps, it supports third-party application development and is the industry’s first module integrated with 5G + V2X technology.
Countries all over the world attach great importance to the development of relevant technologies, and China encourages the development of Internet of Vehicles technologies including C-V2X. A number of ministries and local governments have given clear policy support. According to incomplete statistics, the country has more than 30 test demonstration areas, including Shanghai, Beijing – Hebei, Chongqing, Wuxi (pilot area), Hangzhou – Tongxiang, Zhejiang, Wuhan, Changchun, Guangzhou, Changsha, Xi ‘an, Chengdu, Taixing, Xiangyang and other 16 national demonstration areas. These demonstration areas cover unmanned driving and V2X test scenario construction, LTE-V2X/5G vehicle networking applications, intelligent transportation technology applications and other functions, and provide test content involving safety, efficiency, information services, new energy vehicle applications and communication capabilities.
3 conclusion
Autonomous driving has broad prospects and is the development trend of the automobile industry in the future. Relevant technologies are developing rapidly. It is believed that with the comprehensive application of 5G technology, autonomous driving will achieve greater development.
But we still need to realize that the ultimate goal of autonomous driving, and the real commercial use, is a long way off. In addition to technology and funding, there are legal and ethical issues. More importantly, whether it can finally gain the trust and recognition of users and be accepted by users. At present, it is believed that autonomous driving will be realized gradually in several processes:
Stage 1: Assist driving safety and improve traffic efficiency.
The second phase: autonomous driving in closed parks.
The third stage: autonomous driving on open roads.
References:
[1] Liu Chuan, Chen Jinying, Zhu Zhengmo, et al. Impact analysis of 5G on Driverless Cars [J]. Communications and Information Technology, 2017(3):43-44.
[2] Chen Xiaobo. Challenges and Prospects for the Development of Autonomous Vehicles [J]. Integrated Transport, 2016 (11) : 9-13.
[3] Yu Adong, Chen Ruiwei. Research on Auto Autonomous Driving Technology [J]. Practical Automotive Technology, 2017 (2) : 124-125.
[4] min-hui zhu. Gradually achieve automated driving five levels [J]. Journal of automobile and accessories, 2016 (11) : 4.
[5] Gao Hongbo, ZHANG Xinyu, ZHANG Tianlei, et al. Research on Variable Particle Size Measurement of Intelligent Driving Vehicles based on Cloud Model [J]. Journal Journal of Electronics, 2016,44 (2) : 365-373.
[6] Guo Xu. Analysis and Prospect of Driverless Technology from the perspective of artificial Intelligence [J]. Electronic World, 2017 (20) : 64-65.
[7] Wang Kejun, Zhao Yandong, Xing Xianglei. Research Progress on the Application of Deep learning in the field of driverless cars [J]. Journal of Intelligent Systems, 2012, 13 (1) : 55-69.
[8] 2018 Self-Driving Safety Report [R]. General Motors Cooperation, February 2018.
[9] W. Lu, Y. Zhou, G. Wan, S. Hou and S. Song, “L3-Net: Towards Learning Based LiDAR Localization for Autonomous Driving,” 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019, pp. 6382-6391, doi: 10.1109/CVPR.2019.00655.
[10] Seif H G, Hu Xiaolong. Key Challenges of autonomous Vehicle Industry in Smart Cities: HIGH Definition Map [J]. Engineering, 2016 (02) :27-35.
[11] Xue Jianru, Li Gengxin. Scene Understanding and Autonomous Movement of driverless cars [J]. Unmanned Systems Technology, 2012, 1 (2) : 24-33.
[12] Lillicrap T P, Hunt J J, Pritzel A, Reinforcement Learning with Deep Reinforcement Learning [J]. Journal of Reinforcement Learning, 2015,8 (6) :A187.
[14] Scott P, Hans A, Xinxin D et al. Perception, Planning, Control, and Control. Journal of Autonomous Vehicles, 2017,5 (1).
This article was shared from Huawei Cloud Community “5G and the self-driving thing” by Tianyi_Li.
Click on the attention, the first time to understand Huawei cloud fresh technology ~