The Concept of the Internet of Things was first proposed more than a decade ago. It relies on mobile communication networks to deliver its functions. In some device control scenes of the Internet of Things in the past, we have seen remote control technology more or less. However, limited by the network conditions and technical scenes at that time, most of the applications were simple operations of devices, and did not synchronize too much real-time on-site information. With the continuous development of communication technology and the emergence of 5G technology, intelligent life is getting closer and closer to everyone.
The emergence of 5G brings new features to mobile networks such as high bandwidth, low delay and local shunting. At the same time, remote control, as the forerunner of 5G technology, has important value for the intelligent era, and 5G can meet the needs of more information synchronization in remote control applications. It can be said that the maturity of 5G technology promotes the acceleration and landing of remote control.
At present, the typical application scenarios of 5G remote real-time control are mainly: remote takeover in the case of self-driving vehicle accidents in closed areas such as ports and open-pit mines and open roads, as well as remote operation in high-risk or harsh environments such as crowncrane, crane, chemical industry and underground mine. The former is used as a necessary emergency intervention means to better assist the local intelligent work of devices such as automatic driving; The latter is regarded as a normal operation mode to improve the operation experience of front-line personnel.
With the digital development of the industry, unmanned and remote operations will gradually become the industry trend in the future scenes like mines, ports and logistics, and C-terminal applications such as cloud taxi and cloud proxy driving will also gradually rise. It is expected that 5G remote real-time control will open up more than 10 billion market space and penetrate into various fields to help social development.
5G Main pain points and related technologies of remote real-time control
5G remote real-time control is mainly aimed at solving the remote control of complex equipment such as vehicles. It needs to support human-computer interaction based on real-time scenes.
In order to better restore the real operation scene at the remote end and facilitate personnel to carry out more detailed real-time control, in addition to the traditional status data, real-time synchronization of on-site video, audio and other media data will be introduced in 5G remote real-time control. In order to ensure the safety and smoothness of remote control, the synchronization of rich field data and meticulous remote operation has very high requirements on real-time perception and reliability and timeliness of operation.
Take the remote control scene of vehicle, which is very representative in the FIELD of 5G remote control, for example, it has strict delay requirements for timely transmission of information such as video images at the vehicle end. The following table is a simple analysis of real-time requirements of vehicle remote control in mobile scenarios. It can be seen that a delay of 200ms is recommended for remote driving at low speed, while a delay of 150ms is an ideal indicator. At present, the remote control delay based on traditional video surveillance is usually around 300-400ms. The delay of network, the delay of audio and video communication and the delay and reliability of control signaling are very high.
In order to reduce the end-to-end delay of audio and video in 5G remote control and ensure the reliability and timeliness of control, real-time audio and video communication, control signaling synchronization and 5G network optimization and other technologies should be introduced to jointly improve the control experience.
- Real-time audio and video communication: mainly to solve the real-time audio and video communication; In the remote control end-to-end delay, the ratio of audio and video communication delay often reaches about 80%. Therefore, it is very important to optimize the time delay of voice-visual frequency communication for remote control. In addition, in remote control scenes, multi-channel video streams are often used to restore the scene. A single device may involve the simultaneous transmission of 4-8 channels of HD video streams, which will occupy high network bandwidth. The optimization of video bit rate and lateness rate is also a factor of great concern to remote control.
- Control signaling synchronization: mainly solve the transmission reliability and delay of control signaling; The control signaling will eventually affect the action of the field equipment, so the reliability requirements are very high. On the basis of ensuring the delay as far as possible, the reliability needs to be maximized, and the detection and processing of various unexpected situations should be considered.
- 5G network optimization: mainly solve the low delay transmission of upstream audio and video data and ensure the downlink transmission of control signaling. Both audio and video communication and control signaling synchronization are based on the network. Under the harsh delay and reliability requirements, the application and network need to be coordinated and optimized to improve the end-to-end performance.
It can be seen that these three technologies are optimized and improved around the pain points such as delay and reliability of 5G remote control. Among them, 5G network optimization is the base, real-time audio and video communication is the core of delay optimization, and control signaling synchronization is the key to guarantee the reliability and safety of control. In addition to these technical optimizations, system architecture is also very important in the large-scale application of 5G remote control, which will directly affect the flexibility and expansibility of 5G remote control.
The four main architectures of 5G remote control system
The 5G remote control system mainly contains necessary elements such as controlled end, control end, 5G network, and optional elements such as remote control server. Here are some common system architectures in 5G remote control applications:
1) Architecture A: Direct bicycle connection + separation of video and control
This architecture is based on the simple extension of traditional video surveillance + traditional CAN bus control to achieve remote control in a simple 1-to-1 scenario.
-
Video link: The multi-channel camera is connected to a video gateway like NVR and connected to the 5G private network. The control end uses the IP address of the pre-configured video gateway to pull the stream and obtain the audio and video stream from the remote end.
-
Control link: Based on CAN bus, CAN bus data is transmitted over the IP network provided by THE 5G private network by means of CAN to Ethernet and then to CAN, and the connection between the CONTROLLER CAN interface of the controlled end and the controller CAN interface of the control end is completed.
Although this architecture can simply achieve the basic functions of remote control, the connection between the control end and the control end depends on the IP configuration at both ends in advance and the planning of network channels, so it is not flexible enough to be applied to large-scale multi-vehicle deployment scenarios. In addition, due to the delay of traditional video surveillance, its end-to-end delay is also large.
2) Architecture B: Direct bicycle connection + video and control integration
Also due to single-vehicle direct connection, such an architecture still has great flexibility issues in multi-vehicle scenarios with large-scale deployment.
3) Architecture C: Unified forwarding
Due to the limitations of direct bicycle connection architecture in deployment, a unified forwarding architecture appears. Multiple control and control ends are connected to a unified remote control server. The remote control server plays the role of connection forwarding to ensure the connectivity between the controlled end and the control end.
Based on this architecture, the control end and the controlled end can directly establish a connection according to their RESPECTIVE IDS through the remote control server, without knowing the IP address of the other end in advance, and without relying on the IP reacitability of the network at both ends.
Although this architecture greatly simplifies the complexity of large-scale deployment, the introduction of intermediate servers puts forward higher requirements on the forwarding capability and reliability of servers, and also introduces intermediate forwarding delay for remote control services.
4) Architecture D: Converged architecture
The fusion architecture is proposed by Tencent cloud 5G team and applied to its 5G remote control products. It has been applied in mining areas, ports, terminal logistics and other scenarios.
In this architecture, the remote control server is mainly responsible for the control surface, and manages the remote control gateway of the controlled end and the control end’s PC in a unified manner. Therefore, the control PC can still apply to the remote control server for connection establishment based on the ID of the controlled end, without pre-configuring the IP of the controlled end.
In the transmission process of audio and video data, control commands and sensors, the data surface still adopts the traditional direct network communication mode as far as possible. When the direct network is unreachable, the data surface is transferred through the media transfer server.
This architecture combines the advantages of single-vehicle direct connection architecture and unified forwarding architecture. It not only greatly simplifies the complexity of large-scale deployment scenarios, but also maintains the advantages of single-vehicle direct connection architecture with low latency, and greatly reduces the requirements on remote control servers.
In the long run, the converged architecture is the future development trend of 5G remote control. Because there are many application scenarios of 5G remote control, network scenarios are also complicated, including private network scenarios (such as remote control of mines and ports) and public network scenarios (such as terminal logistics, trunk logistics and cloud taxi). In addition, edge diversion and calculation will be combined with 5G MEC to further reduce network delay. Therefore, in terms of system architecture, it is a very good choice to separate the control plane from the data plane, which can more flexibly deploy the media data plane, adapt to the multi-type network environment, and give full play to the advantages of 5G MEC.
In the future, with the continuous development of 5G remote control applications, in addition to the continuous evolution and upgrading of technology and architecture, it is believed that the standardization of audio and video and control interface protocols at the controlled end and the control end will also be constantly improved, so as to realize interoperability between different vehicles and cockpits.