There are a lot of new technologies in the audiovisual space, but not in the industrial and IoT space. In this Livevideostackcon 2021 Shanghai station conference, we invited Hong Xiaojian, the head of Xerox YOMO framework, to share with us what new blood Xerox and YOMO will bring to the industry and IoT.

By Hong Xiaojian

Organizing/LiveVideoStack

Hello, everyone. Today’s topic is a programmable streaming computing framework. You may all be concerned about the field of audio and video, and the scene we YOMO is facing is more inclined to industry, IoT and other fields. Although these are different scenarios, there are still many technical similarities. You’ll get a lot out of it.

Today’s outline is divided into self-introduction, YOMO project background, YOMO typical use cases, YOMO technology highlights, understanding of edge computing and summary.

About me and Xerox

First of all, let me introduce myself.

I started to do technology research and development in 2007 and have been an e-commerce platform for European users for 12 years. Due to my interest in edge computing and industrial Internet, I joined the current startup company — Xi Le Technology in 2019. At present, I am maintaining the main character of today — Yomo project.

Xi Le technology many friends are the first time to hear, the following to do a simple introduction. Xi Le Technology focuses on industrial Internet and edge computing, creating YOMO open source computing framework and YCloud cloud services. Since 2015, we have applied artificial intelligence technology in the field of industrial manufacturing, such as the quality inspection of flooring by computer vision. Xi Le Technology currently continues to serve more than 40 industrial enterprises. In 2019, China Light Industry Internet Co., Ltd was jointly established with China Light Industry Group, mainly for the implementation of edge computing and other industrial Internet technologies in the light industry.

02 YOMO Project Background

The following will introduce the project background and design principle of YOMO.

The first is openness. With the spread of network infrastructure, such as 5G, low latency seems to be within reach. Five years from now, the competition among enterprises may be the QUIC protocol, an open, User Space based algorithm that can do some flexible congestion control. The hardware and software of the future are likely to be programmable and open.

Looking back at some of the leading technologies in the industry today, when we think of real-time streaming computing we think of things like Flink, and message queues we think of Kafka. Even our microservice deployments a lot of people think of Docker, but these technologies were actually designed years ago, and they’re all part of the consumer Internet. The future will enter the IoT era, the former technology is still the mainstream, but not necessarily suitable for the future. When we build a product, we want to create new opportunities through new technologies.

In recent years, many industrial accidents have been reported in the news, which have caused great casualties and economic losses. In addition, the current national policy has been encouraging safe production, so our customers want to build safe production, such as digital detection and early warning system. China Light Industry Internet served a chemical company last year.

The highest level of production safety is to realize the theory of intrinsic safety. Intrinsic safety means that the occurrence of accidents can be predicted in advance regardless of equipment failure or human improper operation. To be intrinsically safe, it is necessary to calculate the trend of data change for 3 consecutive 3s. At the same time, the AI algorithm will react to possible accidents in the near future. For example, a chemical reactor might explode in the next 30 seconds, and fire retardants would need to be added to it in advance. To do this, you need to perform 30 computations in 1s, each of which is about 33ms. If this computing node is deployed in a cloud computing center, the transmission of optical data may have exceeded that time limit. The 33ms mentioned above not only includes data transfer, but also includes the time of AI calculation. Therefore, in order to achieve intrinsic safety, it is necessary to collect and calculate the data of the sensor in real time.

In order to achieve real-time acquisition, the transmission with low delay is needed. One is to use the protocol similar to QUIC; the other is with the popularization of 5G and WIFI6, it is of great help to guarantee the transmission with low delay. In addition, we need to calculate the collected data at the millisecond level, which needs to be deployed at the edge. If deployed in the cloud, even if the computing speed is very fast, the lack of transfer speed will cause millisecond scale computing to be impossible. In addition to the above two points, it is also necessary to deploy an Edge AI at the Edge to carry out full-dimensional calculation to realize the pre-prediction.

Based on current trends, real-time computing will become an increasingly large part of the future, with IDC predicting that real-time data computing will account for 30 percent by 2025.

These are some of the main protocols in the IoT space today. TCP was born in 1983, almost 40 years ago. The other major MQTT protocol has been around for more than 20 years. With the spread of 5G, these old technologies are like green trains running along bullet train tracks. Google proposed the QUIC protocol in 2012 and carried out the international standardization of IETF in 2016. However, because of the great upgrade cost, there are not many QUICs in the industrial field at present, but there are many applications in the audio-video field at home and abroad. The goal of Xerox is to make the QUIC technology easy to use in the industrial field.

We put forward the concept of GRPC for IoT. GRPC is a very mainstream RPC framework for microservices. GRPC for IoT is the QUIC Transport that hopes to realize full link at the edge end. For example, Client/Server services can be connected through QUIC to become P2P. The problem with the traditional Client/Server is that the Server responds only after the Client has requested it. This pattern is one-way. After using QUIC to build a connection, it is a two-way connection, Peer to Peer, and at the same time it is a long link. Why long links? Because IoT device data is available 24 hours a day, disconnecting and reconnecting with each request can cause a delay effect.

In addition, we have developed our own Codec for IoT. Xerox Technology attaches great importance to the efficiency of network transmission codec in the field of IoT. The mode of IoT data being captured in real time and sent over long links is very similar to live video.

IoT equipment will reach 75 billion by 2025. This means that more and more devices will need to collect data, resulting in more and more apps.

Now the market demands faster and faster APP development, the faster the time to market, the better. Now many low code, no code is to shorten the development time. The YOMO framework places great emphasis on being developer-friendly so that developers can save time when using it.

To save development time, we propose the concept of Streaming Serverless. The advantage of Serverless is that you only need to focus on a few lines of core code, you don’t need to worry about DevOps, you don’t need to worry about auto-elastic scaling, and you need to pay for it on demand, low cost. Data in IoT field is generated 24/7 without boundaries, which is a typical streaming scenario. Although there are many mature Serverless frameworks now, most of the Serverless frameworks in the market are oriented to the traditional HTTP Request/Response mode. Therefore, we propose the Streaming Serverless for this scene.

This is one of the more interesting tweets. The tweet came from Docker’s founder in 2019, when he tweeted that if WebAssembly appeared in 2008, there would be no need for them to build Docker. WebAssembly used to run more on the browser side, but now the trend is to run on the server side.

What are the advantages of WebAssembly over Docker? WebAssembly’s Cold Start is 100 times faster than Docker. Second, the former execution time is 10%-50% faster than the latter. Also, WebAssembly takes up less space. Finally, WebAssembly has a more flexible security policy that allows you to specify different permissions during instantiation depending on the module.

Because resources are limited at the edge nodes, WebAssembly combines lightweight, better performance, increased security, and multilingual features. Multilingualism is especially important for Serverless, since many of the major development languages now support compiling your programs into WebAssembly, and there are many benefits to having this feature.

Integrating all the above aspects, we made the YOMO open source framework.

03 YOMO application case

Let’s share some typical examples.

We deployed a real-time noise sensor in the office to test whether the YOMO framework could achieve low latency. Since MQTT protocol requires the installation of MQTT Broker, we have made an MQTT-compatible API at the data collection end, which can reduce the burden of users and enable them to access YOMO without installing MQTT Broker. In order to test the experiment, we deployed Serverless nodes in AWS in Ningxia to measure the delay from Beijing to Ningxia and then back to Beijing from Ningxia. Our test results show that the delay can be basically stabilized within 30ms. In addition, as for the decibel value displayed on the screen, the traditional approach is to save the sensor data to the database first, and then query and display it, which will cause the loss of delay. Therefore, YOMO is directly displayed on the screen through WebSocket. At the same time, use another Serverless service to drop the data to DB.

Liquor intelligent brewing platform is an industrial level application. One of the characteristics of the liquor industry is that a lot of the brewing process is imparted through the experience of old masters, which is very subjective. We obtained the hardware and related industrial algorithms after working with the China Food Fermentation Industry Research Institute, a professional research institute that has been studying baijiu for decades. Then we carried out real-time process collection and calculation on these equipment, digitized the experience of the old master, so as to obtain a stable process and improve the rate of wine production and efficiency.

An overseas user wanted to track user behavior and analyze what kind of behavior of users on some websites would lead to the reduction of conversion rate and other issues. In view of such a scenario, we made a Geo-Distributed solution to split the traditional centralized architecture into multiple edge nodes close to users.

The final example is a distributed crawler. We serve an overseas SaaS company that provides logistics inquiries. Previously, the query of this company was obtained by proxy, which would cause high delay and low stability. What’s more, data privacy might be leaked. Through the YOMO framework, we deployed a crawler service in the node closer to the Courier company to return the request to the user in the United States through the long connection through the QUIC protocol. These servers are all deployed on the user’s own machine, which ensures data privacy and saves the cost of proxy proxy.

04 YoMo features

Through the previous lecture, the guests here have some understanding of the QUIC protocol, so I will quickly go through these contents.

QUIC has many advantages. The best ones are the bottom two. One is User space, which I mentioned at the beginning of Openness. It is more convenient to upgrade software. The TCP kernel mode upgrade is not so convenient. The second is the congestion control algorithm. According to different scenarios for flexible control, with higher programmability.

Quic is widely used in the field of audio and video. Many large domestic companies began to research audio-video applications two or three years ago. QUICs are very helpful in improving performance, including the lag rate and so on. Because we are optimistic about the prospects of QUICs and have few applications in the industrial field, we want to promote the application of QUICs in the industrial and IoT fields.

The video here borrows the video from Alibaba Mobile Taobao. The multi-channel QUIC is used on the left, but not on the right. If the WiFi is jitter, the left side can run smoothly through the cellular network, while the right side can only use the WiFi and the situation is stalled.

Y3 Codec developed by ourselves is called Faster than Real-time. With traditional JSON, you need to take the complete data and then decode it. For IoT, such as noise mentioned earlier, just get the noise dB field. For this, we use the TLV structure. The Tag in the structure is equivalent to the key in JSON. By monitoring whether the user cares about the Tag, if it is, it will be directly obtained; if not, it will be SKIP, and then it will judge how many bytes to SKIP according to Length.

In performance tests, Y3 Codec was over 10 times better than JSON, and it was a significant improvement over ProtoBuf. The following table shows some of the performance reports.

Reactive programming is graphically illustrated with the application of Excel. Assuming that A = B + C, A will also have a dynamic response when B and C are modified. This mode is very suitable for the scenario where data changes over time.

ReactiveX also provides a good programming model for asynchronous data. ReactiveX was first proposed by Microsoft. You can use some common methods to manipulate asynchronous data, which is just as convenient as working with synchronous data. You can manipulate asynchronous data streams by combining several common functions.

Assuming that data is passed every 30ms, a real scenario might be two 30ms in a row without data, and a third 30ms with three data suddenly appearing. For this scenario we really just need to get the latest data. This is simplified by using RX, which allows you to get the latest data every 30ms using DeBounce.

Streaming Serverless allows users to operate only RX streams instead of QUIC streams. You can combine Operator methods based on business requirements. In addition, the local debugging of many Serverless services in the market is troublesome, so YOMO supports local operation and debugging in the CLI way.

05 Edge Calculation

And you know more or less about edge computing

The trend across the industry has been to move away from mainframes connected through terminals to a decentralized scenario on the PC side. In the era of mobile Internet, it has returned to the centralized cloud computing center. In the era of IoT, due to the huge amount of data, the edge end needs to be distributed to relieve the pressure of cloud computing center. Edge computing is becoming more and more important, but it’s not going to replace cloud computing, it’s going to exist together.

The first advantage of edge computing is to reduce the transmission distance. Second, the nearest calculation is faster response. Third, and more importantly, edge computing can protect security and privacy. Many industrial companies are reluctant to transfer data to public cloud services, so privacy protection is especially important. The last point is low cost. Edge computing can reduce the cost of bandwidth transfer.

The comparison between cloud computing and edge computing shows that cloud computing has stronger performance but higher delay and bandwidth costs, while edge computing is just the opposite. Cloud computing and edge computing are complementary in usage to meet the usage needs of different scenarios.

In this regard, we have done Geo-Distributed Edge Cloud. Users can deploy in different locations based on latency requirements. Low latency can be deployed on city-level nodes. It can be deployed in a private cloud if there is a data regulatory requirement. It is also possible to deploy in a cloud computing center.

06 summary

Finally, a summary of today’s report.

The background of YOMO project is openness to future programming. GRPC for IoT is proposed for network transmission. QUIC and Y3 Codec are used as high-performance Codec for the whole link. In addition, the framework of Streaming Serverless is proposed to speed up the development of APP. For the use of YOMO, WebAssembly will have more advantages than Docker. Finally, in terms of edge computing, YOMO can be deployed nearby based on Geo-Distributed Cloud.

The above is the open source plan of YOMO. I hope friends interested in YOMO can pay more attention to it.

Thank you!