Will WebRTC go mainstream? It’s the age of crowdsourced CDN!

Content Source:On June 24, 2017, Xie Ting, front-end engineer of Lixiang Computing, delivered a speech titled “Acceleration of P2P-CND Streaming Media Based on WebRTC” at the Tencent Web Front-end Conference TFC 2017. IT big said as the exclusive video partner, by the organizers and speakers review authorized release.



Words of Reading:Time: 6 minutes

Video address: t.cn/RCIMlWn

PPT address: t.cn/Rp9hZ7R

Before we get to WebRTC, let’s review the evolution of Web communications. Before AJAX, in 2005, you had to reload an entire web page to update content. With the advent of AJAX, web pages could be updated asynchronously by exchanging small amounts of data in the background with the server. However, AJAX does not communicate duplex with the server, so the server cannot actively push messages to the browser, which can only poll them. The advent of Websocket has changed this situation, browser and server can communicate with full duplex. Whether it’s AJAX or Websocket, you need to send data to the server. In order to transmit data between two users, developers need to purchase a network of servers, which can be very costly. WebRTC, a new technology backed by Google, has changed all that. WebRTC, short for Web Real-Time Communication, enables direct real-time Communication between browsers without server transfer. Google is committed to making it one of the HTML5 standards, and most browsers already support it.

Google chrome officially supports WebRTC natively in 2012. Web developers can develop rich real-time multimedia applications with only a few lines of javascript code, and users can directly open the browser to chat with each other in real time without installing plug-ins. At this time, some sensitive developers began to use WebRTC’s data channel technology to do P2P streaming media, such as a foreign company called Peer5. Alan, the founder of our company, also devoted himself to this research when he worked in Tencent. However, he was disappointed to find that P2P streaming media with WebRTC still had some problems to solve. For example, the online time of users was not stable, and the data channel of WebRTC would be closed when users closed the page. In 2013 and 2014, Firefox and Opera announced support for WebRTC. Alan then came up with a bold idea: since browsers are not stable, how about implementing the same protocol in routers and NAS? We all know that routers are on 24 hours a day, but most of the time they are idle. It would be cool to think that if you could harness all that computing power and network bandwidth, you could have millions of homes as nodes, and your neighbors or even you could be speeding up your videos. Therefore, we proposed the concept of crowd-sourced CDN and applied for a patent. In 2015, Tencent’s X5 browser kernel and wechat also provided support. In the same year, our Pear Computing was officially announced to be established.

You may wonder, will WebRTC really become a mainstream technology in the future? Let’s just use the facts and look at the support of the major browsers. As you can see, most browsers already support WebRTC, including Chrome, Firefox, and Opera, and Microsoft’s Edge browser partially supports WebRTC. Apple also announced WebRTC support for safari11 at its recent WWDC conference. There will be more and more webrTC-based applications in the future, that’s for sure.

Let’s assume that there are two browsers A and B that want to establish A WebRTC peer connection. Peer connection is A direct media connection between two Web browsers. If A wants to contact B proactively, it needs to send an SDP to the signaling server through HTTP. Session Description Protocol is used to describe the media characteristics of peer connections. So what is a signaling server? It’s like a matchmaker, helping two people who don’t know each other. The SDP sent by browser A is called offer, and the signaling server sends it to browser B, who responds with an SDP object called Answer and sends it to A through the signaling server. After exchanging SDP, the two peers try ICE holing. After successful holing, they negotiate the key, and then they can start a secure media or data session.

Due to the limited IP resources provided by IPv4, IPv6 has not been promoted. Most network devices are on the Intranet, and NAT devices are required to connect to the Internet. A router equipped with NAT software is called a NAT router. It has at least one valid external global IP Address. In this way, all hosts using local addresses must translate their local addresses into global IP addresses on the NAT router before they can connect to the Internet. If two peers are on different Lans, you need to know the public IP address and port number of the peer first. A test packet can be sent to the STUN server, which responds with an IP address detected in the test packet and returned as a potential candidate. Get the candidate’s address to be sent via signaling server browser to peers, peer side also do the same thing, after both sides get candidate address try to connect with all, if no successful case, will TURN to a transit server, TURN the server is under the condition of all alternatives are invalid is taken, Because of the high cost. To speed up call setup time, there is a trickle-down scheme called Trickice. The idea is that clients collect candidates and send them to each other as they go. For example, local candidates can be initiated directly without stuN capture, which reduces the time it takes for connectivity checks to complete.

Next, we introduce an important concept — WebRTC Data Channel. We do P2P streaming media based on WebRTC, which is actually the data channel capability. So what exactly is a Data channel? While WebRTC’s hype has focused on its support for real-time audio and video communications, designers have long wanted it to support real-time data transfers as well. Compared with Websocket and HTTP, data channel supports connection with large traffic and low latency, and has the advantages of stability and reliability. In addition, the interface of data Channel is the same as websocket, which also sends data through Send and receives data through omMessage. Then how to control the reliability of data channel data transmission? With the dataChannelOptions javascript object, you can switch the Data channel between UDP and TCP advantages, such as more reliable data transfer, or faster data transfer. Ordered: sets whether the data needs to be transmitted in the order in which the data is transmitted, maxRetransmitTime: sets the length of transmittance when the data is transmitted fails, and maxRetransmits: sets the maximum number of transmits when the data is transmitted fails. The main configuration is that ordered, when true, the data channel behaves more like TCP, and when false, it behaves more like UDP.

In addition, our company has been paying attention to and contributing to WebRTC standardization. Last year, in our research and development process, we found that a third-party WEbrTC protocol stack could communicate with Chrome browser, but could not communicate with Firefox. By comparing SDP, we found that one implementation of Firefox was inconsistent with the standard specification. We got in touch with the Firefox development team and submitted our proposed changes, and they initially thought they were fine, but eventually they took our advice and changed the SDP. This is our small contribution to pushing weBRTC standardization forward. In addition, we have been keeping in touch with Tencent browser kernel team, striving for full support of WebRTC technology and the upper p2P-CDN acceleration protocol shared this time.

After WebRTC’s data channel is clear, we can use it to do P2P streaming media. There is already a well-known open source project developed by foreign giants: WebTorrent, which has more than 10,000 stars on Github. WebTorrent is an open source JS framework based on WebRTC and BT protocol. It is written entirely in javascript and can run on Both Node.js and browser. Because it is based on WebRTC, WebTorrent can run on the browser without installing any plug-ins. Supports Chrome, Firefox and Opera browsers. However, because it is based on BT protocol, it is a pull-based algorithm. This algorithm is a random grab strategy, which randomly grabs buffers of other nodes. However, there is a problem in this way: The captured buffer is not necessarily what is needed at present, nor what is needed by other nodes. Moreover, it will waste downstream bandwidth and upstream bandwidth of other nodes, thus causing both “bandwidth hunger” and “content hunger” problems. Here is an improved version of the pull-based algorithm – FirstAid algorithm. FirstAid is based on window sliding, which triggers a window sliding at regular intervals. Each window can be divided into three segments: Urgent, Normal, and Prefetch, as the name implies, are the buffer nearest to the playing time, so the priority level is the highest, while normal and Prefetch have decreasing priorities. When the parent node transmits buffer as the child node, the requirements of urgent level will be met first, while those of normal level will be suspended. Therefore, the most urgent needs will be met first. When the urgent needs of the child node are met, it needs to go back to make up for the needs of its competitors, so as to achieve a mutually beneficial state. Contrary to the pull-based algorithm, the push-based algorithm is more representative of FashMesh algorithm, which is a P2P algorithm proposed by scholars of HKUST. FashMesh is based on an algorithm called Streaming Mesh, which divides the data stream of the source node into multiple sub-streams and transmits the Mesh to the sub-nodes continuously through multiple spanning trees. The advantages of this algorithm are low latency and high bandwidth utilization. Fast Mesh also dynamically adjusts the network topology based on the upstream bandwidth of each child node, enabling nodes with large upstream bandwidth to be closer to the source node, thus making full use of the existing network capabilities. According to a comparative test, FastMesh is probably the most effective P2P algorithm out there. However, this algorithm also has disadvantages. When the nodes enter or leave the network, the topology needs to be readjust, so it is not suitable for the situation where the nodes change greatly.

The algorithm developed by ourselves — push-pull algorithm combines the advantages of push-based and pull-based algorithms, obtains the buffer with the highest priority from the parent node in the Pull way, and the parent node provides subsequent buffers for it in the Push way. In addition, our algorithm blends HTTP, HTTPS, WebRTC, Websocket and other protocols to maximize the P2P rate while prioritizing user experience. After testing, the push-pull algorithm has the advantages of low latency, high bandwidth utilization, high P2P rate and strong robustness to network topology changes.

PearPlayer (github address: https://github.com/PearInc/PearPlayer.js) is written entirely in JavaScript source HTML 5 streaming framework, A plug-in free Web streaming media acceleration capability that integrates HTTP (including HTTPS, HTTP2) and WebRTC with multi-protocol, multi-source, low latency and high bandwidth utilization is realized. H5-based MSE technology (Media Source Extension) feeds buffers from multiple Source nodes into blocks to the player, together with carefully designed algorithms to achieve optimal scheduling strategies and deal with various abnormal situations. Pear Player maximizes P2P rates while ensuring a smooth video experience for users.

Integrating our Pearplayer. js is also very easy. It only takes a few lines of code to import our JS file into the script tag and pass the VIDEO ID and token as parameters to the function we provide. Demo Demo address: Q.webrtc.win /watch, the following is the Demo screenshot.

In addition to player, we also developed a multi-protocol, multi-source, hybrid P2P-CDN downloader, PearDownloader, which can be used for high-definition images, compressed packages, software release or upgrade packages, music, documents and other large files download or online service scenarios (Github address: Github.com/PearInc/Pea…). , Demo Demo address: q.webrtc.win/ Download.

Finally, talk about fog calculation. What’s the difference between fog computing and cloud computing? Clouds float in the sky, high and out of reach; The data center is far away from end users, and user messages need to go through several hops to reach them. And fog is a cloud close to the ground, a reality that is accessible, right next to you and me. Fog computing is not made up of powerful servers, but rather of a variety of computing devices that are weaker, more dispersed but closer to the user, such as smart routers and network storage devices. Fog can make up for the lack of clouds, and the cloud and cooperate with each other, work together. Our company Pear has been practicing the concept of fog computing. Through cooperation with well-known domestic routers and NAS manufacturers, we have a large number of nodes that provide sustainable and stable services. Most bandwidth, storage, and computing resources are crowdsourced from the edge devices of end users who are stable online. The service capability covers all regions, carriers, and network edges. Our self-developed scheduling system can sense and schedule dynamically and in real time, so that the data transmission distance is as close to “zero hop” as possible.

Relative to the traditional model, we can say that we are standing on the tuyere of sharing economy. We know that traditional CDN manufacturers will first buy bandwidth from ISPs at wholesale price, and then sell it to CP at retail price. CP will buy bandwidth and distribute content to provide CDN services for end users. We partner with hardware manufacturers to embed our software in their devices, so that we have widely distributed nodes in every home. CP manufacturers and traditional CDN manufacturers buy bandwidth from us, and we distribute the content to each node. When the users who hold equipment improve their computing and bandwidth resources, they also get rebates from us, and the pressure on BGP machine room and ISP backbone network is relieved, so as to achieve a win-win situation.