We must have used the function of wechat video call, if let you realize a video call function in your App, how will you achieve it?
You might think it’s complicated, because in the past, if you wanted to add a video calling feature to the App, you needed a whole team to do it, but now, with WebRTC, one person can do it.
What is WebRTC?
WebRTC, which stands for Web Real-time Communication, provides an API for sending any type of real-time data from the browser, but we focus on WebRTC’s ability to send audio and video data. The diagram below shows the architecture of WebRTC.
At the top, WebRTC defines a set of standards and new apis implemented by browsers, currently only in browsers, but will soon be implemented on other devices, such as mobile phones, so that we can use WebRTC in Native apps.
Before WebRTC, the technology of audio and video processing is very complicated and difficult, but after WebRTC, the technology of audio and video processing is encapsulated, especially there are a lot of third-party WebRTC technical solutions, and these solutions are out of the box, so that we can easily realize the video call function in App. Now let’s look at how WebRTC is used.
Make a video call in your browser
The easiest way to use WebRTC is to use it in the browser, because in the browser, we can directly use the API of WebRTC without using any plug-ins, which can be easily connected to the camera and microphone of the computer to develop video call functions.
The API is getUserMedia. This API allows us to access the user’s media device, for example: Camera and microphone, on a visit to these media device, this API can also be used to handle user permissions application problems, on the phone, can also be used to control the call is front-facing camera, or rear camera, but also can be used to handle different resolution video, after get the video and audio data, these data can also be streaming transmission.
Video call, it must be two people on the phone, if small red and xiao Ming to video calls, like call, to know each other’s phone number, telephone to dial, so, if small red to xiaoming and video calls, so little red first to know xiao Ming’s IP, then the little red and will establish a connection between xiao Ming, the connection is a point to point, This connection is then used to send audio and video data. Here’s how to set up this point-to-point connection.
Set up peer-to-peer video chat
WebRTC is designed to allow audio and video to be sent using peer-to-peer connections.
Because of this point-to-point connection, no server is required. For example, when you type www.baidu.com, you don’t actually go directly to Baidu. Instead, you go through the DNS server, resolve the domain name to an IP address, and then go to the server on the IP address. Peer-to-peer communication, also known as P2P, does not require a server in the middle. Two machines on the Internet can connect directly, because in a P2P network, each client is also a server.
So when Xiao Hong wants to have a video chat with Xiao Ming, instead of sending the audio and video data to the server, the server will find Xiao Ming’s address and forward the audio and video data to Xiao Ming. Instead, Xiao Hong and Xiao Ming will connect directly, and Xiao Hong will directly send the audio and video data to Xiao Ming.
To establish this point-to-point connection, WebRTC provides an API: RTCPeerConnection.
With RTCPeerConnection, audio and video data cannot be transmitted. Because the original audio and video data is too large, the original audio and video data needs to be compressed. The tool used for compression is called codec. Next, explain the audio and video codec and transmission.
Audio and video data encoding and decoding and transmission
WebRTC supports a variety of codecs, most commonly H264, MP3, etc., but when deciding which codec to use, make sure that the codec can encode audio and video at both ends.
Therefore, you need to send each other the information about the codec you support. This information is called SDP (Session Description Protocol). Exchanging this information between clients is called signaling, and is usually done using traditional protocols such as HTTP or WebSockets.
Finally, in order to actually establish P2P connections between clients, RTCPeerConnection uses the ICE protocol, known as Interactive Connectivity Establishment. In this process, one client sends all its ICE, IP addresses, and ports available to send media to the other client. The other client then checks all its ICE to find the one that is compatible with the ICE it receives, and once the combination is found, the client can send audio and video data over that connection. The process is shown as follows:
Agora SDK experience essay contest essay | the nuggets technology, the campaign is underway