Before understanding WEBRTC, in this chapter, we first understand the nature of audio and video real-time communication, understand the scope and background of the problem, so as to better learn a good solution.

For example, when Jay Chou holds an offline concert, he also opens an online live broadcast. If the ticket price is the same and conditions permit, I believe that most people will choose to listen to the concert online, because the feeling is different.

Which begs the question, what is it that feels different?

  • Real-time is not enough. Data will be delayed or stuck more or less in the transmission of network fluctuation, and this experience has a great impact on us
  • There is a loss of business data, and the sound image will be lost after the acquisition, compression and decoding of the equipment, which may be different from what people see.

So in essence, the essence of audio and video real-time communication is to restore the effect as close to reality as possible.

Audio and video quality indicators

The communication latency

In the real-time communication system, delay is a very important quality indicator, because delay directly affects the sense of experience. Imagine that when you say a word in a video with others, it takes one second for others to receive your message. Undoubtedly, this kind of experience is very painful.

Between the end-to-end, causes of delay has very many, for example, audio and video collection information, decoding time, network transmission time, buffer buffer time, audio and video rendering time and so on, among which, more often than not, dynamic change is network transmission time, influenced by the network is very large, and we are helpless to the network, The delay time caused by other factors is basically fixed.

Service quality indicators

Since the amount of audio data is very small, it has little impact on the network. This section mainly introduces the service quality indicators of video, and introduces some basic concepts first

  • Resolution refers to the number of pixels an image occupies on the screen. Note that this resolution has nothing to do with the physical size of the screen. The higher the pixel density of the image, the higher the resolution of the image, such as 720P and 1080P, are both referred to the resolution of the image
  • Frame rate refers to the number of frames played per second. The more frames played, the smoother the video will be. Generally, the frame rate of movies is above 24 frames, while that of high-definition videos is above 60 frames
  • Bit rate refers to the flux of data in unit time. In principle, the higher the resolution is, the higher the bit rate will be. Under the same resolution, the higher the bit rate, the better the picture quality will be

And understand the above, you will find that the delay and bit rate and conflict, to reduce the transmission delay, so it is necessary to minimize the amount of data transferred, the rate will reduce, unless the user network is enough bandwidth and good enough network quality, but obviously it is not realistic, often a product is good or bad, is determined by the worst environmental performance.

Optimal point

As mentioned above, the essence of audio and video real-time communication is to restore the effect close to reality as much as possible, so it is necessary to choose the right strategy and trade-offs to optimize the quality of service as much as possible

Optimize your Network connection

Due to the complexity of network links and the diversity of operators, network fluctuations are inevitable. How to optimize network links

The client, in webrTC, has a built-in routing scheme that selects the best network connection by priority

The server allocates more appropriate equipment room networks and provides more reasonable routing scheduling policies based on the carriers and regions that users access

Reduce data volume
  • Using better compression algorithm, we all know that the network data transmission, data is encoded, good enough if encoding algorithm, compression ratio is higher, so there is no doubt that can reduce the amount of data, in recent years introduced H265, than the current mainstream H264 AVI encoder, increased by 25%, 40% of the compression ratio, but at present there are a certain distance away from the commercial or.
  • The basic principle of Simulcate technology is to encode a variety of multi-channel streams with different resolutions, and then select one of the most appropriate bit rates to send to users according to the user’s different bandwidth. However, the only defect is that uplink users need to configure a good bandwidth.
  • Dynamic bit rate, as its name implies, dynamically adjusts the output bit rate. According to link availability, such as packet loss and delay, the output bit rate is continuously increased or decreased to reach a suitable bit rate for the current bandwidth of the user.
  • Loss of frames or reduced service, such as live video, severe bandwidth shortage, can be switched to voice communication.
Appropriately increase delay

For example, a frame of data needs three RTP packets (serial number 1, 2 and 3). It may be 1 or 3RTP packets that arrive first. Therefore, with the permission of network delay, it is appropriate to cache 1 and 3 RTP packets first. If 2 RTP packets arrive within a certain period of time, then we still consider the frame to be valid, but to control the delay time, a small amount of delay can also improve the direct connection of audio and video without affecting the real-time communication effect.

Improve network quality indicators
  • Packet loss: a network with packet loss less than 10% is considered normal by THE RTC
  • Delay: If the data transmission delay between the two ends continues to increase, the network may be blocked
  • Jitter has little impact and can be recovered by caching the first RTP packets through the buffer
  • Fec forward error correction, additional bandwidth costs, restore a small number of lost packets under the mechanism scenario.
  • Congestion control, through the algorithm to calculate the size of the bandwidth, to achieve dynamic adjustment of the bit rate

After having the basic knowledge of this article, then according to these contents, learn the content of WEBRTC.