“🌟🌟🌟🌟🌟“

“Taste: Shrimp balls in a dry pot“

“Cooking time: 10min“

Github github.com/Geekhyt, welcome to the canteen, if you think the food is delicious, a Star is a great encouragement to the canteen owner.

We know something about bitrate, frame rate, resolution, container format, and encoding in the audio and video terms at the beginning of the last column series. If you haven’t already, click the link above to jump to it.

Today, we come to learn about WebRTC, I believe you have heard about this front-end audio and video network celebrity.

WebRTC Web Real-Time Communication

WebRTC, which opened on June 1, 2011 and was endorsed by the likes of Google, Mozilla, Opera and others as a W3C recommendation, provides instant messaging capabilities to browsers and mobile applications.

Advantages and application scenarios of WebRTC

advantage

  • Cross-platform (Web, Windows, MacOS, Linux, iOS, Android)
  • Real-time transmission
  • Audio and video engine
  • Free, plug-in – free, installation – free
  • Major Browser Support
  • Strong hole-making ability

Application scenarios

Online education, online medical care, audio and video conferencing, instant messaging, live broadcasting, shared remote desktop, P2P network acceleration, games (werewolf killing, online KTV), etc.


(Who likes to play werewolf kill? Have time can come together a game, give me a round to listen to the speech time, give you naked point Wolf pit, a pit fault tolerance.

Overall WebRTC architecture

Back to take a look at the overall architecture of WebRTC, I’ve color-coded what each level represents.


  • The Web application
  • Web API
  • WebRTC C++ API
  • Session Management Indicates signaling Management
  • Transport the Transport layer
  • Voice Engine Audio Engine
  • Video Engine Video processing Engine

Let’s look at the core modules again:

Voice Engine Audio Engine

Global IP Solutions, a developer of VoIP software, offers the GIPS engine, arguably the best voice engine in the world, which Google bought and opened as an open source.

  • ISAC: Default codec for WebRTC audio engine, wideband and ultra wideband audio codecs for VoIP and audio streams.
  • ILBC: narrowband voice codec for VoIP audio streams.
  • NetEQ For Voice: Speech signal processing component For audio software implementation. NetEQ algorithm is an adaptive jitter control algorithm and a voice packet loss hiding algorithm, which can effectively deal with the impact of network jitter and voice packet loss on the voice quality.
  • Acoustic Echo Canceler: AEC.
  • Noise Reduction: NR, Noise Reduction.

Video Engine Video processing Engine

The VPx series of video codecs is open source after Google’s acquisition of ON2.

  • VP8: Video image codec, the default codec for the WebRTC video engine.
  • Video Jitter Buffer: Video Jitter Buffer module.
  • Image Enhancements: Image quality enhancement module.

WebRTC communication principle

Media negotiation

Media negotiation means that both parties can find mutually supported media capabilities, such as codecs, so that audio and video communications can be made between them.

SDP Session Description Protocal

The data exchanged through media negotiation is the SDP. In fact, THE SDP is not a real protocol, but a data format that describes the “capabilities” of each end.


The preceding figure shows part of the SDP. For details, see: SDP: Session Description Protocol

WebRTC: Session Description Protocol (SDP

Network consultation

ICE Interactive Connectivity Establishment

In order to establish a connection, we need to get IP and port information of both parties. In today’s complex network environment, ICE has unified various NAT traversal technologies (STUN, TURN), allowing clients to successfully penetrate various firewalls that may exist between the remote user and the network.

STUN and TURN

STUN: Simple UDP penetrating NAT enables the client after NAT(or multiple NAts) to find out its public IP address, the type of NAT in which it resides, and the Internet port bound to the NAT.

As we know, THERE are four main types of NAT:

  • Full cone NAT
  • IP limit cone type
  • Port limit cone type
  • symmetrical

The first three can use STUN penetration, while the fourth type, the symmetric NAT commonly used in large corporate networks, allows the router to accept only connections established by previously connected nodes.

So to deal with this network situation, we need to use TURN (Relay NAT penetrating) technology.

TURN is an extension of STUN, which mainly adds relaying capabilities. Add several TURN servers on the basis of STUN servers. If STUN fails to allocate public IP addresses, TURN servers can request public IP addresses as relay addresses to transfer media data through TURN servers.

Signaling Server Signal Server

Once we have the media information (SDP) and network information (Candidate) of both parties, we also need a signaling server as an intermediary to forward and exchange them.

The signaling server can also implement some IM functions, such as room management, user entry and exit, etc.

summary

In this paper, we have learned the advantages and application scenarios of WebRTC, the overall architecture and main module composition of WebRTC and the communication principle of WebRTC. These are the basic knowledge and concepts that we need to keep in mind

reference

  • Build audio and video Live Broadcasting System from 0 by Li Chao
  • React+Flutter+Go Combat kang Shaojun
  • Webrtc. Making. IO/webrtc – org…
  • Developer.mozilla.org/zh-CN/docs/…
  • www.w3.org/TR/webrtc/

❤️ Love triple punch

1. If you think the food and drinks in the canteen are ok with you, please give me a thumbs-up. Your thumbs-up is my biggest motivation.

2. Pay attention to the front canteen of public account, “eat every meal!”

3. Like, comment, forward === urge more!