Webrtc TURE, STUN, camera open actual combat

Preface:

Good weekend, everyone. Today I will share with you the first article of WeBRTC. I have never shared anything about weBRTC in the previous audio and video articles. Shared an article about the player last weekend, which described the design structure of the player as a whole; My personal learning path is divided into two main directions: FFMPEG and WEBRTC, and then into the specific protocols.

As for the second part of player actual combat learning, I will share my learning notes and feelings next week. Today, I’m going to share some principles of WebrTC call: STUN and TURN, which will cover THE PRINCIPLE of NAT penetration, and I will use the practical example of opening the webcam in Google Browser.

Ok, the details are as follows:

First, weBRTC call principle:

1. What is WeBRTC?

Before I introduce what weBRTC is, let me analyze the current background: Short video don’t know if you have any notice at ordinary times, more and more fire, such as video WeChat number, you trill, headlines, such as micro video, quickly, even on zhihu inside also began to play the video in this year, let alone especially popular live goods, these products can’t depart from the support of audio and video technology, especially the 5 g era, greatly solve the problem of bandwidth, Will lead to greater development and application of this technology; As learners and developers, we have to study hard and master the technology, so that we can have more competitive advantages in the workplace!

Well, said a little nonsense, the main is to let you know, these peacetime life we often play products, all involve the support of audio and video technology; So, what is WebrTC?

Webrtc English name: Web real-time Communication is an open source project of Google in 2011. It is mainly for Communication between browsers. Its emergence really solves many problems. You can go to the official website of WEBRTC, but it is generally not accessible in China now:

https://webrtc.org

Copy the code

2, WeBRTC call principle:

Before introducing this principle, let’s consider a question: how do browsers on different networks communicate in order to enable point-to-point (one-to-one) real-time audio and video conversations?

As shown in the figure above, let’s first consider two questions:

• The browser peer-A video is encoded by VP8(video image codec, the default codec of WebRTC video engine, which is suitable for real-time communication application scenarios, because it is mainly designed for low delay codec), and then sent to the browser peer-B. How should it decode?

• Browser peer-B uses VP9 to encode, so how to decode browser peer-A

Look at this, do you see that there is something missing, because the two sides can not communicate ah, you send me information, you can not use, I send you things, you can not use; Yes, there is no media negotiation SDP(Session Description ProtCOL), so both parties use H264 as the common codec, so both parties can codec correctly!

For example, if the two browsers are not on the same LAN, one end may be in Shenzhen, and the other end may be in Beijing. In this case, Net Address Transiation (NAT) is used. NAT may refer to its type here. This is not the focus of this article, but briefly, there are four types:

• Full cone NAT

• IP restricted cone NAT

• Port restriction cone NAT

• Symmetrical NAT

For details, you can go to the Internet to find information about their differences and principles of use, or you can go to see the course of Teacher Li I recommended before: build a live broadcast system from 0

NAT is a network address mapping, also known as translation, this should be easy to understand, your two browsers are not in the same LAN, that is certainly not direct communication ah, right?

STUN(Session Traversal Utilities for NAT) is required. STUN(Session Traversal Utilities for NAT) is used to provide public IP addresses and Traversal ports for terminals. STUN protocol for detailed analysis, you can see the official website manual:

https://datatracker.ietf.org/doc/html/rfc5389

Copy the code

Sometimes, however, STUN does not always succeed in assigning IP addresses to the calling devices that require NAT, so the problem arises. How can we solve this problem?

Yes, we also need TURN(Traversal Using Relays around NAT), which is an extension of STUN with Relayd; With this, we can solve the problem above

If STUN fails to allocate a public IP address, you can use the TURN server to request a public IP address as a trunk address. For more information about TURN, you can read the manual on the official website:

https://datatracker.ietf.org/doc/html/rfc5766

Copy the code

Network negotiation network negotiation network negotiation network negotiation

Ultimately, for browsers to exchange information, we also need Signal Servers to forward each other’s media and network information. There is no further explanation for the signaling server, which exchanges media negotiation and network negotiation information between the two ends of the browser. Of course, the signaling server also has the function of creating rooms and leaving rooms.

Two, using VScode actual combat examples:

1. Install the Live Server plug-in

Using VScode to install the Live Server plug-in, he can reload the page in real time in the local development environment:

Here may involve some front-end and JS knowledge, with c and C ++ foundation, can be used very quickly, we do not have to spend special time to learn, you can see rookie tutorial on the line.

2. Open the camera in Google Chrome

Code design process:

• Initialize button and video controls

• Bind the “Open camera” response event onOpenCamera

• To open the camera, click the “OpenCamera” button to trigger the onOpenCamera event

  • When the onOpenCamera call is triggered

  • Set the constraint, which is the input parameter to the getUserMedia function

  • GetUserMedia has two cases. One is to open the camera normally and use handleSuccess. One is if the camera fails to open and handleError is used

  • When the camera is normally opened, the video will be displayed by assigning the Stream object returned by getUserMedia to srcObject of the video control

Here is the complete code:

<! DOCTYPE html>

<html>

    <body>

        <video id="Local ‐ video" autoplay playsinline></video>

        <button id="showVideo"</button>

<p> Get the video from getUserMedia() </p>

    </body>

    <script >

            const constraints = {

                audio: false.

                video: true



            };

// The camera is opened successfully

            function handleSuccess(stream) {

                const video = document.querySelector("# local ‐ video");

                video.srcObject = stream;

            }

// Exception handling

            function handleError(error) {

                console.error("getUserMedia error: " + error);

            }



            function onOpenCamera(e) {

                navigator.mediaDevices.getUserMedia(constraints).then(handleSuccess).catch(handleError);

            }

            document.querySelector("#showVideo").addEventListener("click", onOpenCamera);



    </script>

</html>

Copy the code

The final result is as follows:

For the time being, this is only part of it. Add audio and dry, etc., and it will be optimized later!

Iii. Summary:

Well, that’s it for today, and we’ll see you next time!

Interested in audio and video partners can add my personal wechat: TU18879499804!

Related articles reference:

Build audio and video live broadcasting system from 0

https://ke.qq.com/webcourse/index.html#cid=468797&term_id=100561187&taid=4217056589719357&vid=5285890796286162335

Copy the code