preface

As a small developer who thinks he wants to know a little about everything, he has always been interested in WebRTC. This interest came from a few years ago when the company wanted to create a small instant messaging function on the APP, but it was finally shelved due to the change of the final requirements of the project. Nevertheless, I have learned some technical background about this technology, such as P2P communication, Intranet drilling and so on. Through several nights of learning and experiments, I have generally understood the principle and use method of WebRTC. Now I would like to share my learning process.

The preparatory work

As a document enthusiast, always read official documents and articles first to ensure that you get the latest and best first-hand information. WebRTC’s official website documentation is relatively comprehensive, but it seems that it has not been updated for a long time. Presumably, it’s been a long time since we made a feature upgrade. In this study, I referred to some official examples and added my own understanding. We can point out the mistakes and study together. References to the article will be added at the end of the article. So without further ado, let’s get started.

Turn on our cameras

WebRTC was developed by Google with the goal of creating a high quality, reliable communication framework. Literally we can split the Web into two parts: The Web is based on the Web, and the RTC is called Real Time Communications. So its function is to allow us to use the browser (can also be used for APP), a framework for real-time communication. Since it is a communication medium, of course, it is a variety of multimedia information, including video, voice, text and so on. You can even use it to transfer all kinds of files. So let’s start with the most intuitive video communication.

It is very easy to open the camera with the browser, we can directly call the JS API implementation.

  • HTML

      
<html lang="en">
<head>.</head>
<body>
    <h1>Get the video stream</h1>

    <! -- Set autoplay -->
    <video autoplay playsinline></video>
    <script src="js/main.js"></script>
</body>
</html>
Copy the code
  • JavaScript
// Media stream configuration
const mediaStreamConstraints = {
    video: true
};

// Get the video tag element
const localVideo = document.querySelector("video");

// Media stream object
let localStream;

// The callback saves the video stream object and streams it to the video TAB
function gotLocalMediaStream(mediaStream) {
    localStream = mediaStream;
    localVideo.srcObject = mediaStream;
}

// Handle error message
function handleLocalMediaStreamError(error) {
    console.log("Error opening local video stream:", error)
}

// fire!!
navigator.mediaDevices.getUserMedia(mediaStreamConstraints)
    .then(gotLocalMediaStream)
    .catch(handleLocalMediaStreamError);
Copy the code

The code is mainly divided into 2 steps

  1. fromnavigator.mediaDevices.getUserMediaTo obtain the video device.
  2. inthenA callback to stream the video tovideoThe label.

Pretty simple

It’s worth noting that I use Chrome, and the new version of Chrome has tightened security policies for acquiring devices. If you want to turn on devices like cameras, your domain name must be accessed via HTTPS if it’s not a local file or localhost.

Use RTC for P2P transmission

Now that we have the video stream, step 2, let’s use WebRTC’s RTCPeerConnection for local transmission. This Demo is not a real application scenario, because it does not involve network transmission in the real world. We just open two RTCPeerConnection on the same page to transmit the content of one to the other for communication. Before Posting the code, let’s briefly describe the process of creating a connection.

So let’s say that person A wants to play video with person B. Their offer/answer (application? / reply?) The mechanism works like this:

1. 'A' creates an 'RTCPeerConnection' object. 2. 'A' creates an 'offer' (an 'SDP' session description) using the 'createOffer()' method of 'RTCPeerConnection'. 'A' stores his' offer 'in the' offer 'callback using the' setLocalDescription() 'method. 4.' A 'strings his' offer', 5. 'B' receives' A ' 's' offer 'and stores it with' setRemoteDescription() ', so that his' RTCPeerConnection 'knows' A' 's configuration. 6. 'B' calls' createAnswer() 'and sends his local session description with his successful callback: This is the 'answer' of 'B' 7. 'B' sets his' answer 'to the local session description with' setLocalDescription() '8. 'B' then strings his' answer 'back to' A 'using some signaling mechanism 9.' A 'accesses' B's' answer 'as A remote session description using the' setRemoteDescription() 'methodCopy the code

It looks like a lot of trouble, but they actually did something

  1. Creating a session description (SDP)
  2. Exchange session description (SDP)
  3. Store the description of your conversation with the other party

For the format of the SDP, see the link later in this article

So let’s look at the code. Let’s go

  • HTML

      
<html lang="en">
<head>.</head>
<body>
    <h1>RTCPeerConnection transmits video streams</h1>
    <! -- Set autoplay -->
    <video autoplay playsinline id="localVideo"></video>
    <video autoplay playsinline id="remoteVideo"></video>
    <div>
        <button id="startBtn">start</button>
        <button id="callBtn">call</button>
        <button id="hangupBtn">Hang up</button>
    </div>

    <! -- Gasket for unified browser API -->
    <script src="js/adapter.js"></script>
    <script src="js/main.js"></script>
</body>
</html>
Copy the code

The HTML code is relatively simple. We created two videos, one showing remote and one showing local, and added three buttons to simulate dialing. As you may have noticed, we introduced a gasket adapter.js. Those of you who write a lot about the front end will probably be familiar with gaskets, because there’s not just Google’s browser in the world, there’s all kinds of other things. And then the name, the API is all different, so we’re going to use all kinds of spacers to unify our API. No longer suffer from compatibility. Adapter.js just exists. It was officially provided to us by Google. It allows us to operate with a uniform set of apis.

  • JavaScript

Since the code is relatively long, I will only paste the key code. I’ll post links to all the code at the end of the article.

// Start button to open local media stream
function startAction() {
    startButton.disabled = true;
    navigator.mediaDevices.getUserMedia(mediaStreamConstraints)
        .then(gotLocalMediaStream).catch(handleLocalMediaStreamError);
    trace('Local media stream open... ');
}
Copy the code

This is the function that responds to the start button. As in the first example, the main purpose is to open the camera and stream the video to the video TAB with the ID localVideo.

// Dial the button to create a peer connection
function callAction() {
    callButton.disabled = true;
    hangupButton.disabled = false;

    trace("Start dialing...");
    startTime = window.performance.now();
    
    // ...

    const servers = null;  // RTC server configuration

    // Create peer connetcions and add events
    localPeerConnection = new RTCPeerConnection(servers);
    trace("Create a local peer Connetcion object");

    localPeerConnection.addEventListener('icecandidate', handleConnection);
    localPeerConnection.addEventListener('iceconnectionstatechange', handleConnectionChange);

    remotePeerConnection = new RTCPeerConnection(servers);
    trace("Create a remote peer Connetcion object");

    remotePeerConnection.addEventListener('icecandidate', handleConnection);
    remotePeerConnection.addEventListener('iceconnectionstatechange', handleConnectionChange);
    remotePeerConnection.addEventListener('addstream', gotRemoteMediaStream);

    // Add the local stream to the connection and create the connection
    localPeerConnection.addStream(localStream);
    trace("Add local stream to local PeerConnection");

    trace("Start creating local PeerConnection Offer");
    localPeerConnection.createOffer(offerOptions)
        .then(createdOffer).catch(setSessionDescriptionError);
}
Copy the code

This part is the response function for dialing the button. So in this method, we did one thing.

  1. A pair of RTCPeerConnection objects, localPeerConnection and remotePeerConnection, are created for communication

  2. The response functions to the ICecandiDate (important) and iceconnectionStatechange events are registered for the two RTCPeerConnection objects respectively

  3. The response to the AddStream event is registered with remotePeerConnection.

  4. Add the local video stream to localPeerConnection

  5. LocalPeerConnection create offer

There is an ICE Candidate who is not mentioned above. A: Haha, it stands for Interactive Connectivity Establishment. It is a specification, in plain English, for establishing connections, since our WebRTC is for P2P connections, and our network is very complex, and most of it is on the Intranet (requiring holes or through firewalls). So we need a mechanism to set up Intranet connections. I’ll talk more about this in a later article. For now, it’s good to simply understand that achievements are for making connections. The icecandidate response method is used to store and exchange network information when the network is available.

// Define the RTC peer connection
function handleConnection(event) {
    const peerConnection = event.target;
    const iceCandidate = event.candidate;

    if (iceCandidate) {
        const newIceCanidate = new RTCIceCandidate(iceCandidate);
        const otherPeer = getOtherPeer(peerConnection);

        otherPeer.addIceCandidate(newIceCanidate)
            .then((a)= > {
                handleConnectionSuccess(peerConnection);
            }).catch((error) = > {
             handleConnectionFailure(peerConnection, error);
            });

        trace(`${getPeerName(peerConnection)} ICE candidate:\n` +
            `${event.candidate.candidate}. `); }}Copy the code

This code shows the process of saving and exchanging network information (ICE candidate). The Candidate is saved by calling the addIceCandidate method of the RTCPeerConnection object. So if you’re wondering, is there an exchange of Candidate information? Yes, the getOtherPeer method is actually used to get the peer’s RTCPeerConnection object, since our Demo was created on the same page. So you don’t have to swap through any other carrier.

Ok, so with connection creation out of the way, let’s move on to offer creation. Before creating the offer, we have noticed that the local video stream has been added to the RTCPeerConnection object, so the SDP session description that comes with the offer already has relevant information. Let’s start with the createOffer callback method.

/ / create the offer
function createdOffer(description) {
    trace(`Offer from localPeerConnection:\n${description.sdp}`);

    trace('localPeerConnection setLocalDescription start.');
    localPeerConnection.setLocalDescription(description)
        .then((a)= > {
            setLocalDescriptionSuccess(localPeerConnection);
        }).catch(setSessionDescriptionError);

    trace('remotePeerConnection setRemoteDescription starts.');
    remotePeerConnection.setRemoteDescription(description)
        .then((a)= > {
            setRemoteDescriptionSuccess(remotePeerConnection);
        }).catch(setSessionDescriptionError);

    trace('remotePeerConnection createAnswer started.');
    remotePeerConnection.createAnswer()
        .then(createdAnswer)
}
   
Copy the code

Simple and clear, for localPeerConnection is local, so call setLocalDescription to store the offer information. The remotePeerConnection is stored with the setRemoteDescription. Unlike what I said in step 4 earlier in the chapter, there is no conversion to a string. Those of you who are smart can probably guess why, because this is the same page and there is no need to transfer it.

Immediately afterwards remotePeerConnection calls createAnswer to create an answer, let’s continue,

/ / create the answer
function createdAnswer(description) {
    trace(`Answer from remotePeerConnection:\n${description.sdp}. `);

    trace('remotePeerConnection setLocalDescription begins.');
    remotePeerConnection.setLocalDescription(description)
        .then((a)= > {
            setLocalDescriptionSuccess(remotePeerConnection);
        }).catch(setSessionDescriptionError);

    trace('localPeerConnection setRemoteDescription starts.');
    localPeerConnection.setRemoteDescription(description)
        .then((a)= > {
            setRemoteDescriptionSuccess(localPeerConnection);
        }).catch(setSessionDescriptionError);
}
Copy the code

This does much the same with the createOffer callback above, storing the answer in the corresponding description of both parties.

Until now, the connection between the two parties has been established, and the offer and answer have been properly stored. Since remotePeerConnection has previously registered the addStream response method gotRemoteMediaStream and, as mentioned earlier, since the offer was created with the video stream, So gotRemoteMediaStream will now call back, using this method, to display the video stream in the remoteVideo TAB.

// The callback saves the remote media stream object and streams it to the video TAB
function gotRemoteMediaStream(event) {
    const mediaStream = event.stream;
    remoteVideo.srcObject = mediaStream;
    remoteStream = mediaStream;
    trace("Remote node link successful, receiving remote media stream...");
}
Copy the code

Now, we should be able to see two identical images. Notice that the one on the right is transmitted by RTC. Then take ~

Let’s stop here for this one and continue in the next one. The next article will continue to delve into WebRTC architecture and ICE, Signling and the like. Thank you for your reading, after all, I am also a beginner, if there is something wrong in the article, you can comment on it, and then discuss together. Thanks again.

Code and reference documentation

  • DEMO – 1 code
  • DEMO – 2 code
  • The official documentation
  • The official codelabs
  • Interactive Connectivity Establishment
  • Session Description Protocol

Agora SDK experience essay contest essay | the nuggets technology, the campaign is underway