preface

The biggest difference between this article and other similar articles is that: there is a running demo for comparison: Demo address: www.fffuture.top/testRTC Looking back at the whole practice process, the comparison of the parts lies in:

  1. Set up conturn server to provide TURN/STUN service
  2. Domain name registration and HTTPS configuration
  3. Purchase and configure cloud servers
  4. The weBRTC connection process is cumbersome

Only point 4 is recorded in this article. Note: HTTPS/cloud servers can be configured without the need to simply run locally (webRTC requires an HTTPS /localhost environment). I also set up a coturn server. If you want to try it, you can use the turn/ STUn address in the demo.

The demo presentation

One day, Lucy missed Ace very much, so Lucy initiated a chat with Ace:

SequenceDiagram Lucy->>Lucy: Enter name :Lucy Ace-> Ace: Enter name :Ace Lucy->>Lucy: Enter receiver 'Ace' Lucy->>Ace: Send message: 'Come to video 'Ace ->>Ace: Received message from Lucy Ace->>Ace: Enter recipient 'Lucy' Ace->>Lucy: Reply message: "OK"

The demo performance is as follows:

Then they launched a video chat:

SequenceDiagram Lucy->>Lucy: Initialize webRTC Ace-> Ace: Initialize webRTC Lucy->>Ace: Initialize webRTC Send RTCOffer Ace-->>Lucy: Auto reply answerOffer Ace-->Lucy: Video call... Lucy->>Lucy: Close webRTC Ace->>Ace: Close webRTC

The demo performance is as follows:

The signaling server is set up

WebRTC connection and the whole communication process need to be involved in signaling, such as offer initiation, ICECandidate exchange, etc.

A signaling server is required to establish a WebRTC connection between two devices. The signaling server’s role is to act as a middleman to help both parties establish a connection with as little privacy exposure as possible.

WebRTC does not limit signaling server types. The purpose of webRTC is to exchange information, whatever the purpose is. Here I choose the KOA2 framework with WebSocket.

WebRTC does not provide a signaling mechanism, you can use any method you like such as WebSocket or XMLHttpRequest to exchange each other’s token information.

Websocket backend

Use KOA to start a backend service with HTTPS configuration.

    var https = require('https');
    const sslify = require('koa-sslify').default;
    const Koa = require('koa');
    const app = new Koa();
    
    var options = {
        key: SSL certificate key,certPem}; app.use(sslify());var server = https.createServer(options, app.callback());
    server.listen(443.(err) = > {
        if (err) {
          console.log('Service startup error', err);
        } else {
          console.log('IM running on https443 port '); }});Copy the code

Next integrate WebSocket into KOA and do basic message processing:

  • Ws connection initialization
  • Ws Message Forwarding

Note: Since webSocket initialization is a TCP upgrade and cannot carry parameters, a separate initialization message is required.

    const WebSocket = require('ws');
    const wss = new WebSocket.Server({ server });
    
    let wsPool = {};
    
    wss.on('connection'.wss= > {
        wss.on('message'.msg= > {
            msg = JSON.parse(msg);
            if(msg.type === 'init') { // Mount the WS instance to wsPool during initialization
              wsPool[msg.sender] = ws;
            }else { // Other message types are directly forwarded
              if(Reflect.has(wsPool, msg.recipient))
                wsPool[msg.recipient].send(JSON.stringify(msg)); }}});Copy the code

This simple server is done.

Websocket front-end

Next, write a simple front page:

Continue, front-end page websocket initialization, key code:

/ * * *@description Initialize webSocket *@return {Object} Websocket instance * /
function initWS() {
    if(!Reflect.has(window."WebSocket")) {
        console.log("Browser does not support Websocket!!");
        return;
    };

    let WS = new WebSocket("wss://www.fffuture.top");

    WS.onopen = () = > console.log("-- Successfully connected to websocket--");
    WS.onmessage = envelope= > {
        // The received message is displayed in the History Message box
        msgHistory = document.querySelector("#msgHistory");
        msgHistory.innerText += `${envelope.data}\n`;
    }
    WS.onclose = () = > console.log("-- webSocket disconnected --");
    WS.onerror = error= > console.error("-- websoket error:", error);
    return WS;
}
Copy the code

Once you have a WS connection, you also need to send a message function:

/ * * *@description Encapsulate the WebSocket send event@param {Object} {type: String, content: String} */
function wsSend(data) {
    let sender = document.querySelector("#sender").value,    / / the sender
    recipient = document.querySelector("#recipient").value,  / / the recipient
    msgHistory = document.querySelector("#msgHistory");      // History message box

    // Special case: no receiver is required when init
    if(! sender || (! recipient && data.type ! = ="init")) {
        alert(`${sender ? 'Receiver' : 'Sender'}Can't be empty! `);
        return;
    }

    let letter = JSON.stringify({... data, sender, recipient}); msgHistory.innerText +=`${letter}\n`
    WS.send(letter);
}
Copy the code

I expect the data structure of the ‘message’ to be:

{
    sender: string
    recipient: string,
    type: string,
    content: string
 }
Copy the code

Under test:

This ends the Websocket section.

WebRTC connection

At this point, only the front end remains:

The preparatory work

First we enter trickle-ice with Firefox browser to test whether stun and turn services are normal.

Build webRTC

First, continue to improve the front-end web page:

Next, add the corresponding events:

  1. Initialize the webRTC

Initializing a webRTC is essentially creating a peerConnection (an end-to-end connection object) and configuring methods such as getting local/receiving remote media streams and displaying them on the Video TAB.

function initRTC() {
Copy the code
  • Initializing peerConnection requires passing in the RTCConfiguration object, where only iceServers are set. This is where the TURN/STUN server comes in: to determine the best route and protocol to use when communicating between two peers.

IceServers are an array of objects that describe STUN and/or TURN servers at the ICE layer, used when trying to establish a route between caller and callee. These servers are used to determine the best routes and protocols to use when communicating between peers, even if they are behind a firewall or using NAT.

    const config = {
        iceServers: [{urls: "Stun: 139.224.75.6:3478".username:"".credential:""
            },
            {
                urls: "Turn: 139.224.75.6:3478".username: "wsj".credential: "123456"}].iceTransportPolicy:"all".iceCandidatePoolSize:"0"
    };
Copy the code

The RTCPeerConnection interface represents a WebRTC connection from the local computer to the remote. This interface provides implementations of methods to create, hold, monitor, and close connections.

    pc = new RTCPeerConnection(config);
Copy the code
  • Get local media streams
    navigator.mediaDevices.getUserMedia({audio: true.video: true})
    .then(function(localStream) {
        let videoSelf = document.querySelector("#video-self");
        videoSelf.srcObject = localStream;
        localStream.getTracks().forEach(track= > pc.addTrack(track, localStream));
    })
    .catch(E= >{... });Copy the code
  • When a remote media stream is received
    pc.ontrack = media= > {
        document.getElementById("video").srcObject = media.streams[0];
    }
Copy the code
  • Important: Send this local candidate ICEcandidate to the other party when onicecandiDate is triggered: This event is active throughout the video/voice call, it does not continuously provide icecancdiDate (interactive connection candidate) to ensure the best quality throughout the call:

After adding the local descriptor with pc.setLocalDescription(offer), an ICecandidate event will be sent to RTCPeerConnection.

ICECandidate: Interactive connection establishment candidate

The RTCIceCandidate Interface — Part of The WebRTC API — A candidate Interactive Connectivity Establishment (ICE) configuration which may be used to establish an RTCPeerConnection.

    pc.onicecandidate = wapper= > {
        if(! wapper.candidate)return;
        wsSend({type: "candidate".content: wapper.candidate});
    }
    pc.onicecandidateerror = error= > {
        console.error("-- error in obtaining candidate:", error);
    };
Copy the code
  • This method is called when the local webRTC is ready. The demo uses buttons to send offers, so this event is not used:

Once the caller has created its RTCPeerConnection, created the media stream, and added its track to the connection, the browser passes a Negotiationneeded event to RTCPeerConnection to indicate that it is ready to begin negotiating with the other peers.

    pc.onnegotiationneeded = () = > {
        console.log("-- Negotiated connection event ----")}Copy the code
}
Copy the code
  1. Send Offer

Corresponding to the “send RTCOffer” button event, the use of Websocket to forward the message is the main function of signaling service:

/ * * *@description Send offer * /
function invite() {
    const offerOptions = { offerToReceiveVideo: 1.offerToReceiveAudio: 1};
    pc.createOffer(offerOptions)
    .then(gotDescription,noDescription );

    function gotDescription(desc) { //desc: RTCSessionDescription -> sdp
        pc.setLocalDescription(desc)
        .then(() = > {
            console.warn("---- local ready, ready to send offer----");
            wsSend({type:"offer".content: pc.localDescription});
        });
    }

    function noDescription(error) {
        console.log('Error creating offer: ', error); }}Copy the code

RTCSessionDescription Describes how one end of a connection or potential connection is configured. Each RTCSessionDescription consists of a description type that indicates the SDP protocol description of the request/reply negotiation process it describes.

RTCSessionDescription The process of negotiating a connection between two peers involves exchanging objects back and forth, with each description representing a combination of connection configuration options supported by the sender of the description. Once the two peers agree on the configuration of the connection, the negotiation is complete.

When the remote party receives the invitation Offer, it automatically replies to the Offer:

/** * @function answerOffer(data) {const remoteDesc = new RTCSessionDescription(data.content); pc.setRemoteDescription(remoteDesc) .then(function() { pc.createAnswer() }) .then(function(answer) { return Pc.setlocaldescription (answer)}).then(function() {console.log("---- send answer offer-----"); wsSend({type:"offerAnswer", content: pc.localDescription}); }). The catch (err = > {console. Warn (" error - reply offer: ", err)})}Copy the code

When the local place receives answerOffer, set the remote SDP locally:

/ * * *@description Set the SDP of the received AwserOffer to the local PC@param {Object} data* /
function setRemoteSDP(data) {
    const remoteDesc = new RTCSessionDescription(data.content);
    pc.setRemoteDescription(remoteDesc)
    .then(() = >{ console.log("-- INVITE Setting remote SDP successfully"); });
}
Copy the code
  1. Close the webRTC:
function closeRTC() {
  var remoteVideo = document.querySelector("#video");
  var localVideo = document.querySelector("#video-self");

  if (pc) {
    pc.ontrack = null;
    pc.onremovetrack = null;
    pc.onremovestream = null;
    pc.onicecandidate = null;
    pc.oniceconnectionstatechange = null;
    pc.onsignalingstatechange = null;
    pc.onicegatheringstatechange = null;
    pc.onnegotiationneeded = null;

    if (remoteVideo.srcObject) 
      remoteVideo.srcObject.getTracks().forEach(track= > track.stop());

    if (localVideo.srcObject) 
      localVideo.srcObject.getTracks().forEach(track= > track.stop());
    
    pc.close();
    pc = null;
  }

  remoteVideo.removeAttribute("src");
  remoteVideo.removeAttribute("srcObject");
  localVideo.removeAttribute("src");
  remoteVideo.removeAttribute("srcObject");
}
Copy the code

conclusion

The above is just a simple practice of webRTC connection establishment. Most events in demo need to be manually triggered by users, which is obviously unreasonable. A more natural way of interaction can be seen in my other IM application: support video calls, online chat, theme switching, multi-language experience address: ffFuture. top github: github.com/fffuture/IM

reference

  • Developer.mozilla.org/zh-CN/docs/…
  • Developer.mozilla.org/zh-CN/docs/…
  • Developer.mozilla.org/zh-CN/docs/…
  • Developer.mozilla.org/en-US/docs/…