WebRtc enables multi-party video conferences and video recording and uploading

0: preface

Recently, the company made a project demand to achieve multi-person video conference chat, data access, decided to use HTML5 new support WebRtc as video communication. The client can use a BROWSER that supports HTML5, such as Chrome. The Server provides two main service functions: a Signaling Server and a NAT penetration Server. The signaling server uses WebSocket for business functions such as calling, joining a conference, hanging up, etc. The NAT penetrating server uses Chrome’s built-in

const peerConnectionConfig = {
    iceServers: [
        {url: 'stun:stun.l.google.com:19302' }
    ]
};
Copy the code

A simple block diagram is as follows:

1: Advantages and application scenarios of WeBRTC

Advantage:

1 Cross-platform (Web, Windows, MacOS, Linux, iOS, Android)

2 Real-time Transmission

3 Audio and video engine

4 free, plug-in free, installation free

5 Supported by mainstream browsers

6 Strong ability to play holes

Application Scenarios:

Online education, online medical care, audio and video conferencing, instant messaging, live broadcasting, shared remote desktop, P2P network acceleration, games (werewolf killing, online KTV), etc.

2: Some basic concepts about WebRTC

The traditional video streaming technology is generally implemented as follows: The client collects video data and streams it to the server, which then pushes the video data to other clients according to the specific situation. It is similar to live broadcast, but WebRTC is completely different. It can directly build a UDP-based data channel between clients. You can transfer arbitrary data directly between two browsers on different devices. The process includes:

1: open the local camera (the purpose is to create the video stream locally, which is to be sent to the other party) 2: initialize the link foundation of both parties and create PeerConnection 3: Start to create the link basic information, which starts to formally establish the communication foundation with the other party (save an offer locally after the creation is completed) 4: After creating the offer information, A calls setLocalDescription to store the local offer description and then sends it to B. B calls setRemoteDescription to store the description of the remote offer after receiving the offer. Then create the answer information, also need to call setLocalDescription to store the local answer description, and then return to A after A gets the answer, Call setRemoteDescription again to set the description of the remote answer. 5: Listening for ICE candidate information (this process is between sending an offer and creating a reply) 6: Successfully creating a WebRTC connectionCopy the code

3: Create an RTCPeerConnection

The RTCPeerConnection object is the entry to the WebRTC API and is responsible for creating and maintaining a WebRTC connection and transferring data within that connection. At present, most of the new versions of browsers support this object, but due to the current API is not stable, so it needs to add the prefix of each browser kernel, for example, in Chrome, we use webkitRTCPeerConnection to access it. Our first goal is to create two live videos on the same page, one with data directly from your webcam and one with data from a locally created WebRTC connection. It looks like this:

<video id="yours" autoplay></video> <video id="theirs" autoplay></video>

Create a main.js file that encapsulates the userMedia and RTCPeerConnection objects of each browser:

function hasUserMedia() { navigator.getUserMedia = navigator.getUserMedia || navigator.msGetUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia; return !! navigator.getUserMedia; } function hasRTCPeerConnection() { window.RTCPeerConnection = window.RTCPeerConnection || window.webkitRTCPeerConnection || window.mozRTCPeerConnection || window.msRTCPeerConnection; return !! window.RTCPeerConnection; }Copy the code

We then need the browser to call the system’s camera API getUserMedia to get the media stream, making sure that the browser’s camera limits are turned on. Chrome can only open the camera in HTTPS or localhost due to security issues. I’m going to show you how to use HTTP to debug locally in development. Okay

var yourVideo = document.getElementById("yours"); var theirVideo = document.getElementById("theirs"); var yourConnection, theirConnection; if (hasUserMedia()) { navigator.getUserMedia({ video: true, audio: false }, stream => { yourVideo.srcObject = stream; If (hasRTCPeerConnection()) {startPeerConnection startPeerConnection(stream); } else {alert(" No RTCPeerConnection API"); } }, err => { console.log(err); })}else{alert(" No userMedia API")}Copy the code

Next, you need to connect to the signaling server

let wsConnection = new WebSocket(url); Wsconnection.onopen = function () {console.log(" Signaling server connected successfully "); resolve(wsConnection); Wsconnection. onclose = function (evt) {message.error(" connection failed, please close the page and re-enter ")}; wsConnection.onerror = function (evt) { console.log("socket error", evt); }; Wsconnection. onMessage = function (evt) {wsconnection. onMessage = function (evt) {Copy the code

If nothing goes wrong, you should now see a video on the page.

The next step is to implement the startPeerConnection method to establish the ICE communication path required for the transmission of video data. Here we take Chrome as an example:

Function startPeerConnection(stream) {var config = {'iceServers': [{'url': 'stun:stun.services.mozilla.com' }, { 'url': 'stun:stunserver.org' }, { 'url': 'stun:stun.l.google.com:19302' }] }; yourConnection = new RTCPeerConnection(config); theirConnection = new RTCPeerConnection(config); yourConnection.onicecandidate = function(e) { if (e.candidate) { theirConnection.addIceCandidate(new RTCIceCandidate(e.candidate)); } } theirConnection.onicecandidate = function(e) { if (e.candidate) { yourConnection.addIceCandidate(new RTCIceCandidate(e.candidate)); }}}Copy the code

We used this function to create two connection objects. In config, you can specify any ICE server you want, although some browsers do not need to configure the default ICE server, but it is recommended to add these. Next, we have the SDP handshake. Since we are communicating on the same page, we can exchange candidate objects directly from both sides, but on different pages, an additional server may be required to facilitate the exchange process.

4: Establish SDP Offer and SDP Answer

function startPeerConnection(stream) { var config = { 'iceServers': [{ 'url': 'stun:stun.services.mozilla.com' }, { 'url': 'stun:stunserver.org' }, { 'url': 'stun:stun.l.google.com:19302' }] }; yourConnection = new RTCPeerConnection(config); theirConnection = new RTCPeerConnection(config); yourConnection.onicecandidate = function(e) { if (e.candidate) { theirConnection.addIceCandidate(new RTCIceCandidate(e.candidate)); } } theirConnection.onicecandidate = function(e) { if (e.candidate) { yourConnection.addIceCandidate(new RTCIceCandidate(e.candidate)); }} / / his own produced an offer yourConnection. CreateOffer (). Then (offer = > {yourConnection. SetLocalDescription (offer); . / / the other party to receive this offer theirConnection setRemoteDescription (offer); . / / the other to create a answer theirConnection createAnswer (). Then (answer = > {theirConnection. SetLocalDescription (answer); . / / his own to receive an answer yourConnection setRemoteDescription (answer); })}); }Copy the code

Just like the CONNECTION of ICE, we do the SDP handshake on the same page, so we do not need to exchange offer and answer through any other means of communication, and can directly assign values. If you need to exchange between two different pages, you need an extra server to help you do this, using websockets or other means.

5: Join the video stream

Now that we have a reliable video data channel, the next step is to add data flow to the channel. WebRTC directly encapsulates the interface to join a video stream for us. When a video stream is added, the browser on the other side will tell the user via onAddStream that a video stream is joining the channel.

yourConnection.addStream(stream);
theirConnection.onaddstream = function(e) {
    theirVideo.srcObject = e.stream;
}
Copy the code

6: Final effect

7: Upload video recording to the server

Because the company’s project needs to replay the content of both parties’ videos, we have to upload the video recording to the server. Now we mainly talk about how to achieve this

Mainly through this API to implement MediaRecorder(), this method will return the corresponding stream in real time, we just need to save when stop recording conversion upload server

Two methods start and stop

start()

To start recording media, you can set a parameter that will split the recorded media into individual blocks instead of recording a very large block of content as the default.

stop()

Stop recording, and the DataAvailable event is raised to return recorded data that stores the Blob contents

ondataavailable

When recording ends, the event object returns recorded media data

Get the user media access, create media recorded object: / / get the user permissions, media video words parameters {audio: true, video: true} the navigator. MediaDevices. GetUserMedia ({audio: }). Then (stream => {this.recorder = new window.mediaRecorder (stream); this.bindEvents(); }, error => {alert(' error, please make sure browser is allowed to obtain recording rights '); }); Video recording, the literatures to the enclosing chunks bindEvents () {this. Recorder. Ondataavailable = this. GetRecordingData; }, getRecordingData (e) {// Recorded data this.chunks.push(e.ata); }, // save video data saveRecordingData () {let blob = new blob (this.chunks, {'type' : 'video/webm' }), videoStream = URL.createObjectURL(blob); this.chunkList.push({stream: videoStream}); this.chunks = []; },Copy the code

Upload to the server, upload bloB objects to the server as formData, and that’s it.

8:

Fault 1: The Chrome camera can only be opened in HTTPS or localhost due to security issues. So how do you debug on multiple computers on the Intranet

Solution 1: chrome shortcut right click – > properties – > target in the input box an extra – unsafely – treat – insecure – origin – as – secure = “http://10.250.199.175:8881” IP address into their corresponding IP address

Problem 2: A video call containing echoes, a self-articulated voice, requires a video label to be muted via a fraternal attribute

Fault 3: During video recording uploading, you need to clear the data in the storage stream after each upload. Otherwise, the file uploaded to the server will be large

9: communication

If you don’t understand anything, you can add my wechat to communicate