“Perception: 🌟🌟🌟🌟🌟“

“Taste: Xinjiang stir-fried rice noodles“

“Cooking time: 10min“

This article has been included in the front-end canteen namesake warehouse Github github.com/Geekhyt, welcome to the canteen, if you think the food and wine is delicious, reward a Star for the canteen boss is a great encouragement.

Through the study of the last two series of columns, we have a preliminary understanding of the front-end audio and video and WebRTC, it is time to type code to implement a Demo to truly feel the charm of WebRTC real-time communication. If you haven’t seen it, please move on:

  • Front-end audio and video terms
  • A preliminary study of WebRTC for front-end audio and video

RTCPeerConnection

The RTCPeerConnection class is the core class for real-time interactive audio and video systems using WebRTC in the browser. It represents a local computer to a remote WebRTC connection. This interface provides the implementation of methods to create, hold, monitor, and close connections.

To learn more about this class, go to this link, developer.mozilla.org/zh-CN/docs/…

In fact, if you have done socket development, you will understand RTCPeerConnection more easily, which is actually an enhanced version of socket.

In the first exploration of WebRTC of front-end Audio and Video in the last series of columns, we learned the communication principle of WebRTC. In the real scene, media negotiation, network negotiation, setting up signaling server and other operations need to be carried out. I drew a picture to summarize the communication process of WebRTC as follows:

For the sake of simple understanding of RTCPeerConnection, we will not consider the problem of developing and setting up a signaling server. Simply, we will try to simulate audio and video communication between two ends in the same page.

Before we do that, let’s take a look at some of the apis we’ll be using and the steps we take to establish a connection with WebRTC.

The relevant API

  • The RTCPeerConnection interface represents a WebRTC connection from the local computer to the remote end. This interface provides the implementation of methods to create, hold, monitor, and close connections.

  • Pc. createOffer Creates the proposal Offer method, which returns the SDP Offer information.

  • Pc. setLocalDescription Sets the description of the local SDP.

  • Pc. setRemoteDescription Sets the description of the remote SDP, that is, the SDP data sent by the peer.

  • Pc. createAnswer Creates the Answer method of the reply. This method returns the Answer information of SDP.

  • RTCIceCandidate WebRTC network information (IP, port, etc.)

  • PC. AddIceCandidate PC connects to addIceCandidate information of the addIceCandidate, that is, add network information of the addIceCandidate.

Procedure For establishing a WebRTC connection

  • 1. Create an RTCPeerConnection object for both ends of the connection and add a local stream to the RTCPeerConnection object.

  • 2. Obtain local media description (SDP) and exchange the SDP with the peer end.

  • 3. Obtain network information (Candidate, IP address, and port) and exchange with the remote end.

Demo of actual combat

First, we add video elements and control buttons, and introduce adplater.js to fit each browser.

<! DOCTYPEhtml>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="Width = device - width, initial - scale = 1.0">
 <title>Demo</title>  <style>  video {  width: 320px;  }  </style> </head> <body>  <video id="localVideo" autoplay playsinline></video>  <video id="remoteVideo" autoplay playsinline></video>   <div>  <button id="startBtn">Open local Video</button>  <button id="callBtn">Establish a connection</button>  <button id="hangupBtn">disconnect</button>  </div>  <! -- Scripts for different browser apis -->  <script src="https://webrtc.github.io/adapter/adapter-latest.js"></script>  <script src="./webrtc.js"></script> </body> </html> Copy the code

Then, we define the objects we will use.

// Local and remote streams
let localStream;
let remoteStream;

// Local and remote connection objects
let localPeerConnection; let remotePeerConnection;  // Local and remote video const localVideo = document.getElementById('localVideo'); const remoteVideo = document.getElementById('remoteVideo');  // Set constraints const mediaStreamConstraints = {  video: true }  // Set only videos to be exchanged const offerOptions = {  offerToReceiveVideo: 1 } Copy the code

Next, register events for the button and implement the associated business logic.

function startHandle() {
    startBtn.disabled = true;
    // 1. Obtain local audio and video streams
    // Call the getUserMedia API to get the audio and video streams
    navigator.mediaDevices.getUserMedia(mediaStreamConstraints)
 .then(gotLocalMediaStream)  .catch((err) = > {  console.log('getUserMedia mistakes', err);  }); }  function callHandle() {  callBtn.disabled = true;  hangupBtn.disabled = false;   // Video track  const videoTracks = localStream.getVideoTracks();  // Audio track  const audioTracks = localStream.getAudioTracks();  // Determine whether the video track has a value  if (videoTracks.length > 0) {  console.log('The equipment used is:${videoTracks[0].label}. `);  }  // Determine whether the audio track has a value  if (audioTracks.length > 0) {  console.log('The equipment used is:${audioTracks[0].label}. `);  }  const servers = null;   // Create the RTCPeerConnection object  localPeerConnection = new RTCPeerConnection(servers);  // Listen for returned candidates  localPeerConnection.addEventListener('icecandidate', handleConnection);  // Listen to the ICE status change  localPeerConnection.addEventListener('iceconnectionstatechange', handleConnectionChange)   remotePeerConnection = new RTCPeerConnection(servers);  remotePeerConnection.addEventListener('icecandidate', handleConnection);  remotePeerConnection.addEventListener('iceconnectionstatechange', handleConnectionChange);  remotePeerConnection.addEventListener('track', gotRemoteMediaStream);   // Add the audio and video streams to the RTCPeerConnection object  // Note: The addStream method is no longer recommended in the new protocol. The addTrack method should be used instead  // localPeerConnection.addStream(localStream);  // Traverse all orbits of the local Earth stream  localStream.getTracks().forEach((track) = > {  localPeerConnection.addTrack(track, localStream)  })   // 2. Exchange media description information  localPeerConnection.createOffer(offerOptions)  .then(createdOffer).catch((err) = > {  console.log('createdOffer mistakes', err);  }); }  function hangupHandle() {  // Close the connection and set it to null  localPeerConnection.close();  remotePeerConnection.close();  localPeerConnection = null;  remotePeerConnection = null;  hangupBtn.disabled = true;  callBtn.disabled = false; }  // getUserMedia obtains the stream, displays the audio and video stream and saves it to localStream function gotLocalMediaStream(mediaStream) {  localVideo.srcObject = mediaStream;  localStream = mediaStream;  callBtn.disabled = false; }  function createdOffer(description) {  console.log('SDP :\n returned by local offer creation${description.sdp}`)  // Set the description locally and send it to the remote end  // Save the offer locally  localPeerConnection.setLocalDescription(description)  .then((a)= > {  console.log('local Local description set successfully ');  }).catch((err) = > {  console.log('Local failed to set local description', err)  });  // The remote end sets the local description to the remote description  // The remote end saves the offer  remotePeerConnection.setRemoteDescription(description)  .then((a)= > {  console.log('remote Succeeded in setting remote description ');  }).catch((err) = > {  console.log('Remote Failed to set remote description', err);  });  // Create an answer on the remote end  remotePeerConnection.createAnswer()  .then(createdAnswer)  .catch((err) = > {  console.log('Error in creating reply remotely', err);  }); }  function createdAnswer(description) {  console.log('SDP of the remote Answer :\n${description.sdp}`)  // The remote set the local description and send it to the local  // Save the answer on the remote end  remotePeerConnection.setLocalDescription(description)  .then((a)= > {  console.log('Remote set local description successfully');  }).catch((err) = > {  console.log('Remote failed to set local description', err);  });  // Set the remote reply description to the remote description locally  // Save the answer locally  localPeerConnection.setRemoteDescription(description)  .then((a)= > {  console.log('Local Remote description set successfully');  }).catch((err) = > {  console.log('Local failed to set remote description', err);  }); }  // 3. Establish a connection between the two ends function handleConnection(event) {  // Get the RTCPeerConnection object that triggers the icecandiDate event  // Get the specific Candidate  const peerConnection = event.target;  const iceCandidate = event.candidate;   if (iceCandidate) {  // Create the RTCIceCandidate object  const newIceCandidate = new RTCIceCandidate(iceCandidate);  // Obtain the peer RTCPeerConnection  const otherPeer = getOtherPeer(peerConnection);   // Add the locally acquired Candidate to the remote RTCPeerConnection object  // For simplicity, instead of sending candidates through the signaling server, addIceCandidate is used directly to exchange Candidate information  otherPeer.addIceCandidate(newIceCandidate)  .then((a)= > {  handleConnectionSuccess(peerConnection);  }).catch((error) = > {  handleConnectionFailure(peerConnection, error);  });  } }  // 4. Display remote media streams function gotRemoteMediaStream(event) {  if(remoteVideo.srcObject ! == event.streams[0]) {  remoteVideo.srcObject = event.streams[0];  remoteStream = event.streams[0];  console.log('Remote starts accepting remote streams')  } } Copy the code

Finally, you need to register some Log functions and utility functions.

function handleConnectionChange(event) {
    const peerConnection = event.target;
    console.log('ICE state change event: ', event);
    console.log(`${getPeerName(peerConnection)} ICE state: ` + `${peerConnection.iceConnectionState}. `);
}
 function handleConnectionSuccess(peerConnection) {  console.log(`${getPeerName(peerConnection)}AddIceCandidate successful `); }  function handleConnectionFailure(peerConnection, error) {  console.log(`${getPeerName(peerConnection)}AddIceCandidate error: \ n `+ `${error.toString()}. `); }  function getPeerName(peerConnection) {  return (peerConnection === localPeerConnection) ? 'localPeerConnection' : 'remotePeerConnection'; }  function getOtherPeer(peerConnection) {  return (peerConnection === localPeerConnection) ? remotePeerConnection : localPeerConnection; } Copy the code

In fact, when you are familiar with the whole process, you can extract and encapsulate all Log functions. In order to facilitate you to understand the whole process of WebRTC connection more easily in the process of reading the code, there is no extraction.

If all goes well at this point, you have successfully set up a WebRTC connection. The effect is as follows:

(Grabs the year of the Rat penguin at the table)

reference

  • “Build audio and video live broadcasting system from 0” Li Chao
  • React+Flutter+Go Combat by Kun Shaojun
  • Developer.mozilla.org/zh-CN/docs/…

❤️ love triple strike

1. If you think the food and wine in the canteen are palatable, you can click “like” to support it. Your “like” is my biggest motivation.

2. Pay attention to the front canteen of the public account, “Eat every meal!”

3. Like, comment, retweet === urge more!