Introduction to the
Objective To understand the end-to-end communication of WEBRTC
# Use process
git clone https://gitee.com/wjj0720/webrtc.git
cd ./webRTC
npm i
npm run dev
# access 127.0.0.1:3003/test-1.html to demonstrate H5 media stream capture
# visit 127.0.0.1:3003 / local HTML demo RTC local transport
# access 127.0.0.1:3003/p2p.html to demonstrate LAN end-to-end viewing
Copy the codewhat is WebRTC
Web Real-Time Communication (WebRTC) is an API that enables Web browsers to conduct real-time voice and video conversations. Made open source on June 1, 2011 and supported by Google, Mozilla, and Opera, it was incorporated into the W3C recommendation of the World Wide Web ConsortiumCopy the codeGossip: RTP (Real-Time Transport Protocol) A Protocol based on UDP plus HTTP Live Streamin (HLS) An HTTP-based streaming media transmission Protocol implemented by Apple Real Time Messaging Protocol (RTMP) Adobe based on TCP WebRTC Google based on RTPCopy the codeWebRTC components
-
GetUserMedia is responsible for retrieving the user’s local multimedia data
-
RTCPeerConnection establishes P2P connections and transmits multimedia data.
-
RTCDataChannel provides a signaling channel to achieve bidirectional communication
H5 Obtain the media stream
Goal: Turn on the camera to stream media to the page
MediaDevices document
navigator.mediaDevices.getUserMedia({
video: true./ / camera
audio: true / / the microphone
}).then(steam= > {
// srcObject for the video tag
video.srcObject = stream
}).catch(e= > {
console.log(e)
})
Copy the codeRTCPeerConnection
The RTCPeerConnection API provides the implementation of RTCPeerConnection MDN for WebRTC side to create, link, hold, and monitor closed connections
- WebRTC process
For example, A<=>B creates A P2P connection: 1. Create RTCPeerConnection instance peerA 2. Add your own local media streams (audio, video) to the instance, peera.addStream 3. Listen for media stream peera.onAddStream 4 from the remote end. [SDP Offer] is created to initiate a new WebRTC connection peera.createOffer 5 to the remote (at this point the remote is also called the candidate)) peer. 6. After receiving the answer, go to [STUN] service to get its OWN IP address, and send it to the caller B through the signaling service: 1. Received a notification from the signaling service to create RTCPeerConnection peerB, 2. You also need to add your own local media stream to communication peerb.addStream 3. Listen for media stream peera.onAddStream 4 from the remote end. Create [SDP offer] peera.createAnswer 5. Send the Answer to the caller through [signaling server] 6. After receiving the IP address of the caller, also go to [STUN] service to get your IP address and pass it to the callerCopy the code- Signaling service
Signaling server: The system responsible for call establishment, Supervision and Teardown in webRTC is needed: webRTC is a P2P connection, so how to obtain the information of the other party before the connection, and how to send their own information to the other party, which requires signaling serviceCopy the code- SDP
What is SDP? SDP is completely a session description format -- it doesn't belong to a transport protocol it just uses different appropriate transport protocols, Including session notification protocol (SAP), session initial protocol (SIP), real-time stream protocol (RTSP), MIME extended protocol E-mail and Hypertext Transfer Protocol (HTTP) SDP is a text-based protocol with strong scalability, which makes it have a wide range of applications. SDP in WebRTC SDP does not support negotiation of session content or media encoding. In WEBRTC, SDP is used for the description of media information (encoding and decoding information), and media negotiation is realized by RTPCopy the code- stun
1. What is STUN STUN (Session Traversal UtilitiesforNAT (NAT session traversal application) is a network protocol that allows clients behind a NAT (or multiple NAts) to find out their public address, which type of NAT they are behind, and which Internet port the NAT binds to a local port. This information is used to create UDP communication between two hosts that are behind the NAT router. What is NAT? Network Address Translation (NAT) was proposed in 1994. NAT software is installed on the router when some hosts on the private network already have local IP addresses and want to communicate with hosts on the Internet. A router equipped with NAT software is called a NAT router and can pass through a global IP address. In this way, a small number of public IP addresses represent a large number of private IP addresses when all hosts using local addresses communicate with the outside world, which helps to slow down the exhaustion of available IP address space. 3.WebRTC penetration STUN, TURN, ICE, uPnP, etc. ICE is a combination of STUN and TURN. Webrtc uses this free address provided by Google: https://webrtc.github.io/samples/src/content/peerconnection/trickle-ice/Copy the codeSHOW THE CODE
- The front end
<html lang="zh">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="Width = device - width, initial - scale = 1.0" />
<meta http-equiv="X-UA-Compatible" content="ie=edge" />
<title>end-to-end</title>
</head>
<body>
<div class="page-container">
<div class="message-box">
<ul class="message-list"></ul>
<div class="send-box">
<textarea class="send-content"></textarea>
<button class="sendbtn">send</button>
</div>
</div>
<div class="user-box">
<video id="local-video" autoplay class="local-video"></video>
<video id="remote-video" autoplay class="remote-video"></video>
<p class="title">Online users</p>
<ul class="user-list"></ul>
</div>
<div class="mask">
<div class="mask-content">
<input class="myname" type="text" placeholder="Enter a user name to join the room">
<button class="add-room">join</button>
</div>
</div>
<div class="video-box">
</div>
</div>
<script src="/js/jquery.js"></script>
<script src="/js/socket.io.js"></script>
<script>
// A simple encapsulation
class Chat {
constructor({ calledHandle, host, socketPath, getCallReject } = {}) {
this.host = host
this.socketPath = socketPath
this.socket = null
this.calledHandle = calledHandle
this.getCallReject = getCallReject
this.peer = null
this.localMedia = null
}
async init() {
this.socket = await this.connentSocket()
return this
}
async connentSocket() {
if (this.socket) return this.socket
return new Promise((resolve, reject) = > {
let socket = io(this.host, { path: this.socketPath })
socket.on("connect", () = > {console.log("Connection successful!")
resolve(socket)
})
socket.on("connect_error", e => {
console.log("Connection failed!")
throw e
reject()
})
// The call is accepted
socket.on('answer', ({ answer }) => {
this.peer && this.peer.setRemoteDescription(answer)
})
// Called event
socket.on('called', callingInfo => {
this.called && this.called(callingInfo)
})
// The call was rejected
socket.on('callRejected', () = > {this.getCallReject && this.getCallReject()
})
socket.on('iceCandidate', ({ iceCandidate }) => {
console.log('Remote add iceCandidate');
this.peer && this.peer.addIceCandidate(new RTCIceCandidate(iceCandidate))
})
})
}
addEvent(name, cb) {
if (!this.socket) return
this.socket.on(name, (data) => {
cb.call(this, data)
})
}
sendMessage(name, data) {
if (!this.socket) return
this.socket.emit(name, data)
}
// Get the local media stream
async getLocalMedia() {
let localMedia = await navigator.mediaDevices
.getUserMedia({ video: { facingMode: "user" }, audio: true })
.catch(e= > {
console.log(e)
})
this.localMedia = localMedia
return this
}
// Set the media stream to video
setMediaTo(eleId, media) {
document.getElementById(eleId).srcObject = media
}
// The called party responds
called(callingInfo) {
this.calledHandle && this.calledHandle(callingInfo)
}
/ / create the RTC
createLoacalPeer() {
this.peer = new RTCPeerConnection()
return this
}
// Add the media stream to the communication
addTrack() {
if (!this.peer || !this.localMedia) return
//this.localMedia.getTracks().forEach(track => this.peer.addTrack(track, this.localMedia));
this.peer.addStream(this.localMedia)
return this
}
// Create an SDP offer
async createOffer(cb) {
if (!this.peer) return
let offer = await this.peer.createOffer({ OfferToReceiveAudio: true.OfferToReceiveVideo: true })
this.peer.setLocalDescription(offer)
cb && cb(offer)
return this
}
async createAnswer(offer, cb) {
if (!this.peer) return
this.peer.setRemoteDescription(offer)
let answer = await this.peer.createAnswer({ OfferToReceiveAudio: true.OfferToReceiveVideo: true })
this.peer.setLocalDescription(answer)
cb && cb(answer)
return this
}
listenerAddStream(cb) {
this.peer.addEventListener('addstream', event => {
console.log('AddStream event trigger', event.stream);
cb && cb(event.stream);
})
return this
}
// Listen for candidates to join
listenerCandidateAdd(cb) {
this.peer.addEventListener('icecandidate', event => {
let iceCandidate = event.candidate;
if (iceCandidate) {
console.log('Send candidate to remote'); cb && cb(iceCandidate); }})return this
}
// Check the ICE negotiation process
listenerGatheringstatechange () {
this.peer.addEventListener('icegatheringstatechange', e => {
console.log(Ice in negotiation:, e.target.iceGatheringState);
})
return this
}
/ / close the RTC
closeRTC() {
/ /...}}</script>
<script>
$(function () {
let chat = new Chat({
host: 'http://127.0.0.1:3003'.socketPath: "/websocket".calledHandle: calledHandle,
getCallReject: getCallReject
})
// Update user list view
function updateUserList(list) {$(".user-list").html(list.reduce((temp, li) = > {
temp += `<li class="user-li">${li.name} <button data-calling=${li.calling} data-id=${li.id} class=${li.id === this.socket.id || li.calling ? 'cannot-call' : 'can-call'}> call < / button > < / li > `
return temp
}, ' '))}// Update message li table view
function updateMessageList(msg) {$('.message-list').append(`<li class=${msg.userId === this.socket.id ? 'left' : 'right'}>${msg.user}: ${msg.content}</li>`)}// Join the room
$('.add-room').on('click'.async() = > {let name = $('.myname').val()
if(! name)return
$('.mask').fadeOut()
await chat.init()
// User join event
chat.addEvent('updateUserList', updateUserList)
// Message update event
chat.addEvent('updateMessageList', updateMessageList)
chat.sendMessage('addUser', { name })
})
// Send a message
$('.sendbtn').on('click', () = > {let sendContent = $('.send-content').val()
if(! sendContent)return
$('.send-content').val(' ')
chat.sendMessage('sendMessage', { content: sendContent })
})
/ / video
$('.user-list').on('click'.'.can-call'.async function () {
// Called party information
let calledParty = $(this).data()
if (calledParty.calling) return console.log('The other party is on the line');
// Initial local video
$('.local-video').fadeIn()
await chat.getLocalMedia()
chat.setMediaTo('local-video', chat.localMedia)
chat.createLoacalPeer()
.listenerGatheringstatechange()
.addTrack()
.listenerAddStream(function (stream) {$('.remote-video').fadeIn()
chat.setMediaTo('remote-video', stream)
})
.listenerCandidateAdd(function (iceCandidate) {
chat.sendMessage('iceCandidate', { iceCandidate, id: calledParty.id })
})
.createOffer(function (offer) {
chat.sendMessage('offer', { offer, ... calledParty }) }) })// The call was rejected
function getCallReject() {
chat.closeRTC()
$('.local-video').fadeIn()
console.log('Call rejected');
}
/ / called
async function calledHandle(callingInfo) {
if(! confirm('Whether or not to accept${callingInfo.name}Video call ')) {
chat.sendMessage('rejectCall', callingInfo.id)
return
}
$('.local-video').fadeIn()
await chat.getLocalMedia()
chat.setMediaTo('local-video', chat.localMedia)
chat.createLoacalPeer()
.listenerGatheringstatechange()
.addTrack()
.listenerCandidateAdd(function (iceCandidate) {
chat.sendMessage('iceCandidate', { iceCandidate, id: callingInfo.id })
})
.listenerAddStream(function (stream) {$('.remote-video').fadeIn()
chat.setMediaTo('remote-video', stream)
})
.createAnswer(callingInfo.offer, function (answer) {
chat.sendMessage('answer', { answer, id: callingInfo.id })
})
}
})
</script>
</body>
</html>
Copy the code- The back-end
const SocketIO = require('socket.io')
const socketIO = new SocketIO({
path: '/websocket'
})
let userRoom = {
list: [],
add(user) {
this.list.push(user)
return this
},
del(id) {
this.list = this.list.filter(u= >u.id ! == id)return this
},
sendAllUser(name, data) {
this.list.forEach(({ id }) = > {
console.log('> > > > >', id)
socketIO.to(id).emit(name, data)
})
return this
},
sendTo(id) {
return (eventName, data) = > {
socketIO.to(id).emit(eventName, data)
}
},
findName(id) {
return this.list.find(u= > u.id === id).name
}
}
socketIO.on('connection'.function(socket) {
console.log('Join the connection.', socket.id)
socket.on('addUser'.function(data) {
console.log(data.name, 'Join the room')
let user = {
id: socket.id,
name: data.name,
calling: false
}
userRoom.add(user).sendAllUser('updateUserList', userRoom.list)
})
socket.on('sendMessage', ({ content }) => {
console.log('Forward a message:', content)
userRoom.sendAllUser('updateMessageList', { userId: socket.id, content, user: userRoom.findName(socket.id) })
})
socket.on('iceCandidate', ({ id, iceCandidate }) => {
console.log('Forwarding channel')
userRoom.sendTo(id)('iceCandidate', { iceCandidate, id: socket.id })
})
socket.on('offer', ({id, offer}) => {
console.log('forwarding offer')
userRoom.sendTo(id)('called', { offer, id: socket.id, name: userRoom.findName(socket.id)})
})
socket.on('answer', ({id, answer}) => {
console.log('Accept video');
userRoom.sendTo(id)('answer', {answer})
})
socket.on('rejectCall', id => {
console.log('Forward reject video')
userRoom.sendTo(id)('callRejected')
})
socket.on('disconnect', () = > {// Disconnect and delete
console.log('Disconnected', socket.id)
userRoom.del(socket.id).sendAllUser('updateUserList', userRoom.list)
})
})
module.exports = socketIO
// www.js
const http = require('http')
const app = require('.. /app')
const socketIO = require('.. /socket.js')
const server = http.createServer(app.callback())
socketIO.attach(server)
server.listen(3003, () = > {console.log('server start on 127.0.0.1:3003')})Copy the codeBuild STUN/TURN
I didn’t try because I didn’t have the money to buy a server
Coturn says it’s very easy to build STUN/TURN services
# compiler
cd coturn
./configure --prefix=/usr/local/coturn
sudo make -j 4 && make install
# configuration
listening-port=3478 Specifies the port to listen onExternal - IP = 39.105.185.198# Specify the public IP address of the cloud host
user=aaaaaa:bbbbbb The username and password to access the STUn /turn service
realm=stun.xxx.cn # domain, this must be set
# start
cd /usr/local/coturn/bin turnserver -c .. The/etc/turnserver. Conf trickle - ice https://webrtc.github.io/samples/src/content/peerconnection/trickle-ice according to the requirements of the inside of the input Stun /turn The address, user, and password are as follows :stun or turn The URI value is as follows: turn:stun.xxx.cn The user name is aaaaaa and the password is BBBBBBCopy the codeSTUN parameter transfer
let ice = {"iceServers": [{"url": "stun:stun.l.google.com:19302"}, // No password required
// TURN needs to be defined
{
'url': 'turn: 192.158.29.39:3478? transport=udp'.'credential': 'JZEOEt2V3Qb0y27GRntt2u2PAYA='./ / password
'username': '28224511:1379330808' / / user name
},
{
'url': 'turn: 192.158.29.39:3478? transport=tcp'.'credential': 'JZEOEt2V3Qb0y27GRntt2u2PAYA='.'username': '28224511:1379330808'}}]// Multiple iceServers addresses can be provided, but the RTC selects one for negotiation
// Instantiate the RTC argument to get the IP behind the local wall when appropriate
let pc = new RTCPeerConnection(ice);
/ * / / it is said that these free address can use stun:stun1.l.google.com: 19302 stun:stun2.l.google.com: 19302 stun:stun3.l.google.com: 19302 Stun:stun4.l.google.com: 19302 stun: 23.21.150.121 stun:stun01.sipphone.com stun:stun.ekiga.net stun:stun.fwdnet.net stun:stun.ideasip.com stun:stun.iptel.org stun:stun.rixtelecom.se stun:stun.schlund.de stun:stunserver.org stun:stun.softjoys.com stun:stun.voiparound.com stun:stun.voipbuster.com stun:stun.voipstunt.com stun:stun.voxgratia.org stun:stun.xten.com */
Copy the code