Use JS+socket. IO +WebRTC+ nodeJS + Express to build a simple remote video chat

directory

WebRTC

Code principle and flow

Source: [email protected]: DieHunter/myCode. Git repository: myCode/videoSteam

The front end

Attach the HTML and CSS first

The complete socket. Js

Full userlist.js (create online list of users, add invite events, initialize chat rooms)

Problems encountered

Optimized complete video.js

The service side

A full server. Js

Implementation effect

Note:

WebRTC

Web instant messaging, short for Web Real-Time Communication, supports peer-to-peer (between browsers) video and audio transmission, and ensures transmission quality. It can be sent to the local audio tag, video tag or another browser. In this paper, navigator. MediaDevices, RTCPeerConnection object and Socket + Node are used to build remote real-time video chat function. There is a deficiency in this paper, which will be discussed later.

Related documentation: MediaDevices WebRTC API

Refer to the article: https://rtcdeveloper.com/t/topic/13777

Code principle and flow

Source:Gitee.com/DieHunter/m…

The front end

To achieve point-to-point will need to use the socket long link from the server to pager this is my previous a simple use of the socket small case: blog.csdn.net/time_____/a… **socket. IO **. Here I divide front-end JS into three parts, namely socket.js (socket-related operations), userlist. js (page operations),video.js (video chat).

Attach the HTML and CSS first

<! DOCTYPEhtml>
<html lang="en">

<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="Width = device - width, initial - scale = 1.0">
    <meta http-equiv="X-UA-Compatible" content="ie=edge">
    <title>Document</title>
    <link rel="stylesheet" href="./style/main.css">
    <script src="./js/socket.io.js"></script>
    <script src="./js/socket.js"></script>
    <script src="./js/userList.js"></script>
    <script src="./js/video.js"></script>
</head>

<body>
    <div id="login" hidden class="loginBox">
        <input id="userName" autocomplete="false" class="userName" type="text" placeholder="Please enter English user name">
        <button id="submit">submit</button>
    </div>
    <div id="chatBox" class="chatBox" hidden>
        <h1 id="myName" class="myName"></h1>
        <ul id="userList" class="userList"></ul>
    </div>
    <div id="videoChat" hidden class="videoChat">
        <button id="back" hidden>The end of the</button>
        <video id="myVideo" src="" class="myVideo"></video>
        <video id="otherVideo" src="" class="otherVideo"></video>
    </div>
    <script>
        checkToken()

        function checkToken() { // Check whether the user already has a user name
            if (localStorage.token) {
                login.hidden = true
                chatBox.hidden = false
                initSocket(localStorage.token) // Initialize the socket connection
            } else {
                login.hidden = false
                chatBox.hidden = true
                submit.addEventListener('click'.function (e) {
                    initSocket(userName.value) // Initialize the socket connection}}})</script>
</body>

</html>
Copy the code

* {
    margin: 0;
    padding: 0;
}

.loginBox {
    width: 300px;
    height: 200px;
    margin: 50px auto 0;
}

.userName..loginBox button {
    width: 300px;
    height: 60px;
    border-radius: 10px;
    outline: none;
    font-size: 26px;
}

.userName {
    border: 1px solid lightcoral;
    text-align: center;
}

.loginBox button {
    margin-top: 30px;
    display: block;
}

input::placeholder {
    font-size: 26px;
    text-align: center;
}

.chatBox {
    width: 200px;
    margin: 50px auto 0;
    position: relative;
}

.myName {
    position: absolute;
    top: 0;
    left: 0;
    width: 100%;
    height: 50px;
    font-size: 40px;
    text-align: center;
    line-height: 50px;
    background: lightcoral;
}

.userList {
    height: 500px;
    width: 100%;
    padding-top: 50px;
    overflow-y: scroll;
    list-style: none;
}

.userList>li {
    background: lightblue;
    height: 50px;
    font-size: 20px;
    line-height: 50px;
    text-align: center;
}

.videoChat {
    background: lightgreen;
    width: 500px;
    height: 400px;
    margin: 50px auto 0;
}

.videoChat button {
    width: 500px;
    height: 60px;
    border-radius: 10px;
    outline: none;
    float: left;
    font-size: 26px;
}

.myVideo..otherVideo{
    width: 250px;
    height: 250px;
    float: left;
    overflow: hidden;
}
Copy the code

Overall effect

Socket.js first establishes a socket connection, adds connection and disconnect events

let socket // for other pages to call

function initSocket(token) {// Get the id entered by the user and pass it to the server
    socket = io('http://127.0.0.1:1024? token=' + token, {
        autoConnect: false
    });
    socket.open();
    socket.on('open', socketOpen); // Connect to login
    socket.on('disconnect', socketClose); // The connection is down
}

function socketClose(reason) { // Close the socket actively or passively
    console.log(reason)
    localStorage.removeItem("token")}function socketOpen(data) { / / open the socket
    if(! data.result) {// The connection is broken when the server finds the same id
        console.log(data.msg)
        return;
    }
createChatList(data) // Create a user list
    localStorage.setItem('token', data.token)
    login.hidden = true
    chatBox.hidden = false
    videoChat.hidden = true
    myName.textContent = localStorage.token
}
Copy the code

Then add a few event listeners to the socket

    socket.on('dataChange', createChatList); // Add new staff
    socket.on('inviteVideoHandler', inviteVideoHandler); // Invited video
    socket.on('askVideoHandler', askVideoHandler); // Video invitation result
    socket.on('ice', getIce); // Receive ICE from the server
    socket.on('offer', getOffer); // Receive the offer from the server
    socket.on('answer', getAnswer); // Receive answer from the server
    socket.on('break', stopVideoStream) // Hang up the video call
Copy the code

If the user receives an invitation, a confirmation dialog box is displayed and the result is returned to the recipient

function inviteVideoHandler(data) { // execute when the user is invited
    let allow = 0
    if (isCalling) {
        allow = -1 // The line is busy
    } else {
        let res = confirm(data.msg);
        if (res) {
            allow = 1
            startVideoChat(data.token) // The user clicks Agree to initiate the video chat
            localStorage.setItem('roomNo', data.roomNo) // Save the room number
        }
    }
    socket.emit('askVideo', {
        myId: localStorage.token,
        otherId: data.token,
        type: 'askVideo',
        allow
    });
}
Copy the code

When receiving the return invitation result, the back end creates the video chat room and initializes the chat room

function askVideoHandler(data) { // Get the reply from the invited user
    console.log(data.msg)
    if (data.allow == -1) return / / on the phone
    if (data.allow) {
        localStorage.setItem('roomNo', data.roomNo) // Save the room number
        startVideoChat(data.token)
    }
}
Copy the code

When the user hangs up

function breakVideoConnect(e) {
    console.log(localStorage.getItem('roomNo'))
    socket.emit('_break', {
        roomNo: localStorage.getItem('roomNo')}); }Copy the code

The complete socket. Js

let socket // for other pages to call

function initSocket(token) {// Get the id entered by the user and pass it to the server
    socket = io('http://127.0.0.1:1024? token=' + token, {
        autoConnect: false
    });
    socket.open();
    socket.on('open', socketOpen); // Connect to login
    socket.on('disconnect', socketClose); // The connection is down
    socket.on('dataChange', createChatList); // Add new staff
    socket.on('inviteVideoHandler', inviteVideoHandler); // Invited video
    socket.on('askVideoHandler', askVideoHandler); // Video invitation result
    socket.on('ice', getIce); // Receive ICE from the server
    socket.on('offer', getOffer); // Receive the offer from the server
    socket.on('answer', getAnswer); // Receive answer from the server
    socket.on('break', stopVideoStream) // Hang up the video call
}

function socketClose(reason) { // Close the socket actively or passively
    console.log(reason)
    localStorage.removeItem("token")}function socketOpen(data) { / / open the socket
    if(! data.result) {// The connection is broken when the server finds the same id
        console.log(data.msg)
        return;
    }
    createChatList(data) // Create a user list
    localStorage.setItem('token', data.token)
    login.hidden = true
    chatBox.hidden = false
    videoChat.hidden = true
    myName.textContent = localStorage.token
}

function inviteVideoHandler(data) { // execute when the user is invited
    let allow = 0
    if (isCalling) {
        allow = -1 // The line is busy
    } else {
        let res = confirm(data.msg);
        if (res) {
            allow = 1
            startVideoChat(data.token) // The user clicks Agree to initiate the video chat
            localStorage.setItem('roomNo', data.roomNo) // Save the room number
        }
    }
    socket.emit('askVideo', {
        myId: localStorage.token,
        otherId: data.token,
        type: 'askVideo',
        allow
    });
}

function askVideoHandler(data) { // Get the reply from the invited user
    console.log(data.msg)
    if (data.allow == -1) return / / on the phone
    if (data.allow) {
        localStorage.setItem('roomNo', data.roomNo) // Save the room number
        startVideoChat(data.token)
    }
}

function breakVideoConnect(e) {
    console.log(localStorage.getItem('roomNo'))
    socket.emit('_break', {
        roomNo: localStorage.getItem('roomNo')}); }Copy the code

Full userlist.js (create online list of users, add invite events, initialize chat rooms)

function createChatList(data) { // Create a user list
    console.log(data.msg)
    let userData = data.userIds
    let userList = document.querySelector('#userList')
    if (userList) {
        userList.remove()
        userList = null
    }
    userList = createEle('ul', {}, {
        id: 'userList'.className: 'userList'
    })
    chatBox.appendChild(userList)
    for (let key in userData) {
        if(userData[key] ! =localStorage.token) {
            var li = createEle('li', {}, {
                textContent: userData[key]
            })
            li.addEventListener('click', videoStart)
            userList.appendChild(li)
        }
    }
}

function createEle(ele, style, attribute) { // Add tags, set attributes and styles
    let element = document.createElement(ele)
    if (style) {
        for (let key instyle) { element.style[key] = style[key]; }}if (attribute) {
        for (let key inattribute) { element[key] = attribute[key]; }}return element
}

function videoStart(e) { // When the user clicks on a user in the list, the invitation is sent to the server
    socket.emit('inviteVideo', {
        myId: localStorage.token,
        otherId: this.textContent,
        type: 'inviteVideo'
    });
}

function startVideoChat(otherId) { // Initialize the video chat
    videoChat.hidden = false
    login.hidden = true
    chatBox.hidden = true
    localStorage.setItem('otherId', otherId) // Save the id of the peer
    startVideoStream()
}
Copy the code

video.js

Initializes the media objects, and to save the Stream as a global, here due to the navigator. MediaDevices. GetUserMedia is asynchronous method, need to be synchronized, first get the Stream, and then for subsequent operations

async function createMedia() { // Create a local media stream
    if(! stream) { stream =await navigator.mediaDevices.getUserMedia({
            audio: true.video: true})}console.log(stream)
    let video = document.querySelector('#myVideo');
    video.srcObject = stream; // Output media stream to local video to display itself
    video.onloadedmetadata = function (e) {
        video.play();
    };
    createPeerConnection()
}
Copy the code

After the stream is created, the RTCPeerConnection is initialized to establish a video connection. The peer also needs to obtain the RTCPeerConnection synchronously. After the peer obtains the RTCPeerConnection, the peer sends the offer to the peer


async function createPeerConnection() { // Synchronize the initialization description file and add events
    if(! peer) { peer =new RTCPeerConnection()
    }
    await stream.getTracks().forEach(async track => {
        await peer.addTrack(track, stream); // Attach the local stream to the peer
    });
    // await peer.addStream(stream); // The old method (attaching the local stream to the peer)
    peer.addEventListener('addstream', setVideo) When the peer receives other streams, another video is displayed to display the peer
    peer.addEventListener('icecandidate', sendIce) // When a candidate is found, send it to the server
    peer.addEventListener('negotiationneeded', sendOffer) This method is triggered only when the agreed negotiation has been completed
}
Copy the code

When the stream sent by the other party is received, that is, when the addStream event is triggered, the video stream of the other party is put into the local video through setVideo

function setVideo(data) { // Play the other party's video stream
    console.log(data.stream)
    let back = document.getElementById('back')
    back.hidden = false // Displays the hang up button
    back.addEventListener('click', breakVideoConnect) // Hang up the event
    isCalling = true // The line is busy
    let video = document.querySelector('#otherVideo');
    video.srcObject = data.stream;
    video.onloadedmetadata = function (e) {
        video.play();
    };
}
Copy the code

Create an offer, save the local offer, and send the offer to the other party

async function sendOffer() { // Synchronously send the offer to the server
    let offer = await peer.createOffer();
    await peer.setLocalDescription(offer); //peer local attached offer
    socket.emit('_offer', {
        streamData: offer
    });
}
Copy the code

After receiving the offer from the peer, save the remote offer. However, there is a small problem. If the peer has not been created, that is, if the peer has been created first, it will send the offer immediately. If you call setRemoteDescription directly, you get an error, so you can use a try catch, or if(! Peer) return

async function getOffer(data) { After receiving the offer, return the answer to the other party
    await peer.setRemoteDescription(data.streamData); //peer Remote attached offer
    sendAnswer()
}


/ / after optimization
async function getOffer(data) { After receiving the offer, return the answer to the other party
    if(! peer)return // Wait for a response. You can also use a try catch
    await peer.setRemoteDescription(data.streamData); //peer Remote attached offer
    sendAnswer()
}
Copy the code

Create an answer, save the local answer, and send the answer to the peer

async function sendAnswer() {
    let answer = await peer.createAnswer();
    await peer.setLocalDescription(answer); //peer Indicates the attached local answer
    socket.emit('_answer', {
        streamData: answer
    });
}
Copy the code

When receiving the answer, save the local answer

async function getAnswer(data) { // After receiving the answer, the peer remotely attached the answer
    await peer.setRemoteDescription(data.streamData);
}
Copy the code

When the peer triggers the icecandiDate event, that is, setLocalDescription is triggered locally, that is, local offer and local answer are saved, the method is triggered

function sendIce(e) { //setLocalDescription sends ICE to each other when triggered
    if (e.candidate) {
        socket.emit('_ice', {
            streamData: e.candidate }); }}Copy the code

If the peer has not been created in the ICE event, the peer will send an offer immediately if the peer has not been created in the ICE event. If the peer has not been created in the ICE event, the peer may not have been created successfully. If you call the addIceCandidate directly, an error will be reported. So you can call it with a try catch, or if(! Peer) return

async function getIce(data) { // Get the ICE of the other party
    var candidate = new RTCIceCandidate(data.streamData)
    await peer.addIceCandidate(candidate)
}

/ / after optimization

async function getIce(data) { // Get the ICE of the other party
    if(! peer)return // Wait for a response. You can also use a try catch
    var candidate = new RTCIceCandidate(data.streamData)
    await peer.addIceCandidate(candidate)
}
Copy the code

Problems encountered

Finally, hang up method, there is a small problem, when hung up, the original stream cannot delete, lead to camera, although there is no call, but the navigation bar will still have a camera icon (not really closed), the next time when they open the transmission flow (stack) in front of, no solution on the net, if have classmate to know, hope to be able to complement optimization, thank you

function stopVideoStream(data) { // Stop transmitting the video stream
    console.log(data.msg)
    stream.getTracks().forEach(async function (track) { // Here we get a video or audio object
        await track.stop();
        await stream.removeTrack(track)
        stream = null
    })
    peer.close();
    peer = null;
    isCalling = false
    videoChat.hidden = true
    login.hidden = true
    chatBox.hidden = false
}
Copy the code

Optimized complete video.js

var stream, peer, isCalling = false // Initialize the stream to be sent, and describe the file, call status
function startVideoStream(e) { // Start transmitting the video stream
    createMedia()
}

function stopVideoStream(data) { // Stop transmitting the video stream
    console.log(data.msg)
    stream.getTracks().forEach(async function (track) { // Here we get a video or audio object
        await track.stop();
        await stream.removeTrack(track)
        stream = null
    })
    peer.close();
    peer = null;
    isCalling = false
    videoChat.hidden = true
    login.hidden = true
    chatBox.hidden = false
}

async function createMedia() { // Create a local media stream
    if(! stream) { stream =await navigator.mediaDevices.getUserMedia({
            audio: true.video: true})}console.log(stream)
    let video = document.querySelector('#myVideo');
    video.srcObject = stream; // Output media stream to local video to display itself
    video.onloadedmetadata = function (e) {
        video.play();
    };
    createPeerConnection()
}

async function createPeerConnection() { // Synchronize the initialization description file and add events
    if(! peer) { peer =new RTCPeerConnection()
    }
    await stream.getTracks().forEach(async track => {
        await peer.addTrack(track, stream); // Attach the local stream to the peer
    });
    // await peer.addStream(stream); // The old method (attaching the local stream to the peer)
    peer.addEventListener('addstream', setVideo) When the peer receives other streams, another video is displayed to display the peer
    peer.addEventListener('icecandidate', sendIce) // When a candidate is found, send it to the server
    peer.addEventListener('negotiationneeded', sendOffer) This method is triggered only when the agreed negotiation has been completed
}

function setVideo(data) { // Play the other party's video stream
    console.log(data.stream)
    let back = document.getElementById('back')
    back.hidden = false // Displays the hang up button
    back.addEventListener('click', breakVideoConnect) // Hang up the event
    isCalling = true // The line is busy
    let video = document.querySelector('#otherVideo');
    video.srcObject = data.stream;
    video.onloadedmetadata = function (e) {
        video.play();
    };
}

async function sendOffer() { // Synchronously send the offer to the server
    let offer = await peer.createOffer();
    await peer.setLocalDescription(offer); //peer local attached offer
    socket.emit('_offer', {
        streamData: offer
    });
}

async function getOffer(data) { After receiving the offer, return the answer to the other party
    if(! peer)return // Wait for a response. You can also use a try catch
    await peer.setRemoteDescription(data.streamData); //peer Remote attached offer
    sendAnswer()
}

async function sendAnswer() {
    let answer = await peer.createAnswer();
    await peer.setLocalDescription(answer); //peer Indicates the attached local answer
    socket.emit('_answer', {
        streamData: answer
    });
}

async function getAnswer(data) { // After receiving the answer, the peer remotely attached the answer
    await peer.setRemoteDescription(data.streamData);
}

function sendIce(e) { //setLocalDescription sends ICE to each other when triggered
    if(! e || ! e.candidate)return
    socket.emit('_ice', {
        streamData: e.candidate
    });
}

async function getIce(data) { // Get the ICE of the other party
    if(! peer)return // Wait for a response. You can also use a try catch
    var candidate = new RTCIceCandidate(data.streamData)
    await peer.addIceCandidate(candidate)
}
Copy the code

The service side

The back end also uses Socketio for communication. First download express, socket.io after NPM initialization

npm i express --save-dev
npm i socket.io --save-dev
Copy the code

It is then imported into server.js

const express = require('express')
const app = express();
const server = require('http').Server(app);
const io = require('socket.io')(server);
Copy the code

And listen on port 1024

server.listen(1024.function () {
    console.log('Socket Open')});Copy the code

Configure the socket to add some events to the socket

io.on('connect'.socket= > {
    let {
        token
    } = socket.handshake.query
    socket.on('disconnect'.(exit) = > { / / socket to disconnect
        delFormList(token) // Clear the user
        broadCast(socket, token, 'leave') // Broadcast to other users})});Copy the code

In this way, we have the simplest socket

A full server. Js

const express = require('express')
const app = express();
const server = require('http').Server(app);
const io = require('socket.io')(server);
let userList = {} // List of users, all connected users
let userIds = {} // The user ID list is displayed in the front end
let roomList = {} // Room list, video chat
io.on('connect'.socket= > {
    let {
        token
    } = socket.handshake.query
    socket.on('disconnect'.(exit) = > { / / socket to disconnect
        delFormList(token) // Clear the user
        broadCast(socket, token, 'leave') // Broadcast to other users
    })
    socket.on('inviteVideo', inviteVideoHandler) // Invite the user
    socket.on('askVideo', inviteVideoHandler); // Respond to whether the invitation is successful
    if (userList[token]) { // Jump out of the function if you find the same user name
        socket.emit('open', {
            result: 0.msg: token + 'pre-existing'
        });
        socket.disconnect()
        return;
    }
    addToList(token, socket) // Add to userList when user connects
    broadCast(socket, token, 'enter') // Advertising other users, someone joined
});

function addToList(key, item) { // Add to userList
    item.emit('open', {
        result: 1.msg: 'You have joined the chat',
        userIds,
        token: key
    });
    userList[key] = item
    userIds[key] = key
}

function delFormList(key) { // Delete the user
    delete userList[key];
    delete userIds[key]
}

function broadCast(target, token, type) { // Broadcast function
    let msg = 'Join the chat'
    if(type ! = ='enter') {
        msg = 'Leave the chat'
    }
    target.broadcast.emit('dataChange', {
        result: 1.msg: token + msg,
        userIds
    });
}

function inviteVideoHandler(data) { // Invite method
    let {
        myId,
        otherId,
        type,
        allow
    } = data, msg = 'Inviting you into the chat room', event = 'inviteVideoHandler', roomNo = otherId // The default room number is the inviter id
    if (type == 'askVideo') {
        event = 'askVideoHandler'
        if (allow == 1) {
            addRoom(myId, otherId)
            roomNo = myId // Save the room number
            msg = 'Accepted your invitation.'
        } else if (allow == -1) {
            msg = 'On line'
        } else {
            msg = 'Refused your invitation'
        }
    }
    userList[otherId].emit(event, {
        msg: myId + msg,
        token: myId,
        allow,
        roomNo
    });
}

async function addRoom(myId, otherId) { // Add the user to the video chat room after the user agrees, only do 1 to 1 chat function
    roomList[myId] = [userList[myId], userList[otherId]]
    startVideoChat(roomList[myId])
}

function startVideoChat(roomItem) { // Video chat initialization
    for (let i = 0; i < roomItem.length; i++) {
        roomItem[i].room = roomItem
        roomItem[i].id = i
        roomItem[i].on('_ice', _iceHandler)
        roomItem[i].on('_offer', _offerHandler)
        roomItem[i].on('_answer', _answerHandler)
        roomItem[i].on('_break', _breakRoom)

    }
}

function _iceHandler(data) { // The user sends ice to the server, which forwards it to another user
    let id = this.id == 0 ? 1 : 0 // Determine whether the user is one or the other
    this.room[id].emit('ice', data);
}

function _offerHandler(data) { // The user sends the offer to the server, which forwards it to another user
    let id = this.id == 0 ? 1 : 0
    this.room[id].emit('offer', data);
}

function _answerHandler(data) { // The user sends the answer to the server, which forwards it to another user
    let id = this.id == 0 ? 1 : 0
    this.room[id].emit('answer', data);
}

function _breakRoom(data) { // Hang up the chat
    for (let i = 0; i < roomList[data.roomNo].length || 0; i++) {
        roomList[data.roomNo][i].emit('break', {
            msg: 'Chat down'
        });
    }
}
server.listen(1024.function () {
    console.log('Socket Open')});Copy the code

Implementation effect

Print a stream before we send it and print a stream after we receive it. We see that the streams of both parties are swapped, that is, the entire media object is swapped

Test the results with a colleague on two computers

Note:

Front-end project must run under local or HTTPS service, because the navigator. MediaDevices. GetUserMedia needs to be run in safe mode, users need to authorize the camera or audio access was used for the first time, so the computer needs to have related functions

Read here, hope to leave your valuable suggestions

Use JS+socket. IO +WebRTC+ nodeJS + Express to build a simple remote video chat

WebRTC

Code principle and flow

Source:Gitee.com/DieHunter/m…

The front end

Attach the HTML and CSS first

The complete socket. Js

Full userlist.js (create online list of users, add invite events, initialize chat rooms)

Problems encountered

Optimized complete video.js

The service side

A full server. Js

Implementation effect

Note:

Related Posts

NextTick is fully resolved in VUE

Nodejs RESTFUL interface to return common HTTP status codes

Looking back at Javascript: Execute context/scope chain/closure