Purely their own understanding and scraped to achieve ha ha

The basic idea of web video chat

1. Obtain camera information using a browser

2. Obtain the video information of the camera

3. Encode the video information

4. The server forwards information

5. The client decodes the data for playback

The basic idea of web broadcast

1. With video chat, simple live broadcast means that the server side forwards and pushes the client side

Problems encountered

1. For a simple video chat 1 to 1 can be achieved without caton convection of splicing can direct the operation, there is a problem is the video information section alone play a failure after subsequent lookups, lack of key frames, I understanding of here is limited, so the problem can’t meet halfway when live in users.

2. Solved the first problem: frames obtained by some other means can be played independently, but there is a problem with the client receiving data through the SRC switch load, that is, the lag is obvious switch data load.

3. The solution to the second problem is to get the last picture of the current play when cutting SRC and set it as the preview picture of the next play. This is the solution I have found here so far.

implementation

Server: NETTY (websocket)

Client browser, video playback video.js

Transport protocol JSON

Part of the code

1. Send videos 1 to 1

try { var options = {mimeType: mimeType}; mediaRecorder = new MediaRecorder(stream, options); } catch (e) { console.error('Exception while creating MediaRecorder: ' + e); log('Exception while creating MediaRecorder: ' + e); alert('Exception while creating MediaRecorder: ' + e + '. mimeType: ' + options.mimeType); return; } // There is a trigger that gets a mediaRecorder.start(1000) every second; / / to get into the data can be turned the base64 code by websocket push forward to the server for the mediaRecorder. Ondataavailable = handleDataAvailable; Function handleDataAvailable(event) {if (event.data && event.data.size > 0) {console.log(' Sending data... '); var reader = new FileReader(); reader.readAsDataURL(event.data); reader.onload = function (e) { //console.info("videoStr:"+reader.result); SendVideo (reader.result); sendVideo(reader.result); //window.URL.revokeObjectURL(localVideoPlayer.src); } //socket.send(event.data); }}Copy the code

2. Receive videos 1 to 1

Var index=0; var successStream; var oSourceBuffer, oMediaSource; var mimeCodec='video/webm; codecs="vp8,opus"'; if ('MediaSource' in window && MediaSource.isTypeSupported(mimeCodec)) { console.log("oMediaSource"); oMediaSource = new MediaSource(); var url = URL.createObjectURL(oMediaSource); remoteVideoVid.src = url; oMediaSource.addEventListener('sourceopen', sourceOpen); } else { console.log("The Media Source Extensions API is not supported.") } function sourceOpen () { console.log("oMediaSource2:"+this.readyState); // open oSourceBuffer = oMediaSource.addSourceBuffer(mimeCodec); oSourceBuffer.addEventListener('updateend', streamPlay); }; function streamPlay(e) { remoteVideoVid.play(); } function remotStream(stream){if (oSourceBuffer) {if (oSourceBuffer) { oSourceBuffer.appendBuffer(dataURItoArray(stream)); } else { console.log('no init sourceBuffer'); }}Copy the code

3. Processing and distribution of live push

SetInterval ("stopSend()",1000); setInterval("stopSend()",1000); Function stopSend(){console.log("stop: "+sendIndex); if(mediaRecorder){ mediaRecorder.stop(); mediaRecorder.start(); console.log("testRequestData"); } else {the console. The log (" noStop: "+ sendIndex); }}Copy the code

4. Processing of live reception

remoteVideoPlayer2.on('ended',function(e) { console.log("ended2:"+videoStart+"_"+index); DrawImage (remoteVideoVid2, 0, 0, 400, 300); drawImage(remoteVideoVid2, 0, 0, 400, 300) let src=canvas.get()[0].toDataURL("image/jpeg"); remoteVideoPlayer2.poster(src); Remotevideoplayer2.src (new_source.slice(index-1)); }Copy the code

5. The server forwards information

/ / netty forward information ChannelManager. GetChannel (message. GetAccept () getUserCode ()) .writeAndFlush(newTextWebSocketFrame(this.getMapper(). writeValueAsString(message)));Copy the code

The effect

Welcome star and suggestions