Screen sharing is called DesktopSharing in English, collodly speaking, it is to share the screen of your computer to others, which can be the whole computer screen, application or a certain open web page and so on.

With the in-depth development of the audio and video field, complete functions are produced under the background of the surging demand of users. Whether it is in the learning, life or entertainment scene, screen sharing as a way to achieve interaction is increasingly used in daily life by more and more users:

1. TeamViewer: Control the remote computer, complete the collaboration, etc.

2. Online meeting: Participants only need to view the shared documents on their computer screens, and watch the presentation of documents, etc.

3, online classroom: screen sharing can show the teacher’s courseware, notes, lecture content and other pictures to students to watch;

.

As a result, screen sharing is a derivative feature that has been used successfully in more and more scenarios, so how to achieve screen sharing? In this article we will detail the practice of screen sharing on the Web side.

How to implement screen capture on the Web

Can Web browsers share screens? You can do it on a computer. Screen sharing is divided into two steps: screen capture + streaming.

Screen capture: obtain data to provide data source for streaming media transmission;

Streaming: Transfers audio and video data from one client to another. The current mature scheme is to use WebRTC protocol to provide low latency and anti-weak network capability to ensure the experience;

The WebRTC protocol requires that the stream data provided must be a MediaStream object, so the stream collected by the screen must also be of type MediaStream. Taking Chrome 72 as an example (which varies a bit from browser to browser), the code that captures the screen looks like this:

async function startCapture(displayMediaOptions){
   let captureStream = null;
   try{  
    captureStream = await navigator.mediaDevices.getDisplayMedia(displayMediaOptions);
   }catch(err){  
     console.error("Error: "+ err);
   }
   return captureStream;
}
Copy the code

The effect is as follows:

​

Zegoim.github. IO /express-dem…

Key syntax:

/ / constraints may refer to https://developer.mozilla.org/zhCN/docs/Web/API/MediaDevices/getDisplayMedia let promise = navigator.mediaDevices.getDisplayMedia(constraints);Copy the code

Screen capture compatibility issues and solutions

As a Web front-end development, I believe this issue is of concern to everyone. Currently, the compatibility of screen capture interface is complicated, and screen capture is only supported in the following desktop browsers:

  • Chrome 58 or later
  • Firefox 56 or later
  • Edge 80 or later
  • Safari 13 and later for macOS

Chrome can be further divided into plugins and no plugins.

  • There are plugins: You need to install additional plugins in the browser to take advantage of Chrome’s ability to capture the screen.
  • No add-on: no add-on is required, but Chrome must be 72 or later.

Chrome 74 or later is required to support audio sharing without plugins. Currently, Safari only supports sharing the entire screen. You cannot select apps and browser pages.

After the compatibility of browsers is clear, you can judge the browser. The general idea is to determine the browser type, platform, and version based on navigator. AppVersion, and determine whether the interface exists.

Code is pure manual work, if you don’t want to deal with the browser type and version one by one, ZEGO namely construction technology zego-Express-Engine-Webrtc. js to help you do a good job of internal compatibility, and provide detection interface easy to get started quickly

Click to view: doc-zh.zego.im/article/630…

Browser to achieve custom size sharing

We look at the interface provided by the browser, no matter what type or version of the browser, the smallest granularity can only capture a page, if the page or program to share more sensitive information, do not want to be all shared out, what should we do?

There is no way to do this just by looking at the browser interface, but MediaStream objects can also be retrieved from media labels such as Video or Canvas. If we render the captured streaming media object to the canvas, and then capture the part of the picture we want to share from the canvas, we can achieve the specified size sharing.

The implementation pseudocode is as follows:

/ / get screen capture flow let screenStream = await the navigator. MediaDevices. GetDisplayMedia (displayMediaOptions); Const CTX = Canvas. GetContext ('2d'); // Scroll the render video. Let timer = null; function videoDrawInCanvas(){ timer = setTimeout(async () => { videoDrawInCanvas( ctx, source, canvas, videoX, videoY, videoWidth, videoHeight); }, 60); } const canvasMedidaStream = Canvas.captureStream (25);Copy the code

The effect is as follows:

​

The ZEGO WebRTC team also provides the corresponding demo and source code for free reference and experience, and encapsulates the corresponding library rangeShare. Js to help you quickly get started and use directly, you can click the link to see: https://zegodev.github.io/zego-express-webrtc-sample/screen/index.html.

In the course of practice, we have found that there are still some problems that deserve your attention:

  • Problem 1: Canvas rendering consumes a high amount of CPU in the browser. Poor performance devices may cause the whole page to lag.
  • Problem 2: Although the picture can be redrawn, the redrawn stream is pure video. If the captured stream contains audio, the audio will be lost.

For problem 1: The ZEGO WebRTC team did a lot of device testing and adjusted the parameters in the rangeshare.js library to fit as many devices as possible. In addition, the Demo provides the code for estimating the CPU usage. If the CPU usage of the device is too high, the developer can replace the device or stop other processes with high CPU usage as prompted.

For issue 2: Rangeshare.js also takes this into account for you. If the screen capture stream is detected to contain audio, it will automatically cache the audio track and mix it with the canvas stream on output.

conclusion

The above is the technical interpretation of the realization of screen sharing on the Web side. With the arrival of 5G technology and the improvement of hardware device performance, there have been mature solutions to realize low-delay transmission of streaming media and streaming media data processing on the browser, such as screen capture on the browser.

Audio and video can express a lot more information than text and pictures, so audio and video transmission is becoming more popular, and with modern browsers constantly updating their APIS for audio and video manipulation, browsers may be able to do more than we thought.

The ZEGO WebRTC team will continue to share with you interesting technical issues related to audio and video on the Web.