In recent years, related industries in the field of audio and video, such as live video, live delivery, online education and online medical care, have become very hot and become the focus of public attention.

In the near future, audio and video technology permeates all walks of life, everywhere. From the Internet of Things (IoT) network to individual users’ mobile devices, audio and video technology exists in various forms on various terminals and application platforms. Therefore, the realization of cross-platform application development is very important, a set of code for multiple platforms, cross-platform development reduces the workload of engineers, let the product rapid iteration.

This article will introduce how to build a desktop cross-platform audio and video application using Electron.

First, desktop audio and video applications are difficult to develop, and cross-platform development is becoming popular

In the past, the preferred language for developing a desktop video application was C++. Desktop UI frameworks are usually QT, MFC, Duilib, SoUI, or paid UI libraries, but no matter which framework, there are significant limitations. Either it’s not cross-platform, it’s a high barrier to use, it doesn’t perform well, or it’s not stable enough to have a lot of holes in the development process. It can be said that when C++ does desktop application development, the choice of UI framework is relatively difficult.

With the continuous innovation of the development of front-end technology and the continuous emergence of various front-end frameworks, more and more desktop applications are approaching the Web. It is becoming more and more popular to use front-end technology to realize the UI cross-platform of desktop applications.

With the advent of Electron technology, developers can build cross-platform desktop applications using JavaScript, HTML, and CSS. Applications built based on Electron are compatible with multiple systems such as Mac, Windows, and Linux, eliminating differences and version restrictions between browsers. VS Code is a great desktop application to show the world what Electron technology can do.

Second, the limitations of building desktop applications through WebRTC

Electron uses the Chrome kernel to do front-end rendering. If you want to talk about audio and video based on Chrome, the most popular audio and video solution is WebRTC.

Based on the WebRTC technology, a simple one-to-one video call scenario can be realized. Google’s recommended solution supports video P2P calls and server forwarding. Developers basically only need to research and develop signaling content, and almost do not do any development on the server side, which can be said to be a relatively complete and mature solution.

However, current real-time audio and video applications are far from limited to simple one-to-one video calls. Seemingly common and reasonable demands, such as video preprocessing and adding a beautiful face, voice changing or background music during live broadcast, are almost impossible to achieve if WebRTC scheme is directly used.

There is a popular saying in the audio and video industry that there are 10,000 WebrTCs between demo and product. Although this is a slight exaggeration, it reflects that there are many audio and video professional problems to be solved in the process of using WebRTC.

It is too simple to just use WebRTC to achieve a good real-time audio and video product. The audio and video field involves a lot of professional content, even a simple demo can not be completed in a few lines of code, in the actual product development, through signaling exchange SDP, candidate and processing timing and other links, developers need to be careful to complete the video connection.

Third, the Electron platform SDK was launched to achieve high-performance optimization

The importance of cross-platform development of audio and video products has been gradually highlighted. As a professional real-time voice and video cloud service provider, Jiqi Technology has launched the SDK based on Electron platform, enabling developers to quickly build cross-platform applications by invoking the instant SDK. ZEGO Electron SDK extended plug-in call based on Electron C++ is the implementation of the native SDK, and the interaction between js and the native SDK is optimized to achieve high performance.

The main optimization contents of ZEGO Electron SDK include:

**1, reduce the chances of JS code processing video data. ** that is to use js and c++ shared memory for processing, processing video frame data is all in the c++ layer code, js layer is only responsible for conveying this memory data to the graphics card memory, through webgl rendering.

**2. Optimize video rendering performance. ** Adopts V-Sync technology for rendering, which effectively solves the problem of rendering performance in the case of multi-channel video.

Through Zego Electron SDK, we can not only obtain powerful audio and video capabilities of native platform, but also greatly reduce the development threshold and greatly improve the development efficiency.

Step 4: Build desktop cross-platform audio and video applications

Based on Zego Electron SDK, video can be sent and received through four simple functions.

First, initialize the SDK. Initialize the SDK by applying AppId and AppSign on the console ** of ZEGO official website.

** Second, log in to your room. ** After the SDK is successfully initialized, the user can specify a room ID to log in to the room.

** Third, audio and video streaming. ** After a successful login, you can push the local real-time camera data to ZEGO’s server with only a push stream function and a custom push stream ID.

** Fourth, audio and video pull flow. ** Video streams from ZEGO server can be pulled to local display by using a pull function specifying the id of the pull stream.

In four simple steps, developers can easily build a desktop cross-platform application to send and receive real-time audio and video data. The audio and video application realized by Zego Electron SDK can also communicate with other Zego platform SDKS, such as Android, iOS, Web js (webrtc version), wechat small program, window(c++, c#), Mac and so on, further reducing the workload of cross-platform development.

—————————————— Manual partition ——————————————————

Zego Electron SDK Integration document please click:

Doc – useful. Zego. Im/useful / 2193 HTM…

For example source code, please click:

Github.com/zegoim/zego…

About cross-platform development technology in the audio and video industry, please leave a comment and communicate with us.