Wechat opened its mini program live interface at the end of last year. Small program from only suitable for reading, life services, tools and other applications of traffic entry, has become a new platform for many audio and video applications. The opening of the new function allows more apps to use wechat’s social network of acquaintances to quickly attract new apps, provide convenient value-added services, or accelerate app monetization. Our client, Litchi FM, realizes voice social live broadcast on the mini program, and Huajiao Live broadcast also realizes the scene of sending thousands of dollars of questions through the mini program.

At present, small program live has opened interfaces to online education, finance, medical care, government affairs and people’s livelihood, and social networking industries. With the addition of interactive limai, it will realize richer business functions in these scenarios and expand more application scenarios, such as:

  • Online classes: one-to-one and one-to-many live online classes, suitable for vocational education, small-class education, preschool education and other scenes, realizing real-time interaction between teachers and students;

  • Video conference: quickly establish multi-person video conference in one second to achieve efficient remote collaboration;

  • Online medical treatment: break through the geographical limitations of medical resources and system platform to achieve remote multi-party video consultation and reduce diagnostic costs;

  • Online shopping: live show products, while the audience interact with the microphone, further promote the transaction;

  • VIP customer service: exclusive video customer service, 1-to-1 real-time communication;

  • Bank opening: dedicated network, no need to install App, quickly realize information authentication and video opening through small programs;

  • Remote alarm: one-button alarm can be realized from wechat mini program to quickly connect relevant departments, and through real-time video communication, the police can control the scene situation in real time.

By adding low-latency, high-quality audio and video interaction, we can extend a broader application scenario for wechat mini programs. However, there are three difficulties to solve in the development process.

The realization and difficulty of small program interactive link mai live broadcast

The iOS wechat kernel browser does not support WebRTC. If the developer through WebRTC to achieve small program live, live can only be achieved on the Android platform. Therefore, in order to support iOS and Android platforms, it is necessary to use wechat and < live-Player > to achieve the two live broadcast interfaces.

However, both interfaces are based on the RTMP protocol for pulling and pushing streams. This TCP-based protocol naturally has a high latency and is not applicable to live broadcast scenarios with mics.

We have analyzed in detail the generation and response of delay in different stages in the article “Detailed Explanation of Delay in Audio and Video Live Broadcast”. The generation process of delay is simplified as shown in the figure above, and the delay consists of two parts:

  • The delay of serving. There will be different degrees of delay (T1) between the acquisition terminal and the playback terminal. The delay of this section is affected by device performance, codec algorithm, resolution and frame rate selected during live broadcast/call, etc. The delay may be tens of milliseconds.

  • Transfer delay (T2) between end and server. This delay is affected by network quality, transmission line selection, cross-operator, cross-country and other factors. As mentioned above, the API limit can only push/pull RTMP streams. If directly transmitted through CDN, the delay can reach 2-5s. The tens of milliseconds of latency on the device is trivial compared to this.

Therefore, the essence of solving the delay problem of small program is to shorten the delay of T2.

The method of acoustic network to reduce the delay is that after the protocol conversion, the audio and video stream is not transmitted through CDN, but through Agora’s private UDP protocol, low delay transmission. Ordinary UDP builds unreliable transmissions, so deep development is needed to prevent packet loss and make the transmissions reliable. Moreover, in order to ensure the transmission quality, it is necessary to build a whole set of low-delay UDP transmission network, which has a high technical and financial threshold. If it is directly transmitted on the public network, it is like running naked.

How to realize the small program interactive link mai live?

As shown in the figure above, there are three key points in the implementation logic:

  • Protocol conversion: protocol conversion is carried out on the audio and video stream sent by the small program side through the protocol converter at the edge node.

  • Low delay transmission: The converted audio and video stream is transferred to Agora SD-RTN™ virtual communication network through the private UDP protocol.

  • Full platform compatibility: Agora SD-RTN™ can be transmitted to various platform apps integrated with the SDK of Sound network through the sound network private UDP protocol.

The audio network small program live interactive Lianyai solution based on the Audio network Agora SD-RTN™ real-time cloud to achieve low latency, high quality real-time audio and video transmission. Audio network small program live interactive link to the mic solution can help developers to solve many difficulties in development:

1. Perfect solution for iOS small programs with live mic

With the help of the push-pull flow interface of wechat applet itself and the cooperation of Agora SD-RTN™ global virtual communication network, sonnet’s private UDP protocol enables users to communicate with Android applet through iOS applet with low delay. Sound net can help developers to solve the protocol conversion and low delay transmission in small program Lianmai live.

2. Seamless communication between applets and Native App

Agora Native SDK supports iOS, Android, Windows, macOS and Web platforms. The small program developed based on the small program live interactive Lianyama solution can achieve seamless communication with Native apps of other platforms under the low-delay real-time network environment of Agora SD-RTN™.

Agora SD-RTN™, with nearly 200 nodes deployed worldwide, uses a distributed architecture and dynamic routing algorithm to avoid network congestion and backbone network failures. Due to the global deployment, transnational and trans-continental transmission can also ensure high availability and low latency, which can provide strong support for developers with a global perspective.

3. Support simultaneous video connection for 7 people

Social live broadcast and online education scenarios need to realize multi-person live broadcast with mics. At the same time, the small program of sonnet can support 7 people to interact with each other online at the same time while ensuring the picture resolution and audio quality.

4. High quality, low delay

Both applets and clients use a “near” access policy, allowing users to access the data nodes with the best quality. Sd-rtn ™ software defines optimal routes to select the optimal path, automatically avoiding network congestion, avoiding the impact of backbone network failures, and ensuring audio and video data transmission with the best quality.

If you are interested in our solutions, please visit our Agora q&A section and post to communicate with our engineers.