Have a "sound" gather together | RTE 2021 programming challenge successfully packaged

On June 12, the “RTE 2021 Programming Challenge”, co-sponsored by Agora and Huexin, came to a successful conclusion. From more than 200 teams out of the 46 final teams with excellent defense for more than two months of the competition drew a successful full stop.

This year’s “RTE 2021 Innovative Programming Challenge” was divided into two tracks. The application innovation track continued the challenge of “Developing Apps with Agora SDK”. The developers of the technology innovation track can “develop self-developed plug-ins and function Demo by using the plug-in interface of the acoustic network cloud market”.

Although the competition system and questions put forward higher requirements for the participating teams, it also provides unique technical innovation space for everyone. Both tracks had nearly twice as many teams and submissions as last year.

The final of this competition and the awarding of the prize were both held online via the Agora Video Call App, and the whole event was streamed live via Station B. In the final, there were 1, 2 and 3 winners of the application innovation track, 1 winner of the special award of “environmental information” and 6 winners of the “excellence award”. One “Technical Innovation Special Award” and one “Excellence Award” of the Technical Innovation Track.

Application Innovation Track

First Prize: Agora Home AI

With the performance improvement of intelligent devices and the rapid development of network, intelligent hardware based on audio and video is also booming. Cross-brand and cross-product device management has also become a topic haunting users in their daily use.

“Agora Home AI” is a smart Home themed system that uses a cloud signaling SDK to achieve remote control of IoT devices. At the same time, the 1V1 video between man and machine is realized through the Agora RTC SDK of the audio network, and the video collected by the robot is sent to the PC control center for AI intelligent detection and event response.

The system uses open source YOLO V3 algorithm for various video data processing, support C#, C++ calls; Unity 3D, VS series development. At present, it supports YOLO basic 80 kinds of object recognition, helmet recognition, hockey identification files and so on. The cloud signaling SDK provided by Acoustic.com is used for remote device control, and the group room is built for real-time message communication, which supports intelligent hardware control through custom protocol.

“Agora Home AI” can help users realize the access and control of wearable devices, smart furniture devices and video monitoring devices. Including intelligent lighting, intelligent doors and Windows, intelligent door locks, intelligent security, intelligent bracelet monitoring, intelligent home appliances control and other supporting products, so that users can achieve a variety of brands of intelligent devices in the unified interactive platform interconnection, unified management, intelligent linkage. To create a more comfortable, safer and more energy-efficient home living environment for users.

Second Prize: Agora Fiow

The work “Agora Flow” that won the second place is a Low Code Web sharing editor for audio and video based on audio network + ring message SDK.

The inspiration of this work comes from the process of using the Agora SDK of audio network, the author has been thinking about the audio and video service in the form of SDK to provide services and empower developers, is there any other form? Low Code is one such possible solution. Modular integration of audio and video related functions to provide a graphical interface, so that developers can use the form of PPT to complete the functions they want to achieve.

The work provides a Web-based template project integrated with RTC Chat SDK through audio and video transmission and cloud signaling SDK products, and generates configuration items through CodeGen. The automatic generation of Low Code project in online flowchart editor is realized. Almost everything in the work can be done by dragging and dropping.

One developer joked that while many of the entries in the competition are about creating an App, “Agora Flow” is about creating a project to help developers develop better apps.

Third Prize: Urban Explorer

“Urban Explorer” project is a social product built using map LBS + cloud signaling SDK + real-time audio and video. This product is designed for people who want to find new friends with common interests and travel and explore the city together.

The use of the product is very simple and convenient. After users register and log in, the location of the user will be automatically updated through LBS map. Just click “Initiate Task” and select the number of “Adventure”, users can have a new urban adventure with their friends.

For the RTE scene, the product has realized the scene of chatting with multiple people in the room. In addition, the use of cloud signaling SDK combined with the actual business scenarios, to expand the current language chat scene. In the future, the product will also be connected with the function of video chat, so that those who cannot travel immediately can participate in urban exploration together.

Environmental Information Award: Pavilion of Forget Worry

“The Hall of Forget Worry” is a warm work, hoping to help people in modern life to forget their worries, relieve their worries and spread positive energy by talking to each other.

It’s a stranger social App that contains a stream of information. It combines several of the most common forms of social products, including text chat, phone calls, etc. Some troubles that are inconvenient to share with relatives and friends can find resonance and comfort when communicating with strangers.

Excellence Award: Vchat

“VChat” uses facial bone recognition and cloud signaling SDK to realize real-time communication of virtual 3D characters. The WebGL engine of tensorflow.js is used as the back end, and the existing open source face recognition model is used to recognize the position of the face and the state of the facial features through the camera. Js and vrm. js to update the face data to the virtual 3D model in real time.

In the video part, the real-time message RTM SDK is used to transmit the face bone data to the channel in real time for other users to subscribe and restore the face. In the voice part, the voice is transmitted in real time through the RTC SDK and subscribed by users. Can realize synchronous face change, voice chat and other functions.

In addition to the “Vchat” mentioned above, Other works of “Smart Classroom Answering Module”, “Agora X-Runtime”, “WeLN”, “Huanxin (BLA-BLA. APP)” and “Intelligent AR brush copying teaching system /CopyTeachWorks” won the Excellence Award in the “Application Innovation Track” of the competition. Some of these excellent works may not be able to be presented here. If you are interested, you can visit our GitHub repository to check them out:

https://github.com/AgoraIO-Co…

Technical Innovation Track

Special Award for Technology Innovation: Face Recognition

“Technical Innovation Special Award” is a special award set for the “Technical Innovation Track”. The winning work is a “face recognition” plug-in based on C++ language encapsulated on the iOS platform using AgoraEngineKit2 development.

The work interacts with the external through the TysmExtensionManger class, and handles the related logic implemented by the plug-in internally. Keep the IExtensionProvider, IVideofilter, and your development Processer in the same place. Publicize the two files in framework mode, which is easy for developers to consult, and can also be passed as parameters to enhance the code’s readability.

The plug-in can support face detection, tracking, and multi-face tracking recognition, and provide 3D dot feedback to the identified areas of the face contour, eyes, eyebrows, nose, mouth and so on.

Excellence Award: Water Mask

“Water Mask” project is the competition work of “Technical Track”. Through the video collection or play link of the audio network SDK, the hidden watermark of image or text type is added on the YUV domain or after coding.

After the addition of invisible watermark (blind watermark), users cannot directly see the watermark information in the video. While protecting the copyright of video publishers, it also guarantees the video viewing experience of users. In the future, “Water Mask” also hopes to expand the voice print watermarking in the audio processing, and bring more and better experience to the industry in the scene of video copyright traceability, authentication and anti-counterfeiting.

These are some of the winning entries and teams from this year’s RTE 2021 Coding Challenge. More works of this challenge will be open source on GitHub, if you are interested, you can go to check them out: