Due to the epidemic, many companies choose to work online, which makes me more interested in real-time audio and video, so I decided to learn about it.

First of all, let’s draw a pie and talk about my familiar process of understanding.

  1. Learn what webRTC is and what it can do.

  2. Choose one can do real-time audio and video tripartite integration experiment, but also want to understand how they are implemented.

Several companies were investigated: Tencent, Sonnet and Rongyun. Of these three, Rongyun was finally selected. First of all, I would like to talk about why I choose Rongyun: First of all, as an individual developer, and as a user who wants to know more about this area, Rongyun gave me a quick return visit after MY registration. I explained my needs and someone introduced me in detail, and I had a good experience. That you try first, however have a little not to give force is, some of the problems are encountered in integration, but carry work order or solved. Although the process is a little bit bumpy ~~~~

First follow the steps of drawing the pie to say WebRTC

WebRTC is a real-time communication solution initiated by Google, including video and audio collection, encoding and decoding, data transmission, audio and video display and other functions, we can quickly build an audio and video communication application through technology. Although its name is WebRTC, it actually supports not only audio and video communication between the Web, but also Android and IOS. In addition, because the project is open source, we can also compile C++ code, so as to achieve the interoperability of the whole platform.

WebRTC architecture

Cough cough ~~ introduction said a little stiff, this check WebRTC related reference room to see. However, I have studied the architecture diagram. If you have gained something, you can also take a closer look. If you don’t understand, you can have a deeper understanding

Say some questions about integrating rongyun audio and video:

  1. 8335 Uncaught (in Promise) DOMException: Failed to execute ‘setLocalDescription’ on ‘RTCPeerConnection’: Failed to parse SessionDescription. m=video 9 UDP/TLS/RTP/SAVPF 98 99 96 97 100 101 127 Invalid value:

  2. Failed to obtain local resources. Procedure

Tell me how I solved it:

Let’s start with the second question. The resource acquisition failed because I have two browsers open. One browser has occupied the resource, and the resource acquisition failure was reported when the second browser accessed the resource. Well, that’s it. Don’t use two browsers

As for the first problem, I was interested in RTC peerConnection after reading the newspaper because I was holding a learning attitude to understand RTC. I can sort it out well. To understand RTCPeerConnection and SDP

RTCPeerConnection: An RTCPeerConnection object allows users to communicate directly between two browsers.

What are the attributes of the SDP and what are their meanings?

// Version v=0//<username> <sess-id> <sess-version> <nettype> <addrtype> <unicast-address>o= -3089712662142082488 2 IN IP4 127.0.0.1// Session name s=-// Session start time and end time.0 indicates that there is no limit. T =0 0// Indicates that audio transmission and data channel transmission share the same transmission channel. A =group:BUNDLE audio data//WebRTC Media Streama=msid-semantic: WMS//m=audio indicates that this session contains audio, 9 means that audio is transmitted through port 9, which is not normally used in WebrTC. If set to 0, no audio is transmitted through UDP. SAVPF represents the feedback mechanism of SRTCP to control the communication process. //111 103 104 9 0 8 106 105 13 110 112 113 126 indicates the supported encoding. M =audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 110 112 113 126// C =IN IP4 0.0.0.0// The address and port used to transmit RTCP, Webrtc don't use a = RTCP: 9 IN IP4 0.0.0.0 / / ice security authentication information IN the process of negotiation a = ice - ufrag: ubhda = ice - the PWD: l82NnsGm5i7pucQRchNdjA6B / / support trickle, That is, SDP only describes media information. Information about ICE candidates will be notified separately a= ICE-options: Trickle-Down // Authentication information required during DTLS negotiation A =fingerprint: SHA-256 CA: 83: D0:0 F: 3 b: moreover, C: 8 F: F4: DB: 34:58: AC: A6:5 d: 36:01:07:9 F: 2 b: 1 d: 95:29: AD: 0 C: F8:08:6 8:34 blessed: D8:62: A7a = setup: active / / BUNDLE in front of the line A =extmap:1 URN: IEtf :params:rtp-hdrext: SSRC -audio-level// The current client only accept data, do not send data, Recvonly sendonly, inactive, sendrecva = recvonly / / RTP and RTCP packets using the same port to transmit a = RTCP - mux / / the following are the m = audio media encoding added this line, points out the code number, sampling rate, A = rTPmap :111 OPus /48000/2a=rtcp-fb:111 transport-cc Useinbandfec =1: OpUS encodings built-in FEC feature a= FMTP :111 minptime=10; Useinbandfec =1 useInbandfec =1 M = Application 9 DTLS/SCTP 5000C =IN IP4 0.0.0.0// It can be CT or AS. The CT mode is to set the bandwidth for the entire conference, and the AS mode is to set the bandwidth for a single session. The default bandwidth is kilobits per second b = AS: 30 a = ice - ufrag: ubhda = ice - the PWD: l82NnsGm5i7pucQRchNdjA6Ba = ice - options: tricklea = fingerprint: sha - 256 CA: 83: D0:0 F: 3 b: moreover, C: 8 F: F4: DB: 34:58: AC: A6:5 d: 36:01:07:9 F: 2 b: 1 d: 95:29: AD: 0 C: F8:08:6 8:34 blessed: D8:62: A7a = setup: active / / BUNDLE in front of the line A =mid:data// Port 5000 is used. The size of a message is 1024 bits. A = SCTPMap :5000 webrTC-datachannel 1024Copy the code

Okay, well, that didn’t work out, so I put in a work order, uh

~ Let me upgrade SDK said chrome upgrade optimized protocol, SDK has a companion optimization update. fine

~

Problem solved. I am very happy about this experience. Although the Demo is very simple and borrowed from Rongyun’s Demo, it is just for learning WebRTC. The whole process is still fruitful.

The overall SDK structure of Xiarongyun is roughly divided into two concepts.

  1. Concept of room

  2. The concept of resource flow

A room is an abstract concept that aims to join a room and then make a call using PeerConnection. Therefore, the room model provides interfaces such as joining and leaving to obtain information in the room

Resources is mainly refers to the media flows, use is WebRTC navigator. MediaDevices. GetUserMedia to obtain video streams, in through the establishment of PeerConnection connection, for themselves and others to publish and subscribe flow operation, etc. Specifically, you can see the connection introduced by PeerConnection in the reference article. I also understand it here, but it is not very thorough and only a superficial understanding.

Rongyun provides interface or quite simple, very convenient to use. I want you to take a look at the last couple of pieces of code and read more about them if you’re interested.

Screenshot reference rongyun official documents, not posted their own code, because my code will have some of their own business logic.

This article is a summary of personal learning. In order to understand WebRTC, we also integrate the SDK of Xiarongyun! Don’t spray if you don’t like it ~~~ ❤

Reference copy:

Developer.mozilla.org/en-US/docs/…

Baike.baidu.com/item/WebRTC…

Integrating cloud’s official website: https://www.rongcloud.cn/

Integrated document: https://docs.rongcloud.cn/v2/views/rtc/meeting/intro/ability.html