Varun Singh is CEO of CallStats. IO. He is a member of IETF and IEEE and holds many invention patents. He was a development engineer at Stmicroelectronics, a technical consultant at Nokia, and a researcher at Helsinki Technical University. In 2013, Varun Singh founded CALLSTATS I/O, which focuses on quality monitoring of Internet audio and video communications QoE.

Welcome to the RTC developer community to share your experience with more WebRTC developers.

On June 19 and 20, the WebRTC Working Group held an impromptu meeting to discuss WebRTC’s future. All browser vendors have very positive reviews of WebRTC V1.0. WebRTC V1.0 was updated in June 2018 to fix several bugs. The new WebRTC V1.0 apis include:

  • RTCRtpSender.setStreams()
  • RTCRtpTransceiver.currentDirection
  • RTCSctpTransport.maxChannels
  • RTCPeerConnection.onstatsended
  • RTCStatsEvent interface

In this article, let’s take a look at how WebRTC might change.

Application scenarios of WebRTC

Before we discuss future changes to the WebRTC API, we should consider its practical application. When we built WebRTC V1.0 in 2011, we only discussed a few application scenarios. Many changes have taken place in the industry since 2011, most notably the mobile Internet. We can provide the end user with a fully immersive experience through mobile apps, virtual reality, augmented reality and other means. We’ve also seen images become more important and interactive websites become the new normal on the Internet. Therefore, the WebRTC API today, and any changes it may make in the future, should be considered with these new application trends in mind.

Unfortunately, some of these scenarios are not well implemented or adapted to the current WebRTC API. Therefore, we need to enhance the capabilities of the API. This reinforcement mainly involves two aspects: application scenarios and ease of development.

Unification of media and data

The conference also extensively discussed the unification of media and data, including several aspects:

  • Synchronization of multiple media streams and data streams;

  • IoT device communication;

  • Live;

  • Games, including VR/AR;

  • Media Pipline control

In order to better control media Pipline, several strategies were discussed, including:

Pluggable Congestion Control: There are several proponents of Pluggable Congestion Control, including CallStats.io. One of the main reasons to support it is the use of multipath protocol for multimedia real-time communication. We have long-standing investments in this area, including work on multimedia congestion control and related optimizations.

Alternatives to browser implementations: Alternatives to browser implementations mean developers will be able to use their own Jitter buffers, FEC algorithms (e.g. LDPC, Raptor, etc.), encoders and decoders (codecs), etc.

The next evolution of WebRTC

At the conference, Google’s Peter Thatcher raised a number of possibilities for WebRTC’s next evolution. Let’s take a look at each of these proposals.

Keep in mind that with each update to the API, application developers gain more control over the channel. It also means that apis will become more complex, but signaling will be more reliable and flexible to handle.

In general, we believe that the more control an app developer has, the better the product will be. First, reduce the complexity that some protocols and algorithms bring to browser development.

Second, Web developers already know how to develop better with SHIM and make it reusable by other developers.

ORTC

Several objects first proposed in ORTC were added to WebRTC V1.0. Unlike WebRTC V1.0, ORTC does not use SDP as a control interface, allowing developers to directly control media and data transfer channels. More objects can be controlled directly. For example, with ORTC, you can use and control extensible video codecs.

Pluggable transmission

Considering further splitting media pipeline objects, pluggable transport can achieve more control over media pipeline functions. For example, adding or removing metadata to an encoded/decoded frame, or controlling media quality.

To achieve this, and to make the media transfer more controllable. We need to separate the encoder and decoder from RTCRtpSender and RTCRtpReceiver, respectively. Further, we can transfer media and data separately, such as RTP over UDP or QUIC or SCTP. In addition to pluggable transport, this will enable large-scale conference services to use different encryption keys for hop-by-hop encryption (via DTLS/SRTP) and end-to-end encryption.

Media raw data and full control

Providing full control over pipeline will give the App full control over encoding and decoding, media congestion control, security (any form of encryption), processing of media frames (such as FEC, RTX), media synchronization on the decoding side, etc. This increase in flexibility also requires the App to support more functions and more effort in development. Of course, it is up to the developer to decide whether to do or not.

summary

There will be two changes:

  • Create more objects for components in the audio, video, and data channels;

  • Provides the permission to access media raw data.

These changes will also raise some questions:

  • Whether to encrypt raw data;

  • JavaScript is not real-time.

Regarding the security discussion, we believe that media raw data should be encrypted, and applications will not receive unencrypted data.

Questions about JavaScript. If you manage a full pipeline in the main thread, you can only process 1 frame per second or less. Therefore, we need a new set of JavaScript and browser features, such as WebWorkers, WebAssembly(WASM). There are other issues with JavaScript, in which case the Web side also needs to be able to access the media stream and the App side can track the expected task status.

For more on what’s likely to happen to WebRTC, and our thoughts on the future evolution of the real-time Internet. We will share and discuss with you in depth at RTC 2018.

Tips

WebRTC is still very new in the near future. For developers facing a lot of problems with WebRTC development, Varun Singh will be sharing more on the web and Quality in Real Time at RTC 2018. Meanwhile, Huib Kleinhout, Google’s WebRTC product manager, will share more about WebRTC’s future plans at the main conference the next day. The tickets are free for a limited time, seats are limited. If you want to know more about it, please click to register.