Livestreaming is so popular recently that many large companies and small and medium-sized entrepreneurs want to seize the opportunity to do something. “How do you build a complete live video system?” This is a big question, and not one or two answers can explain it, but I will try to give the subject as much information as possible from a technical and entrepreneurial perspective.

As Yao Dong said, a complete live broadcast system roughly includes the following steps: acquisition, pre-processing, coding, transmission, decoding and rendering. A server processing is added to the transfer between the two ends. The general model is as follows:



The links involved in the host push stream include collection, pre-processing and coding, and the links involved in the audience are decoding and rendering. The transmission channel established between the two ends is the server, which is responsible for receiving the host push stream and distributing it to the audience after processing:

1. The collection

Acquisition is the first link in the playback process. The iOS system is relatively simple because there are few kinds of software and hardware and the hardware adaptation is good. Android is different. There are so many hardware models out there that it’s hard to fit all of them in one library. Collection on PC is also related to various camera drivers. OBS: Open Broadcaster Software is recommended to be used on PC

Tutorial: Douyu game live tutorial -OBS live software [recommended]

2. Before processing

As @Yao Dong said, “80 percent of anchors are unwatchable without a beautiful face.” Not only beauty, but also many other video processes such as blur effects and watermarks. At present, the well-known library on iOS is GPUImage, which provides rich end preprocessing effects. You can also write your own algorithm based on this library to achieve richer end effects: github-Bradlarson /GPUImage: An open source iOS framework for GPU-based image and video processing

Github-cyberagent/android-gpuImage: Android filters based on OpenGL (idea from GPUImage for iOS)

At the same time, the official Google to open a great library, covering the Android many multimedia and graphic images above related processing: https://github.com/google/grafika

3. The coding

There are two main difficulties in coding: 1. Dealing with hardware compatibility. 2. Strike a balance between high FPS, low bitrate and sound quality.

IOS hardware compatibility is good, you can directly use hard programming. The support of Android hardware is much more difficult, need to support a variety of hardware models, recommend the use of software.

4. The transmission

Transmission involves many ends:

  • From the anchor side to the server side
  • From the receiving server to the edge node
  • From the edge node to the spectator end


Push flow end and distribute end theory needs to support concurrent users should be level, but generated content push flow end in only a few, and consumption content end play end is not an order of magnitude, but their demands to push flow stability and speed is much higher than play the end, it involves all can see live broadcast end, and how the live end quality.

Many people ridicule the current CDN as unreliable, and I also admit that the traditional CDN seems to have a shortage of power in the new era. You can quickly implement large scale stream distribution with CDN, but steady high speed stream uploads may require a lot of work on your own. This is one of the reasons why we at Seven Cows do so much work in this area.

If you’re doing it yourself, the best reference on the server side is probably this one: v3_CN_Home · ossrs/ SRS Wiki · GitHub

Yang Wuming, national Chief and Panda TV chief architect, shared a “Golang’s High performance practice in live video platform” at the Gopher Beijing gathering, which is worth referring to: https://mp.weixin.qq.com/s?__biz=MzAwMDU1MTE1OQ==&mid=404230356&idx=1&sn=6b73f971c4cf1170adaf4d249480ed9a&scene=1&srcid= 0301qrbBZPrLxORTiD5D5IO1&key=b28b03434249256be5dfe068862f4c51d10c5e16b3659de35ce26bd4868f3a186366844bc7a8d3eaf48192b7553 6009 b & ascene = 0 & uin = MTUwMTI2NjQ2MA % % 3 d & 3 d devicetype = iMac + MacBookPro11%2 c2 + OS + OS + 10.11.4 + build (15 e65) & version = 11020201 & p ass_ticket=TplbLKZJE2ZwCTACdAM6vZw%2FaJtVhaRrbn0An3uI6OAbctOQaAXHM9DZBdalM5Fi

PPT address: PPT/GolangPerformancePractice. PDF at master, yangwm/PPT, dead simple

5. Server processing

In order to adapt the streams uploaded by anchors to different protocols on different platforms, some stream processing work needs to be done on the server side, such as transcoding into different formats to support different protocols, such as RTMP, HLS and FLV, and converting to multi-channel streams to adapt to different network conditions and terminal devices with different resolutions.

6. Decode and render

Decoding and rendering, that is, audio and video playback, currently have good playback compatibility on iOS, and HLS protocol is the best choice when the delay is acceptable. Android hardware decoding and encoding also have compatibility issues, currently the better open source player is based on ffPlay ijkPlayer: github-bilibili/ijkPlayer: Android/iOS Video Player based on FFmpeg N3.0, with MediaCodec, VideoToolbox Support.

At present, we have done a lot of work in client acquisition, encoding and decoding, and pushing and pulling stream acceleration. The above works are also compiled based on the holes we step on in this process: Pili Streaming Cloud · GitHub

Now is working, must take into account the early investment and the commercialization of the future, this aspect I suggest first take a look at the panda TV ZhuangMingHao long analysis: http://zhuanlan.zhihu.com/p/20717041

Before he invested in Panda TV, he deeply observed and analyzed the video and live broadcast industry from the perspective of investment for two years.