Before the words

Hello everyone, from today to the end of December, we will start a new series of articles “leisurely court walk talk about the front end”, hope that in this series, you will have more goods in the front end field, but also to draw a successful end to the end of their own. Interested partners can follow the public account, continue to pay attention to ~~~ today to introduce some basic knowledge related to live broadcast and video. Video playback for front-end personnel is actually very simple just need to get the video broadcast address and then put in the video label, in fact, we do not need to pay attention to the video encoding, decoding and other processes, but some of the terms and small knowledge still need to understand. (Some things may have problems please point out and understanding, thank you guys)

Video prototype

  • Page turning animation: it is a series of static graphics, and then pin together after manual fast page turning to form a similar video effect
  • Film animation: The old film, a big piece of film, was quite big

Basic concept

  • Video encoding formats: H.261, H.263, H.264, etc. (Convert the original video format to another video format by some compression technology)
  • Package format:.flV,.MP4,.avi, etc. (video track and audio track combined file)
  • Frame rate: FPS (number of frames transmitted per second)
  • 1. Byte (a unit of computer storage)
  • Bit: bit (the transmission unit of a computer)
  • Bit rate: Kbps (ps is /s)

B: bit, B: byte, 1B= 8B

Bit rate

Bit rate is the rate at which a digital signal can transmit information per second, usually measured in Kbps, or thousands of bits per second. The higher the bit rate per unit, the higher the accuracy, the higher the clarity, but also the disadvantage is that the higher the bit rate, the higher the speed of the network

The clarity of different bit rates is completely different:

  • 4000 yards rate:
  • 10000 yards rate:

Frame rate

The higher the frame rate is, the smoother the picture will be. If the FPS drops below 30, you will think it is PPT moving slowly. If the FPS drops above 60 frames, it will obviously improve the sense of interaction and realism.

Speaking of 60 frames brings to mind a method called requestAnimationFrame(callback), which is used to implement animation effects. Callbacks are typically executed 60 times per second, but in most browsers that follow W3C recommendations, the number of callbacks usually matches the number of browser screen refreshes.

So how to achieve smooth animation effect is to reach 60 frames, the animation is refreshed every 16.67 milliseconds, and then the human eye will feel things in the smooth movement after capturing the animation, this time is actually 1000/60 = 16.66666666666… Material of 16.67.

Here is a simple demo effect and code:

const element = document.getElementById('test'); let start; function step(timestamp) { if (start === undefined) { start = timestamp; } const elapsed = timestamp - start; // Use 'math.min ()' here to make sure the element stops at 400px. Element.style. transform = 'translateX(' + Elapsed, 400) + 'px '; If (elapsed < 4000) {/ / stop the animation window in four seconds. RequestAnimationFrame (step); } } window.requestAnimationFrame(step);Copy the code

PS: through a software screen recording may be the refresh rate and recording reasons lead to the feeling of a bit kaka, the actual effect of animation will be smoother than the GIF effect natural, you can test yourself. The DOMHighResTimeStamp parameter specifies the time at which the callback function currently sorted by requestAnimationFrame() is triggered. There is one caveat to the official prompt, make sure to always calculate the time interval between each call using the first argument (or some other method to get the current time), otherwise animations will run faster on screens with high refresh rates.

RequestAnimationFrame’s execution mechanism is determined by the system’s callback function and is calculated automatically without causing frame loss or lag, which is a significant advantage over setInterval. The setInterval task is put into an asynchronous queue. The setInterval task is executed only after the main thread task is finished. Therefore, the actual execution time is always later than the set time. The fixed interval set by setInterval is not necessarily the same as the screen refresh rate, causing frame loss. Another point is that in most browsers, requestAnimationFrame is suspended while running in a background TAB or hidden iframe to improve performance and battery life.

Back to the subject, how do you stitch together images into a video? This requires a thing encoder, which can encode multiple images and generate a lot of GOP (Group of Pictures). When playing, the decoder reads a segment of GOP for decoding and then renders. This is why many players download decoders when playing local video files (such as quickcast, Storm, Thunderbolt etc.)

GOP: The interval between two I frames

I frame, P frame, B frame

In video compression, each frame represents a still image. In actual compression, various algorithms are adopted to reduce the volume of data, among which IPB is the most common

  • I frame: also known as key frame, independent frame with all information can be independently decoded without reference to other images, which can be understood as a static picture
  • P frame: also known as differential frame, it refers to the previous I frame or P frame and then superimposes the difference of the buffer frame to generate the final picture
  • B frame: also called two-way difference between frames, book frame and frame difference before and after, through the front and rear frame with the frame to the final picture, can have higher compression ratio and the decoding speed increase, can multithreading decoding, the decoding faster (compression ratio check some information all say not clear just need to know, want to probe to the research, the ps: Please inform me when the study is clear.)

Therefore, a GOP will render a video only after loading I frame, which leads to a lot of problems such as the video took a short time, the delay of just opening etc…

  • Flower screen: We assume that this GOP only has 3 frames, IPP, if the first P frame is lost, then the reference frame of the second P frame will become I frame, so there will be a problem of not being able to hard. Then the GOP will be full of splintered screen. Until the next GOP is loaded (P frame only refers to P frame or I frame, as shown in the picture below)
  • Delay: Sometimes when you open a video or live broadcast, you will find that it does not play immediately, because a GOP starts from frame I, so when you open it, it may be just a few frames after frame I, so you have to wait until it is loaded to the next frame I before it can play

live

I didn’t have a specific investigation, but in my impression, the earliest was YY live broadcast in 2012. In my opinion, the rise of the live broadcast industry is about 14 ~15 years. Douyu is the most popular, and then came Huya, Zhanqi, Dragon Ball, Panda, e-Commerce CC, Penguin e-sports and so on

Then is the recent big event douyu and Huya merger, douyu will be out of the market after the merger, and penguin e-sports users are unified management, just roughly look at the next did not have a deep understanding of the interested students can learn more about…

Light moment

In fact, there is an interesting story in the middle, there is A Japanese bullet screen website called Niconico, and then China imitated A similar website is the original Acfun.

In 2013, station A launched A segment called “Live Broadcasting”, and then it went independent in 2014 and became Douyu TV, which surpassed station A in scale, number of users and profits.

Again later! In the 2000s, there was A problem inside Station A, and then A veteran user of Station A set up A website similar to that of Station A, and then completely surpassed that of Station A. This is station B

Ps: Stop A is too bad

Live broadcast process and common software

  • Anchor: Get stream address -> push stream
  • Server: Stream -> Stream processing -> Distribution
  • User: Pull stream -> decode -> Watch
  • Commonly used software: OBS and self-encapsulated live streaming software pushed by various live streaming platforms (students who want to be anchors can study OBS, which has many functions)

Live streaming media protocol

  • HTTP: The basic use of VOd is HTTP protocol, is used by various television stations
  • RTMP: An open protocol for transferring audio, video, and data between the Flash player and the server based on TCP (mainstream). Using Flash for playback can reduce latency and sound issues. However, Chrome does not support Flash immediately.
  • Hls: Real-time streaming media transmission protocol based on HTTP. The server generates new small files of live broadcast data. It downloads several small files and plays them continuously in sequence.
  • Http-flv: Encapsulates audio and video data into HLV and transmits the data to clients through HTTP
  • WebRTC: Open source protocol that allows browsers to conduct real-time voice and video conversations, now being pushed by companies like Google to make it a W3C standard (not really a protocol)
  • UDP: no connection is established can send IP packets encapsulation protocol, and some live platform using UDP protocol as the underlying developing their own proprietary protocol, because in weak network environment has very good effect, disadvantage is the high requirement of CDN (know how but I didn’t understand, interested students can go to further study the)

General rules for streaming addresses

  • RTMP: rtmp://x/appName/StreamName? Authentication string
  • FLV: http://x/appName/StreamName.flv? authentication string
  • HLS: http://x/AppName/StreamName.m3u8? authentication string
  • UDP: artc://x/AppName/StreamName? Authentication string

The content of the relevant live broadcast agreement is no longer described here. We have detailed introduction of the previous article of the Public account of Dazhuan. If you are interested, please click on it

At the end

Roughly introduced here, this article just introduces the basic knowledge and some vocabulary, may not be accurate please forgive! There are other articles in the back of the in-depth introduction of relevant technology, welcome readers to pay attention to the public number, convenient for you to browse other technical articles later (xiaobian warm tips: part of the article at the end of the gift oh ~~~).