The road to audio and Video Learning – Understand the basics of audio and video

preface

The previous article learned some about C/C++ and NDK related knowledge points, so now to a simple introduction to some related concepts about audio and video, a tall building grows from scratch, or from the most basic start.

The body of the

View the encapsulation format of audio and video files

Here we need to clarify a concept, is the encapsulation format, such as we are familiar with MP4, AVI, etc., these are encapsulation format, rather than any audio or video encoding format, see the following figure:

Use the MediaInfo tool

Earlier we used the player to analyze the content of audio and video files by viewing their properties. Here we can use a software called MediaInfo to do the same. This makes it faster and clearer to view the file information.

You can view the audio and video information through MediaInfo as follows:

Principles of video Player

For the principle of the player, actually corresponds to the generation process of audio and video, mainly including encapsulation technology, video compression and coding technology, audio compression and coding technology, if considering network transmission, also including streaming media protocol technology.

Looking at the image above, we can see how the player works.

Here to do a summary, mainly is to understand the basic steps, as well as the common format.

Streaming media protocol

Since the above illustration illustrates some of the basic steps and protocols, let’s take a look at some of the major streaming protocols.

RTSP+RTP is often used in IPTV because it adopts UDP to transmit audio and video, supports multicast, and has high efficiency. Its disadvantage is that packets will be lost when the network is not good. Therefore, the Internet network environment is unstable, so Internet video services often use TCP as its streaming media transmission layer protocol, so protocols such as RTMP and HTTP are widely used in Internet audio and video services.

Encapsulation format

Encapsulation format is also said in the previous, is the video stream and audio stream according to a certain format stored in a file.

Here is a list of some common ones. Only AVI does not support streaming media, that is, it cannot be played at the same time. Some encapsulation formats support more audio and video, while others support less formats.

Video coding

The function of video coding is to compress pixel data (RGB, YUV, etc.) into video code stream, thus reducing the amount of video data.

If the video is not encoded by compression, the volume is very large, a movie has hundreds of GIGABytes, so video coding is very important technology.

Here are some of the most common video encodings:

Audio coding

The main function of audio coding is to compress audio sampling data, such as PCM, into audio code stream so as to reduce the amount of audio data. Audio coding is also a very important technology, but in general, audio data is much smaller than video data, so even if the use of slightly backward audio coding standards, resulting in a little increase in audio data, it will not have a big impact on the total amount of audio and video data.

Here are some common audio encodings:

Network video service platform comparison

At present, there are mainly two kinds of online audio and video services. One is live broadcasting platform, such as network TV station and various live broadcasting apps, and the other is on-demand platform, which can play corresponding video programs according to the needs of users, such as video websites.

The main architecture of the live broadcast platform is RTMP+FLV+H.264+AAC, while the vod platform is generally HTTP+FLV+H.264+AAC, which will be further understood in further study.

conclusion

This article mainly introduces some basic and common names about audio and video, just have an impression, and will be more analysis later.