With the rapid development of the Internet today, people’s social needs are constantly upgrading. All the time, social platforms are exploring new ways to interact to meet people’s ever-changing social needs. From the hot live broadcast in 2018 to the real-time audio and video increasingly relied on by the public, the rapid development of communication technology provides more possibilities for the exploration of social platforms.

As we’ve seen in previous outbreaks, telecommuting has become a standard requirement for many businesses. Telecommuting will become a new normal, and it will bring the prosperity of audio and video sectors, and the boom of audio and video conferencing will erupt. Especially, 5G projects will be gradually implemented, bringing new impetus to the development of video conferencing.

In the future, there will be fewer ways to communicate with text and text, while audio and video will become the mainstream, and the salary will be quite good. As shown below:

This creates a huge demand for audio and video. But at present, developers with audio and video development technology are few and far between.

However, nowadays, the knowledge of audio and video is so complicated that it is very difficult to learn by yourself. It requires both solid basic knowledge and a lot of engineering experience.

There are basically two levels of audio and video practitioners: the first level is engaged in audio and video algorithm. There are very few such people, who need to have deep mathematical ability and algorithm background. Such people generally choose to work in large companies, and the salary of millions is the lowest.

On the other hand, there are more people engaged in engineering. They have solid theoretical foundation, strong technical foundation and are very familiar with audio and video. These people earn about 20% more than the average developer; With the arrival of THE 5G era, audio and video have gradually become a necessity in People’s Daily life. So now there are a lot of companies looking for audiovisual talent, a little bit better audiovisual talent now maybe 3 or 4 companies are competing for it. Therefore, the demand for audio and video talents has changed from the minority to the public, which is more the result of people’s expectation of the future market.

At present, there are no systematic tutorials or books, and the blog articles on the Internet are scattered. In this small series, I spend a lot of time collecting and sorting out the knowledge points of audio and video, and finally organize them into a special topic. Today, I take this article to share with friends who are interested in audio and video.

Let’s start with a directional study guide:

1. Draw an image on Android platform, using at least 3 different APIS: ImageView, SurfaceView, custom View 2. Using AudioRecord and AudioTrack API on Android platform to complete audio PCM data collection and playback, and realize reading and writing audio WAV files 3. Camera API was used for video collection on Android platform, SurfaceView and TextureView were respectively used to preview Camera data, and NV21 data callback 4. Learn Android platform MediaExtractor and MediaMuxer API, know how to parse and package MP4 files 5. Learn Android platform OpenGL ES API, understand the basic process of OpenGL development, draw a triangle with OpenGL 6. Learn Android platform OpenGL ES API, learn texture rendering, can use OpenGL display a picture 7. Learn MediaCodec API, complete audio AAC hardcoding and harddecoding Learn MediaCodec API, complete hard coding and hard decoding of video H.264 9. Series the entire audio and video recording process, complete audio and video collection, coding, packaging into MP4 output 10. Series the whole audio and video playing process, complete the analysis of MP4, audio and video decoding, playing and rendering 11. Learn more about OpenGL, learn how to implement video clipping, rotation, watermarking, filters, and learn OpenGL advanced features such as: VBO, VAO, FBO, etc. 12. Learn Android graphics image architecture, able to draw Camera preview screen using GLSurfaceviw 13. Delve into audio and video related network protocols such as RTMP, HLS, and packet formats such as FLV, MP4 14. In-depth study of some open source projects in audio and video field, such as WebrTC, FFMPEG, ijkPlayer, LIBRtmp, etc. 15. Transplant ffMPEG library to Android platform, combined with the experience accumulated above, write a simple audio and video player 16. Porting x264 library to Android platform, combining the experience accumulated above, completing video data H264 software editing function 17. Transplant librtmp library to Android platform, combine the experience accumulated above, complete the push stream function of Android RTMP 18. With the experience accumulated above, I can make a short video APP to complete functions such as breakpoint shooting, adding watermarks, local transcoding, video clips, video splicing, MV special effects and so on

Trust me, if you do it all, you’ll be hot in the audiovisual job market. The following is to share with you a small edition spent 86 days to organize the audio and video development from the beginning to the advanced, has been organized into a PDF document: if you need the full version, you can click here for free.

First, the basic introduction:

1 drawing a Picture

1. ImageView draws pictures

2. SurfaceView draws the image

3. Customize the View to draw pictures

Second, AudioRecord API details

Three, the use of AudioRecord recording, and generate WAV

  • Create an AudioRecord object
  • Initialize a buffer
  • Start the recording
  • Create a data stream that reads the audio data from the AudioRecord into the initialized buffer and imports the buffer data into the data stream.
  • Shutting down the data stream
  • Stop the recording

4. Play PCM audio with AudioTrack

1. Basic use of AudioTrack

  • MODE_STATIC mode
  • MODE_STREAM mode

2. The AudioTrack explanation

  • Type of audio stream
  • The concept of Buffer allocation and Frame
  • AudioTrack construction process

3. Comparison between AudioTrack and MediaPlayer

  • The difference between
  • contact
  • SoundPool

V. Use Camera API to collect video data

1. Preview Camera data

2. Callback NV21 data

Use MediaExtractor and MediaMuxer API to parse and package MP4 files

1. MediaExtractor API is introduced

2. MediaMuxer API is introduced

3. Use context

MediaCodec API details

1. MediaCodec is introduced

2. MediaCodec API specification

3. MediaCodec flow control

  • Basic concepts of flow control
  • Android hardcoded flow control
  • Android flow control policy selection

Due to the limited length of the article, the remaining content is too much, and the illustrations are limited. Only screenshots and contents can be shown below:

Two, intermediate advanced chapter:

  • Android OpenGL ES development (a) : OpenGL ES introduction
  • Android OpenGL ES development (ii) : OpenGL ES environment construction
  • Android OpenGL ES development (three) : OpenGL ES defines shapes
  • Android OpenGL ES development (four) : OpenGL ES draw shapes
  • OpenGL ES uses projection and camera views
  • Android OpenGL ES development (6) : OpenGL ES add motion effects
  • OpenGL ES responds to touch events
  • Android OpenGL ES development (eight) : OpenGL ES shader language GLSL
  • Android OpenGL ES development (nine) : OpenGL ES texture mapping
  • Android OpenGL ES development (10) : Interact with shaders via GLES20
  • Use OpenGL to display an image
  • GLSurfaceviw draws Camera preview and takes photos
  • Use OpenGL ES to complete video recording and achieve video watermark effect

Advanced Inquiry:

  • In-depth study of audio and video coding, such as H.264, AAC, research using open source codec libraries, such as X. 264, JM, etc
  • In-depth study of audio and video related network protocols such as RTMP, HLS, and packet formats such as FLV, MP4
  • In-depth study of some open source projects in audio and video field, such as WebrTC, FFMPEG, ijkPlayer, Librtmp and so on
  • Ffmpeg library transplant to Android platform, combined with the experience accumulated above, write a simple audio and video player
  • Porting x264 library to Android platform, combining with the experience accumulated above, the video data H264 software editing function is completed
  • The librtmp library is transplanted to Android platform, and combined with the experience accumulated above, the push stream function of Android RTMP is completed

Audio and video coding and decoding technology

  • Audio and video codec technology (I) : MPEG-4/H.264 AVC codec standard
  • Audio and video coding technology (II) : AAC audio coding technology

Streaming media protocol

  • Streaming media protocol (1) : HLS
  • Streaming media protocol (2) : RTMP

Multimedia File Format

  • Multimedia file format (I) : MP4 format
  • Multimedia file format (II) : FLV format
  • Multimedia file format (3) : M3U8 format
  • Multimedia file format (4) : TS format
  • Multimedia file format (V) : PCM/WAV format

FFmpeg learning record

  • FFmpeg Command line tool learning (1) : view the media file header information tool ffprobe
  • FFmpeg command line tool learning (2) : play media file tool FFplay
  • FFmpeg command line tool learning (three) : media file conversion tool FFmpeg
  • FFmpeg command line tool learning (four) : FFmpeg acquisition equipment
  • FFmpeg command line tool learning (five) : FFmpeg adjust audio and video playback speed

  • FFmpeg Learning part 1: Introduction to FFmpeg
  • Install FFmpeg on Mac
  • FFmpeg learning (3) : FFmpeg porting to Android platform
  • FFmpeg learning (4) : FFmpeg API introduction and general API analysis
  • FFmpeg learning (five) : FFmpeg codec API analysis
  • FFmpeg learning (6) : FFmpeg core module libavFormat and libavCodec analysis

  • FFmpeg structure learning (A) : AVFormatContext analysis
  • FFmpeg structure learning part II: AVStream analysis
  • FFmpeg structure learning part III: AVPacket analysis
  • FFmpeg structure learning (4) : AVFrame analysis
  • FFmpeg structure learning: AVCodec analysis
  • FFmpeg structure learning (VI) : AVCodecContext analysis
  • FFmpeg structure learning (7) : AVIOContext analysis
  • FFmpeg structure learning (8) : relationships between important structures in FFmpeg

More directory screenshots:

Conclusion: * * * * the above is about the audio and video data and summarizes the strongest part (and no screenshots), believe should be the most complete cut, all materials are free to share with everyone, and save the friends interested in audio and video of time looking for information, if you just need to be available free here

The prospect of audio and video development, I believe you can see from the relevant recruitment website, whether streaming media development or audio and video processing development, are a large number of recruitment, and due to the relatively high threshold of entry, learning cycle is long, the salary is also high.

In addition: Xiaobian also collect and sort out some audio and video related supporting videos, I believe that we can make learning more quickly and effectively:

All the information in this article is free to share with you,Just click here with your finger to get free.

The road to being an Android architect is a long one. Learning is like rowing upstream. If you don’t advance, you’re going to fall back. If you slack off, you’re going to fall back. Be a diligent and upward person, study harder, seize the center, ning jing not miscellaneous, ning designed not more.

If you need the full Android audio and video development documentation, just support this article.

Quick Access: Download it here! Sincerity full!!

Sorting is not easy, feel helpful friends can help like share support small series ~

Your support, my motivation; I wish you a bright future and continuous offer!!