I don’t know if you have noticed, in recent years, the demand for audio and video talents in China is growing, and the salary on a recruitment website is as high as 60K.

In terms of the general trend of the future, with the arrival of 5G era, audio and video gradually become a necessity in People’s Daily life. In addition to online education, audio and video conferencing, and instant messaging, which must use audio and video technology, other products also need audio and video elements. For example, will the present website registration and login of the text and text all changed to voice mode is more secure? We believe that with the popularization of audio and video technology, more creative products will be born like bamboo shoots after a spring rain.

Now audio and video have been so hot, now learn to have a chance?

At the bottom of the rapid development of audio and video industry is more and more efficient information communication, chip equipment more and more powerful and intelligent. With the application of 5G and 6G, it is believed that audio and video will not be the development direction in the future, but basic functions like water and electricity. Based on this cognitive change, I firmly believe in the development path of audio and video.

Code reference [FFmpeg audio and video development 20 years] in a text

In the next 10 years, the technology will be the same technology, and the main player will be AI, edge computing, instead of mobile apps. Had it not been for the XX war, the 4T AI chip of Haisi would have dropped below $10 this year. The large-scale production of chips, the price drop, will drive a new round of changes in the industry.

Software developers, there will be corresponding new opportunities, audio and video, network communication, OpenGL/ES are a must to master, engaged in related development, plus OpenCV, Tensorflow-Lite. Fundamentals of deep learning. A personal point of view, only for reference.

So simply talk about FFmpeg has no meaning, talk about audio and video, digital image processing, network communication, with FFmpeg related, perhaps more reference value. What is the future of FFmpeg? A technology can only solve a particular problem for a suitable period of time.

Is it necessary to develop software when codec is done by hardware?

Codec no matter hard solution soft solution code amount is very small. Dozens of lines of code. It’s a small percentage of a project. A lot of code lies in network communication, image processing, software logic and so on.

How to learn systematic audio and video?

Here’s how TO learn, and trust me, if you do it, you’ll be hot in the audiovisual job market

First, the primary introduction:

First, draw pictures

  1. ImageView draws images

  2. The SurfaceView draws the image

  3. Custom View drawing pictures

Second, AudioRecord API details

Three, the use of AudioRecord recording, and generate WAV

  • Create an AudioRecord object
  • Initialize a buffer
  • Start the recording
  • Create a data stream that reads the audio data from the AudioRecord into the initialized buffer and imports the buffer data into the data stream.
  • Shutting down the data stream
  • Stop the recording

4. Play PCM audio with AudioTrack

1. Basic use of AudioTrack

  • MODE_STATIC mode
  • MODE_STREAM mode

2. The AudioTrack explanation

  • Type of audio stream
  • The concept of Buffer allocation and Frame
  • AudioTrack construction process
  1. AudioTrack vs. MediaPlayer
  • The difference between
  • contact
  • SoundPool

V. Use Camera API to collect video data

1. Preview Camera data

2. Callback NV21 data

Use MediaExtractor and MediaMuxer API to parse and package MP4 files

1. MediaExtractor API is introduced

2. MediaMuxer API is introduced

3. Use context

MediaCodec API details

1. MediaCodec is introduced

2. MediaCodec API specification

3. MediaCodec flow control

  • Basic concepts of flow control
  • Android hardcoded flow control
  • Android flow control policy selection

Due to the limited length of the article, the remaining content is too much, and the illustrations are limited. Only screenshots and contents can be shown below:

All knowledge points are detailed in the open source project”Lot”You can help yourself if you need.

Two, intermediate advanced chapter:

  • Android OpenGL ES development (a) : OpenGL ES introduction
  • Android OpenGL ES development (ii) : OpenGL ES environment construction
  • Android OpenGL ES development (three) : OpenGL ES defines shapes
  • Android OpenGL ES development (four) : OpenGL ES draw shapes
  • OpenGL ES uses projection and camera views
  • Android OpenGL ES development (6) : OpenGL ES add motion effects
  • OpenGL ES responds to touch events
  • Android OpenGL ES development (eight) : OpenGL ES shader language GLSL
  • Android OpenGL ES development (nine) : OpenGL ES texture mapping
  • Android OpenGL ES development (10) : Interact with shaders via GLES20
  • Use OpenGL to display an image
  • GLSurfaceviw draws Camera preview and takes photos
  • Use OpenGL ES to complete video recording and achieve video watermark effect

Advanced Inquiry:

  • In-depth study of audio and video coding, such as H.264, AAC, research using open source codec libraries, such as X. 264, JM, etc
  • In-depth study of audio and video related network protocols such as RTMP, HLS, and packet formats such as FLV, MP4
  • In-depth study of some open source projects in audio and video field, such as WebrTC, FFMPEG, ijkPlayer, Librtmp and so on
  • Ffmpeg library transplant to Android platform, combined with the experience accumulated above, write a simple audio and video player
  • Porting x264 library to Android platform, combining with the experience accumulated above, the video data H264 software editing function is completed
  • The librtmp library is transplanted to Android platform, and combined with the experience accumulated above, the push stream function of Android RTMP is completed

Audio and video coding and decoding technology

  • Audio and video codec technology (I) : MPEG-4/H.264 AVC codec standard
  • Audio and video coding technology (II) : AAC audio coding technology

Streaming media protocol

  • Streaming media protocol (1) : HLS
  • Streaming media protocol (2) : RTMP

Multimedia File Format

  • Multimedia file format (I) : MP4 format
  • Multimedia file format (II) : FLV format
  • Multimedia file format (3) : M3U8 format
  • Multimedia file format (4) : TS format
  • Multimedia file format (V) : PCM/WAV format

FFmpeg learning record

  • FFmpeg Command line tool learning (1) : view the media file header information tool ffprobe
  • FFmpeg command line tool learning (2) : play media file tool FFplay
  • FFmpeg command line tool learning (three) : media file conversion tool FFmpeg
  • FFmpeg command line tool learning (four) : FFmpeg acquisition equipment
  • FFmpeg command line tool learning (five) : FFmpeg adjust audio and video playback speed

  • FFmpeg Learning part 1: Introduction to FFmpeg
  • Install FFmpeg on Mac
  • FFmpeg learning (3) : FFmpeg porting to Android platform
  • FFmpeg learning (4) : FFmpeg API introduction and general API analysis
  • FFmpeg learning (five) : FFmpeg codec API analysis
  • FFmpeg learning (6) : FFmpeg core module libavFormat and libavCodec analysis

  • FFmpeg structure learning (A) : AVFormatContext analysis
  • FFmpeg structure learning part II: AVStream analysis
  • FFmpeg structure learning part III: AVPacket analysis
  • FFmpeg structure learning (4) : AVFrame analysis
  • FFmpeg structure learning: AVCodec analysis
  • FFmpeg structure learning (VI) : AVCodecContext analysis
  • FFmpeg structure learning (7) : AVIOContext analysis
  • FFmpeg structure learning (8) : relationships between important structures in FFmpeg

Summary: The above is about the audio and video information strongest arrangement and summary (and part of the content is not screenshots out), I believe it should be the most complete network, all the information is free to share to everyone, but also save on the audio and video interested partners looking for information around the time.

If that’s what you needGet it for free here.

The road to being an Android architect is a long one. Learning is like rowing upstream. If you don’t advance, you’re going to fall back. If you slack off, you’re going to fall back. Be a diligent and upward person, study harder, seize the center, ning jing not miscellaneous, ning designed not more.

Sorting is not easy, feel helpful friends can help like share support small series ~