purpose

Prepare a package format file (mp4.mp3, etc.), read the audio track data, use DSP decoded into PCM file, intercept one of the audio, save into a new file

This exercise will prepare you for the video remix and familiarize yourself with the API

steps

  1. Read the track information of MP3 files through MediaExtractor
  2. Read PCM raw stream data of audio track and save it as PCM file
  3. Manipulate the PCM file to intercept the fragment

The relevant knowledge

  • MediaExtractor: audio and video files can be separated from the audio and video, and extract the corresponding data channel, and then operate.

Important methods:

  1. SetDataSource (String path) sets the data source
  2. GetTrackCount () to get the total number of tracks. The tracks of audio and video files are video tracks, audio tracks, letter tracks, etc. This method is used to traverse track information
  3. MediaFormat getTrackFormat(int index), returns a MediaFormat object, which is actually a Map
  4. SeekTo (long timeUs, @seekmode int mode), seek to a certain time, corresponding to the progress bar on the audio and video app
  5. Advance (), advances to the next sample, skipping the current
  6. GetSampleTime () to get the microsecond time of the current sample
  7. ReadSampleData (@nonNULL ByteBuffer byteBuf, int offset) Reads data
  • MediaMuxer: Generates an audio or video file; You can also mix audio and video into a single audio and video file

Important methods:

A link to the

Parse and encapsulate MP4 files with MediaExtractor and MediaMuxer API

MediaExtractor and MediaCodec use method

Practice code

  • Separate the audio track from the video file and save it as WAV
/** * @author : kai.mao * @date : 2021/1/23 */ public class AudioClipHelper { private static AudioClipHelper INSTANCE; private MediaExtractor mMediaExtractor; private AudioClipHelper() { } public static AudioClipHelper getInstance() { if (INSTANCE == null) { INSTANCE = new AudioClipHelper(); } return INSTANCE; } public void setDataSource(String filePath) { if (mMediaExtractor == null) { mMediaExtractor = new MediaExtractor(); } try { mMediaExtractor.setDataSource(filePath); } catch (IOException e) { e.printStackTrace(); }} public void readAuido(int startTime,int endTime) throws IOException {log.e ("David", "start conversion "); int maxBufferSize; Int audioIndex = selectAudioTrack(mMediaExtractor); / / 2. Specify the audio track mMediaExtractor. SelectTrack (audioIndex); mMediaExtractor.seekTo(startTime, MediaExtractor.SEEK_TO_CLOSEST_SYNC); / / 3. Decoding audio data, from seal to format - > bare PCM data MediaFormat originAudioFormat = mMediaExtractor. GetTrackFormat (audioIndex); if (originAudioFormat.containsKey(MediaFormat.KEY_MAX_INPUT_SIZE)){ maxBufferSize = originAudioFormat.getInteger(MediaFormat.KEY_MAX_INPUT_SIZE); }else { maxBufferSize = 100 * 1000; } ByteBuffer buffer = ByteBuffer.allocateDirect(maxBufferSize); // Decodes the audio data using the MediaCodec decoder MediaCodec.createDecoderByType(originAudioFormat.getString(MediaFormat.KEY_MIME)); mediaCodec.configure(originAudioFormat,null,null,0); File pcmFile = new File(Environment.getExternalStorageDirectory(), "out.pcm"); FileChannel writeChannel = new FileOutputStream(pcmFile).getChannel(); mediaCodec.start(); MediaCodec.BufferInfo info = new MediaCodec.BufferInfo(); int outputBufferIndex = -1; / / read the data after decoding the while (true) {/ / access to free the buffer int decodeInputIndex = mediaCodec. DequeueInputBuffer (100000); DecodeInputIndex >= 0 (decodeInputIndex >= 0) {long sampleTimeUs = mMediaExtractor.getSampleTime(); if (sampleTimeUs == -1){ break; }else if (sampleTimeUs < startTime){ mMediaExtractor.advance(); continue; }else if (sampleTimeUs > endTime){ break; } / / get the data info. Size = mMediaExtractor. ReadSampleData (buffer, 0); info.presentationTimeUs = sampleTimeUs; info.flags = mMediaExtractor.getSampleFlags(); Byte [] content = new byte[buffer.remaining()]; buffer.get(content); FileUtils.writeContent(content); ByteBuffer inputByteBuffer = mediaCodec.getInputBuffer(decodeInputIndex); inputByteBuffer.put(content); mediaCodec.queueInputBuffer(decodeInputIndex,0,info.size,info.presentationTimeUs,info.flags); The e (" David ", "presentationTimeUs:" + sampleTimeUs); mMediaExtractor.advance(); } / / remove the bare PCM data after decoding outputBufferIndex = mediaCodec. DequeueOutputBuffer (_000 info, 100); while (outputBufferIndex >= 0){ ByteBuffer decodeOutputBuffer = mediaCodec.getOutputBuffer(outputBufferIndex); writeChannel.write(decodeOutputBuffer); //MP3 1 pcm2 mediaCodec.releaseOutputBuffer(outputBufferIndex, false); outputBufferIndex = mediaCodec.dequeueOutputBuffer(info, 100_000); } } writeChannel.close(); mMediaExtractor.release(); mediaCodec.stop(); mediaCodec.release(); File wavFile = new File(Environment.getExternalStorageDirectory(),"output.mp3" ); new PcmToWavUtil(44100, AudioFormat.CHANNEL_IN_STEREO, 2, AudioFormat.ENCODING_PCM_16BIT).pcmToWav(pcmFile.getAbsolutePath() , wavFile.getAbsolutePath()); Log. I ("David", "mixAudioTrack: Conversion completed "); } private int selectAudioTrack(MediaExtractor mediaExtractor) { int trackTotal = mediaExtractor.getTrackCount(); for (int index = 0; index < trackTotal; index++) { MediaFormat mediaFormat = mediaExtractor.getTrackFormat(index); If (mediaformat.getString (mediaformat.key_mime).startswith ("audio/")) {return index; } } return -1; }}Copy the code