Zero, preface,

Today is relatively simple, first of all, the recording and playback of the four players

Let’s talk about how to use SoundPool and PCM to wav and how to use C++ files on Android, the legendary JNI and finally how to play variable and tune-sandhi

AudioRecord and MediaRecorder, AudioTrack and MediaPlayer

0. Four classes have been touched so far:

Day 1: AudioRecord, AudioTrack

Day 3: MediaRecorder(MediaPlayer)

1.AudioRecord(based on byte stream)
Advantages: Real-time processing of audio, suitable for streaming media and voice phone Disadvantages: The output is PCM voice data, which needs to be processed by yourself. If the data is saved as an audio file and cannot be played by the player, the data collected by PCM needs to be played by AudioTrack. AudioTrack can also convert PCM data into other formatsCopy the code
1.1: Audio source:int audioSource

1.2: Sound channel information:int channelConfig

The track information IN the recording is IN

1.3: Data output format:audioFormat

2.MediaRecorder(based on file recording)
Advantages: the MediaRecorder recording audio file is has been integrated with the tape after compression, coding, compression, etc., to support some of the audio format file (.. Arm,. Mp3, 3 gp, the aac, the mp4, the webm) operation is simple, need not oneself handle byte stream, the incoming file Disadvantages: Can not achieve real-time processing of audio, audio format output less.Copy the code
2.1: Audio source:int audio_source

Basically the same as AudioRecord

2.2: Output format:int output_format

2.3: Audio coding mode:int video_encoder

3.AudioTrack
AudioTrack can only play PCM streams that have been decoded (waV audio format files)Copy the code
3.1: Stream type:int streamType

3.2: mode:int mode
MODE_STREAM: Suitable for large files to write audio data to AudioTrack over and over again. The Buffer data supplied by the user --> the Buffer inside the AudioTrack, which can cause the introduction delay to some extent. MODE_STATIC: Suitable for small files where all data is passed to an internal buffer in AudioTrack via a single write call. This mode is suitable for files with low memory usage and high latency requirements, such as ring tones.Copy the code

3.3: Sound channel:int channelConfig

The track information of the recording is added OUT

3.4: Data output format:int audioFormat

This is the same as an AudioRecord

4.MediaPlayer
MediaPlayer can play a variety of sound files (MP3, W4A, AAC). MediaPlayer also instantiates AudioTrack in the Framework layer. The essence of MediaPlayer is to decode audio files in the Framework layer and generate PCM streams. The agent is then delegated to AudioTrack, and finally AudioTrack is passed to AudioFlinger for mixing before it is passed to hardware playbackCopy the code

2. Use of SoundPool

For frequent short sound effects, SoundPool is better

SoundPool source code is 616 lines, a lot of small, see pool must be pool

1. Initialization

Make an effect of playing two sound effects one at a time

private SoundPool mSp; private HashMap<String, Integer> mSoundMap = new HashMap<>(); private boolean isOne; private void initSound() { SoundPool.Builder spb = new SoundPool.Builder(); // Set the maximum number of synchronized streams that can be played at the same time spb.setMaxStreams(10); MSp = spb.build(); mSoundMap.put("effect1", mSp.load(this, R.raw.fall, 1)); mSoundMap.put("effect2", mSp.load(this, R.raw.luozi, 1)); }Copy the code
2. Play

Note: The resource will be loaded a little later, and will not work if the load and play are executed upstream or downstream

You can load it initially and play it later when there is an action, or you can listen for it to finish loading

Public void onViewClicked() {// source Id, left volume, right volume, priority, loop number, rate int Id = msoundmap.get (isOne? "effect1" : "effect2"); Msp.play (id, 1.0f, 1.0f, 1, 2, 1.0f); isOne = ! isOne; }Copy the code
3. The listener is loaded

Three arguments: soundPool, number, status (0==success)

mSp.setOnLoadCompleteListener((soundPool, sampleId, status) -> {
   
});
Copy the code

PCM and WAV

PCM can not be played by the player, waV can be played by the player

But their essence is almost the same, WAV is equivalent to a dress (file header), let the player recognize it, PCM to waV is not complicated, just add a head on the line, there are many online, see here

Conforms to the Resource Interchange FileFormat (RIFF) specification. All WAVs have a file header that specifies the encoding parameters of the audio stream. Data blocks are recorded in little-Endian byte order, and the identifier is not a string but a single symbolCopy the code
1. Code implementation: PcmToWavUtil
Public class PcmToWavUtil {/** * Cache audio size */ private int mBufferSize; / / private int mSampleRate; /** * mChannel */ private int mChannel; /** * @param sampleRate sample rate * @param channel channel * @param encoding Audio data format */ public PcmToWavUtil(int sampleRate, int channel, int encoding) { this.mSampleRate = sampleRate; this.mChannel = channel; this.mBufferSize = AudioRecord.getMinBufferSize(mSampleRate, mChannel, encoding); } /** * Convert PCM file to wav file ** @param inFilename Source file path * @param outFilename Target file path */ public void pcmToWav(String inFilename, String outFilename) { FileInputStream in; FileOutputStream out; long totalAudioLen; long totalDataLen; long longSampleRate = mSampleRate; int channels = mChannel == AudioFormat.CHANNEL_IN_MONO ? 1:2; long byteRate = 16 * mSampleRate * channels / 8; byte[] data = new byte[mBufferSize]; try { in = new FileInputStream(inFilename); out = new FileOutputStream(outFilename); totalAudioLen = in.getChannel().size(); totalDataLen = totalAudioLen + 36; writeWaveFileHeader(out, totalAudioLen, totalDataLen, longSampleRate, channels, byteRate); while (in.read(data) ! = -1) { out.write(data); } in.close(); out.close(); } catch (IOException e) { e.printStackTrace(); / / private void writeWaveFileHeader(FileOutputStream out, long totalAudioLen, Long totalDataLen, Long totalDataLen, long longSampleRate, int channels, long byteRate) throws IOException { byte[] header = new byte[44]; // RIFF/WAVE header header[0] = 'R'; header[1] = 'I'; header[2] = 'F'; header[3] = 'F'; header[4] = (byte) (totalDataLen & 0xff); header[5] = (byte) ((totalDataLen >> 8) & 0xff); header[6] = (byte) ((totalDataLen >> 16) & 0xff); header[7] = (byte) ((totalDataLen >> 24) & 0xff); //WAVE header[8] = 'W'; header[9] = 'A'; header[10] = 'V'; header[11] = 'E'; // 'fmt ' chunk header[12] = 'f'; header[13] = 'm'; header[14] = 't'; header[15] = ' '; // 4 bytes: size of 'fmt ' chunk header[16] = 16; header[17] = 0; header[18] = 0; header[19] = 0; // format = 1 header[20] = 1; header[21] = 0; header[22] = (byte) channels; header[23] = 0; header[24] = (byte) (longSampleRate & 0xff); header[25] = (byte) ((longSampleRate >> 8) & 0xff); header[26] = (byte) ((longSampleRate >> 16) & 0xff); header[27] = (byte) ((longSampleRate >> 24) & 0xff); header[28] = (byte) (byteRate & 0xff); header[29] = (byte) ((byteRate >> 8) & 0xff); header[30] = (byte) ((byteRate >> 16) & 0xff); header[31] = (byte) ((byteRate >> 24) & 0xff); // block align header[32] = (byte) (2 * 16 / 8); header[33] = 0; // bits per sample header[34] = 16; header[35] = 0; //data header[36] = 'd'; header[37] = 'a'; header[38] = 't'; header[39] = 'a'; header[40] = (byte) (totalAudioLen & 0xff); header[41] = (byte) ((totalAudioLen >> 8) & 0xff); header[42] = (byte) ((totalAudioLen >> 16) & 0xff); header[43] = (byte) ((totalAudioLen >> 24) & 0xff); out.write(header, 0, 44); }}Copy the code
2. Use:
private static final int DEFAULT_SAMPLE_RATE = 44100; Private static final int DEFAULT_CHANNEL_CONFIG = audioform.channel_in_mono; Private static final int DEFAULT_AUDIO_FORMAT = audioformat. ENCODING_PCM_16BIT; // Output format: 16-bit PCM String inPath = "/sdcard/ PCM recording /keke.pcm"; String outPath = "/sdcard/ PCM recording /keke.wav"; PcmToWavUtil pcmToWavUtil = new PcmToWavUtil(DEFAULT_SAMPLE_RATE,DEFAULT_CHANNEL_CONFIG,DEFAULT_AUDIO_FORMAT); pcmToWavUtil.pcmToWav(inPath,outPath);Copy the code

Four, speed play

0. Recall the introduction to sound from day 1: The three elements of sound
[1] the volume (loudness) acoustic vibration amplitude - A - db: [2] tone voice frequency (high - frequency - - frequency slow - pointed the bass sound sink) - f - Hz [3] tone: timbre) related to the material nature is harmonicCopy the code

Realization of speed change:

When playing, the sampling frequency is doubled to make the period change. For example, when the speed is twice, the sampling frequency is *2, and the period of the wave is halved. The original 2s wave can be played in 1s. Due to the change of sound frequency, the sound effect also changes accordingly. The frequency is slow, the bass is low, the sound sink is 2 times the speed is like some short video the speed of the voice change dub, 0.5 times the speed is like the monster roar...Copy the code
1. Code implementation

The code to play the PCM stream was implemented on day 1, so let’s make some changes based on that

AudioTrack can set the sampling frequency when reading PCM, and then extract variables and pass them in

Public void startPlay(String path, int rate) {try {isStart = true; setPath(path); / / set the path -- flow generated dis mMinBufferSize = AudioTrack. GetMinBufferSize (rate, DEFAULT_CHANNEL_CONFIG. AudioFormat.ENCODING_PCM_16BIT); AudioTrack = new AudioTrack(DEFAULT_STREAM_TYPE, rate, DEFAULT_CHANNEL_CONFIG, DEFAULT_AUDIO_FORMAT, mMinBufferSize * 2, DEFAULT_PLAY_MODE); mExecutorService.execute(new PlayRunnable()); } catch (Exception e) {e.printStackTrace(); }}Copy the code
2. Used in the Activity

The layout is pretty simple. No more nonsense

private float rate = 1; / / SeekBar sliding to monitor mIdSb. SetOnSeekBarChangeListener (new SeekBar. OnSeekBarChangeListener () {@ Override public void onProgressChanged(SeekBar seekBar, int progress, boolean fromUser) { rate = progress / 100.f; setInfo(); } @Override public void onStartTrackingTouch(SeekBar seekBar) { } @Override public void onStopTrackingTouch(SeekBar seekBar) { } }); / / click play mIvStartPlay. SetOnClickListener (e - > { PCMAudioPlayerWithRate. GetInstance (). StartPlay ("/sdcard/PCM audio / 20190107075814. PCM ", (int) (44100 * rate)); });Copy the code

Five, some simple understanding of JNI

1. Create an Android project that supports C++. What’s the difference

2. App Gradle:

3. Cmakelists.txt

4. Follow suit

C++ file download address —– specific algorithm parsing address

5. Create native functions

Five, the change of pitch

Refer to the moOCs free tutorial for this section: see

1. Java classes

The two temporary float arrays are intended to correspond to C++ functions used to process data streams

/** * Author: Zhang Feng Jieteilie <br/> * Time: 2019/1/70007:9:50 <br/> * Email: [email protected]<br/> * Description: */ public class AudioEffect {private int mBufferSize; private byte[] mOutBuffer; private float[] mTempInBuffer; private float[] mTempOutBuffer; System.loadlibrary ("audio-effect"); } public AudioEffect(int bufferSize) { mBufferSize = bufferSize; mOutBuffer = new byte[mBufferSize]; mTempInBuffer = new float[mBufferSize/2]; mTempOutBuffer = new float[mBufferSize/2]; } @param rate @param rate @param rate @param rate @param rate @param rate @param rate @param rate @param rate @param rate @param rate @param rate @param rate @param rate @param rate @param rate */ public synchronized byte[] process(float rate,byte[] in,int simpleRate) { native_process(rate,in,mOutBuffer,mBufferSize,simpleRate,mTempInBuffer,mTempOutBuffer); return mOutBuffer; } private static native void native_process(float rate, byte[] in, byte[] out, int size, int simpleRate,float[] tempIn, float[] tempOut); }Copy the code
2. Data processing: smbpitchshift.cpp
#include <jni.h> extern "C" JNIEXPORT void JNICALL Java_top_toly_sound_audio_effect_AudioEffect_native_1process(JNIEnv *env, jclass type, jfloat rate, jbyteArray in_, jbyteArray out_, jint size, jint simpleRate, jfloatArray tempIn_, Jbyte *in = env->GetByteArrayElements(in_, NULL); jfloatArray tempOut_) {jbyte *in = env->GetByteArrayElements(in_, NULL); jbyte *out = env->GetByteArrayElements(out_, NULL); jfloat *tempIn = env->GetFloatArrayElements(tempIn_, NULL); jfloat *tempOut = env->GetFloatArrayElements(tempOut_, NULL); Byte [] float[] for (int I = 0; i < size; i += 2) { int lo = in[i] & 0x000000FF; Int hi = in[I + 1] &0x000000ff; Int frame = (hi << 8) + lo; TempIn [I >> 1] = (signed short) frame; // } smbPitchShift(rate, 1024, 1024, 4, simpleRate, tempIn, tempOut); //float[] byte for (int I = 0; i < size; i += 2) { int frame = (int) tempOut[i >> 1]; out[i] = (jbyte) (frame & 0x000000FF); Out [I + 1] = (jbyte) (frame >> 8); Env ->ReleaseByteArrayElements(in_, in, 0); env->ReleaseByteArrayElements(out_, out, 0); env->ReleaseFloatArrayElements(tempIn_, tempIn, 0); env->ReleaseFloatArrayElements(tempOut_, tempOut, 0); }Copy the code
3. Play convection operation:In the PCMAudioPlayerWithRat
//private float rate = 1; Public void setRate(float rate) {this.rate = rate; ----- if (mAudioEffect == null) {L.d(mMinBufferSize + L.l()); //7072 mAudioEffect = new AudioEffect(2048); } tempBuffer = rate == 1? tempBuffer : mAudioEffect.process(rate, tempBuffer, DEFAULT_SAMPLE_RATE);Copy the code
4. Play in the Activity

The layout is basically the same, you can set the rate of sound change when you drag, and click to play

5. Episode

One problem, the squeaky sound, was tested and found to be a bufferSize pot

If the buffer size on the read is the same as the AudioEffect buffer size, it will squeak a little bit and after a little bit of tuning, mMinBufferSize/3.388598 is fine, a little squeak and then finally print mMinBufferSize = 7072, 7072*/3.388598=2086.99 and then inspiration, is not 2048? —— then perfect solution… It took me over an hour… Okay, that’s it. I can seriously say this… Just touched the Android multimedia door here (that is, no entry)

Postscript: Jie wen standard

1. Growth record and Errata of this paper
Program source code The date of note
V0.1 – making The 2018-1-7 Android Multimedia SoundPool+ PCM stream audio operation
2. More about me
Pen name QQ WeChat hobby
Zhang Feng Jie te Li 1981462002 zdl1994328 language
My lot My Jane books I’m the nuggets Personal website
3. The statement

1—- This article is originally written by Zhang Fengjie, please note if reproduced

2—- welcome the majority of programming enthusiasts to communicate with each other 3—- personal ability is limited, if there is something wrong welcome to criticize and testify, must be humble to correct 4—- see here, I thank you here for your love and support