Many developers will be confused about which common interfaces need to be designed when doing video on Android platform or RTSP lightweight service and RTMP push module. This paper mainly takes Daniu Live SDK (official) as an example to briefly introduce all audio related interfaces of Android live push SDK. Interested developers can check it out.
1. Set audio encoding type, default AAC, if lower bit rate is required, set SPEEX encoding, considering the versatility of video operation, it is recommended to choose AAC encoding.
/** * Set audio encoder type **@param type: if with 1:AAC, if with 2: SPEEX
*
* @return {0} if successful
*/
public native int SmartPublisherSetAudioCodecType(long handle, int type);
Copy the code2. Set the AAC audio coding rate.
/** * Set audio encoder bit-rate, currently only valid for AAC **@paramKbit_rate: bit rate (KBPS). If the value is 0, the default bit rate is used and must be greater than or equal to 0 * *@return {0} if successful
*/
public native int SmartPublisherSetAudioBitRate(long handle, int kbit_rate);
Copy the code3. Set the SPEEX audio encoding quality to 8.
/** * Set speex encoder quality **@param quality: range with (0, 10), default value is 8
*
* @return {0} if successful
*/
public native int SmartPublisherSetSpeexEncoderQuality(long handle, int quality);
Copy the code4. Set noise suppression. Noise suppression can be used to eliminate the collected audio data such as noise background sound
/** * Set Audio Noise Suppression **@param isNS: if with 1:suppress, if with 0: does not suppress
*
* @return {0} if successful
*/
public native int SmartPublisherSetNoiseSuppression(long handle, int isNS);
Copy the code5. Set the audio automatic gain control. The purpose of AGC is that the system can automatically adjust the volume according to the sound size, so that the sound size is as consistent as possible
/** * Set Audio AGC **@param isAGC: if with 1:AGC, if with 0: does not AGC
*
* @return {0} if successful
*/
public native int SmartPublisherSetAGC(long handle, int isAGC);
Copy the code6. Set echo cancellation to eliminate audio echo in one-to-one interaction scenarios
/** * Set Audio Cancellation **@param isCancel: if with 1:Echo Cancellation, if with 0: does not cancel
*
* @param delay: echo delay(ms), if with 0, SDK will automatically estimate the delay.
*
* @return {0} if successful
*/
public native int SmartPublisherSetEchoCancellation(long handle, int isCancel, int delay);
Copy the code7. Set up the mixing, which can be used for example, two-channel audio mixing scenes. For details, please refer to the one-to-one interactive demo
/** * Set audio mixing, currently supports two audio mixing **@paramIs_mix: 1 to mix, 0 to not mix, default to not mix * *@return {0} if successful
*/
public native int SmartPublisherSetAudioMix(long handle, int is_mix);
Copy the code8. Real-time mute, real-time mute as the name implies, during the push process, pause the audio push
/** * Set mute or not during publish stream **@param isMute: if with 1:mute, if with 0: does not mute
*
* @return {0} if successful
*/
public native int SmartPublisherSetMute(long handle, int isMute);
Copy the code9. It is used to set the input volume. The range is [0.0, 5.0]
/** * Set the input volume, this interface is not recommended to call, in some special cases may be used, generally not recommended to amplify the volume **@paramIndex: usually 0 and 1. If there is no mixing, use 0. If there is mixing, set the volume to 0 and 1 respectively * *@paramThe default value is 1.0. The range is [0.0, 5.0]. The default value is 0@return {0} if successful
*/
public native int SmartPublisherSetInputAudioVolume(long handle, int index, float volume);
Copy the code10. For external real-time PCM data collection, considering the convenience of integration, we provide the following three interfaces. Note that the PCM audio data interface transmits audio data for the first time every 10ms
/** * Pass PCM audio data to SDK every 10ms **@paramPcmdata: PCM data, you need to use ByteBuffer. AllocateDirect allocation, ByteBuffer. IsDirect () is true. *@paramSize: PCM data size x@paramSample_rate: sampling rate. Currently, only {44100, 8000, 16000, 24000, 32000, 48000} is supported. 44100 * is recommended@paramChannel: indicates the channel. The current channel supports single-channel (1) and dual-channel (2). Single-channel (1) * is recommended@paramPer_channel_sample_number: this is passed in sample_rate/100 */
public native int SmartPublisherOnPCMData(long handle, ByteBuffer pcmdata, int size, int sample_rate, int channel, int per_channel_sample_number);
/** * Pass PCM audio data to SDK every 10ms **@paramPcmdata: PCM data, you need to use ByteBuffer. AllocateDirect allocation, ByteBuffer. IsDirect () is true. *@paramOffset: indicates the offset * of pcmdata@paramSize: PCM data size x@paramSample_rate: sampling rate. Currently, only {44100, 8000, 16000, 24000, 32000, 48000} is supported. 44100 * is recommended@paramChannel: indicates the channel. The current channel supports single-channel (1) and dual-channel (2). Single-channel (1) * is recommended@paramPer_channel_sample_number: this is passed in sample_rate/100 */
public native int SmartPublisherOnPCMDataV2(long handle, ByteBuffer pcmdata, int offset, int size, int sample_rate, int channel, int per_channel_sample_number);
/** * Pass PCM audio data to SDK every 10ms **@paramPcm_short_array: PCM data, short is native Endian order *@paramOffset: array offset *@paramLen: Number of array items *@paramSample_rate: sampling rate. Currently, only {44100, 8000, 16000, 24000, 32000, 48000} is supported. 44100 * is recommended@paramChannel: indicates the channel. The current channel supports single-channel (1) and dual-channel (2). Single-channel (1) * is recommended@paramPer_channel_sample_number: this is passed in sample_rate/100 */
public native int SmartPublisherOnPCMShortArray(long handle, short[] pcm_short_array, int offset, int len, int sample_rate, int channel, int per_channel_sample_number);
Copy the code11. Set the transmission of remote data, mainly used for one-to-one environment, pull the audio pulled to the stream end, callback up, through the push end of the interface Settings, complete the overall echo elimination
/**
* Set far end pcm data
*
* @param pcmdata : 16bit pcm data
* @param sampleRate: audio sample rate
* @param channel: auido channel
* @param per_channel_sample_number: per channel sample numbers
* @param is_low_latency: if with 0, it is not low_latency, if with 1, it is low_latency
* @return {0} if successful
*/
public native int SmartPublisherOnFarEndPCMData(long handle, ByteBuffer pcmdata, int sampleRate, int channel, int per_channel_sample_number, int is_low_latency);
Copy the code12. PCM audio data interface for mixing, note that the audio data is passed in every 10ms
/** * Pass PCM mix audio data to SDK every 10ms **@paramStream_index: currently, only 1 can be passed, other returns an error *@paramPcm_data: PCM data, you need to use ByteBuffer. AllocateDirect allocation, ByteBuffer. IsDirect () is true. *@paramOffset: indicates the offset * of pcmdata@paramSize: PCM data size x@paramSample_rate: sampling rate. Currently, only {44100, 8000, 16000, 24000, 32000, 48000} * is supported@paramChannels: The current channel supports single (1) and double (2) channels *@paramPer_channel_sample_number: this is passed in sample_rate/100 */
public native int SmartPublisherOnMixPCMData(long handle, int stream_index, ByteBuffer pcm_data, int offset, int size, int sample_rate, int channels, int per_channel_sample_number);
/** * Pass PCM mix audio data to SDK every 10ms **@paramStream_index: currently, only 1 can be passed, other returns an error *@paramPcm_short_array: PCM data, short is native Endian order *@paramOffset: array offset *@paramLen: Number of array items *@paramSample_rate: sampling rate. Currently, only {44100, 8000, 16000, 24000, 32000, 48000} * is supported@paramChannels: The current channel supports single (1) and double (2) channels *@paramPer_channel_sample_number: this is passed in sample_rate/100 */
public native int SmartPublisherOnMixPCMShortArray(long handle, int stream_index, short[] pcm_short_array, int offset, int len, int sample_rate, int channels, int per_channel_sample_number);
Copy the code13. Used for audio data docking after encoding, mainly used by forwarding modules
/** * Set audio data (AAC/PCMA/PCMU/SPEEX) **@param codec_id:
*
* NT_MEDIA_CODEC_ID_AUDIO_BASE = 0x10000,
* NT_MEDIA_CODEC_ID_PCMA = NT_MEDIA_CODEC_ID_AUDIO_BASE,
* NT_MEDIA_CODEC_ID_PCMU,
* NT_MEDIA_CODEC_ID_AAC,
* NT_MEDIA_CODEC_ID_SPEEX,
* NT_MEDIA_CODEC_ID_SPEEX_NB,
* NT_MEDIA_CODEC_ID_SPEEX_WB,
* NT_MEDIA_CODEC_ID_SPEEX_UWB,
*
* @paramData audio * *@param size data length
*
* @paramIs_key_frame Whether I frame, if with key frame, please set 1, otherwise, set 0, audio Ignore * *@param timestamp video timestamp
*
* @paramParameter_info is used to fill AAC special config information * *@param parameter_info_size parameter info size
*
* @return {0} if successful
*/
public native int SmartPublisherPostAudioEncodedData(long handle, int codec_id, ByteBuffer data, int size, int is_key_frame, long timestamp,ByteBuffer parameter_info, int parameter_info_size);
/** * Set audio data (AAC/PCMA/PCMU/SPEEX) **@param codec_id:
*
* NT_MEDIA_CODEC_ID_AUDIO_BASE = 0x10000,
* NT_MEDIA_CODEC_ID_PCMA = NT_MEDIA_CODEC_ID_AUDIO_BASE,
* NT_MEDIA_CODEC_ID_PCMU,
* NT_MEDIA_CODEC_ID_AAC,
* NT_MEDIA_CODEC_ID_SPEEX,
* NT_MEDIA_CODEC_ID_SPEEX_NB,
* NT_MEDIA_CODEC_ID_SPEEX_WB,
* NT_MEDIA_CODEC_ID_SPEEX_UWB,
*
* @paramData audio * *@paramOffset Data offset * *@param size data length
*
* @paramIs_key_frame Whether I frame, if with key frame, please set 1, otherwise, set 0, audio Ignore * *@param timestamp video timestamp
*
* @paramParameter_info is used to fill AAC special config information * *@param parameter_info_size parameter info size
*
* @return {0} if successful
*/
public native int SmartPublisherPostAudioEncodedDataV2(long handle, int codec_id,
ByteBuffer data, int offset, int size,
int is_key_frame, long timestamp,
byte[] parameter_info, int parameter_info_size);
/** * Set audio data (AAC/PCMA/PCMU/SPEEX) **@param codec_id:
*
* NT_MEDIA_CODEC_ID_AUDIO_BASE = 0x10000,
* NT_MEDIA_CODEC_ID_PCMA = NT_MEDIA_CODEC_ID_AUDIO_BASE,
* NT_MEDIA_CODEC_ID_PCMU,
* NT_MEDIA_CODEC_ID_AAC,
* NT_MEDIA_CODEC_ID_SPEEX,
* NT_MEDIA_CODEC_ID_SPEEX_NB,
* NT_MEDIA_CODEC_ID_SPEEX_WB,
* NT_MEDIA_CODEC_ID_SPEEX_UWB,
*
* @paramData audio * *@paramOffset Data offset * *@param size data length
*
* @paramIs_key_frame Whether I frame, if with key frame, please set 1, otherwise, set 0, audio Ignore * *@param timestamp video timestamp
*
* @paramParameter_info is used to fill AAC special config information * *@param parameter_info_size parameter info size
*
* @paramSample_rate The sampling rate, which must be the correct value * * if recording is required@paramChannels Number of channels. If recording is required, the correct value must be passed. It is usually 1 or 2 * *@return {0} if successful
*/
public native int SmartPublisherPostAudioEncodedDataV3(long handle, int codec_id,
ByteBuffer data, int offset, int size,
int is_key_frame, long timestamp,
byte[] parameter_info, int parameter_info_size,
int sample_rate, int channels);
Copy the code14. It is used to call back encoded audio data for easy connection with third-party systems
/** * Start output Encoded Data **@return {0} if successful
*/
public native int SmartPublisherStartOutputEncodedData(long handle);
/**
* Stop output Encoded Data
*
* @return {0} if successful
*/
public native int SmartPublisherStopOutputEncodedData(long handle);
Copy the codeconclusion
There are also 20 interfaces related to optical Audio, which shows how troublesome it is to develop a live push module for Android platform RTMP. Of course, because the RTMP push mentioned in this paper is actually used by a large number of companies or developers. If it is only for project requirements, many of them will not be used, so it can be appropriately simplified to make a small and beautiful module. Not losing is a good choice.