The most simple iOS push stream code, video capture, soft coding (FAAC, X264), hard coding (AAC, H264), beauty, FLV coding, RTMP protocol, updated code parsing, you want to learn the knowledge here, willing to understand live technology students come to see!!

Source: https://github.com/hardman/AWLive

# description SPS/PPS/AudioSpecificConfig sps&pps/AudioSpecificConfig paper has mentioned many times before.

Sps&pps is a concept in H264, which contains some coding information, such as profile, image size and so on. In FLV, contains sps&pps part called AVC Sequence header (namely AVCDecoderConfigurationRecord, refer to the ISO – 14496-15 AVC file format).

AudioSpecificConfig is a concept in AAC that contains audio information such as sampling rate, number of channels, etc. The part of the FLV that contains AudioSpecificConfig is called the AAC Sequence header (AudioSpecificConfig, see ISO-14496-3 Audio).

Both data formats can be referred to in standard documentation or blog posts on the web, but here’s how to capture and process the data in hard/soft coding.

As you can see, these two concepts are actually encoded as a configuration file, holding some common attributes of the subsequent audio and video data.

sps&pps

After h264 coding, spS&PPS data can be obtained directly.

The code for obtaining SPS&PPS data by soft coding is in aw_x264.c

Static void aw_encode_x264_header(aw_x264_context *aw_ctx) x264_encoder_headers(aw_ctx->x264_handler, &aw_ctx->nal, &aw_ctx->nal_count); // Save the spS&PPS data to aw_ctx->sps_pps_data // Save the SPS PPS data uint8_t *sps_bytes = NULL; int8_t sps_len = 0; uint8_t *pps_bytes = NULL; int8_t pps_len = 0; int i = 0;for (; i < aw_ctx->nal_count; i++) {
        if (aw_ctx->nal[i].i_type == NAL_SPS) {
            sps_bytes = (uint8_t *)aw_ctx->nal[i].p_payload + 4;
            sps_len = aw_ctx->nal[i].i_payload - 4;
        }else if(aw_ctx->nal[i].i_type == NAL_PPS){
            pps_bytes = (uint8_t *)aw_ctx->nal[i].p_payload + 4;
            pps_len = aw_ctx->nal[i].i_payload - 4;
        }
    }
    
    aw_data *avc_decoder_record = aw_create_sps_pps_data(sps_bytes, sps_len, pps_bytes, pps_len);
    memcpy_aw_data(&aw_ctx->sps_pps_data, avc_decoder_record->data, avc_decoder_record->size);
    free_aw_data(&avc_decoder_record);
}
Copy the code

Hard-coded SPS&PPS data can be retrieved via keyframes. The code is in AWHWH264Encoder. M

/ / a hard-coded h264 obtain sps&pps data static void vtCompressionSessionCallback (void * CM_NULLABLE outputCallbackRefCon, void * CM_NULLABLEsourceFrameRefCon, OSStatus status, VTEncodeInfoFlags infoFlags, CM_NULLABLE CMSampleBufferRef sampleBuffer ){ ... . . . // Whether it is a keyframe, keyframe and non-keyframe should be clearly distinguished. Push flow should also be noted. BOOL isKeyFrame = ! CFDictionaryContainsKey( (CFArrayGetValueAtIndex(CMSampleBufferGetSampleAttachmentsArray(sampleBuffer,true), 0)), kCMSampleAttachmentKey_NotSync); // SPS PSS are also part of H264. They can be considered as special H264 video frames, which hold some necessary information about H264 video. // H264 video is difficult to parse without this part of data. // For data processing, SPS PPS data can be placed as a normal H264 frame at the front of the H264 video stream. BOOL needSpsPps = NO; // select sps&pps only onceif(! encoder.spsPpsData) {if(isKeyFrame) {// Get avcC, this is the SPS and PPS data we want. // To save the data to a file, add [0 0 0 1] 4 bytes before the data and write it to the top of the H264 file. // If you push the stream, put this data into the FLV data area. CMFormatDescriptionRef sampleBufFormat = CMSampleBufferGetFormatDescription(sampleBuffer); NSDictionary *dict = (__bridge NSDictionary *)CMFormatDescriptionGetExtensions(sampleBufFormat); encoder.spsPpsData = dict[@"SampleDescriptionExtensionAtoms"] [@"avcC"]; } needSpsPps = YES; }... . . .Copy the code

After obtaining SPS&PPS data successfully, you can use aw_encoder_create_sps_pps_tag in aw_SW_X264_encoder. C to create a corresponding video tag, and then send it as a common video tag.

Extern aw_flv_video_tag *aw_encoder_create_sps_pps_tag(aw_data *sps_pps_data){// Create a common video tag aw_flv_video_tag *sps_pps_tag = aw_sw_encoder_create_flv_video_tag(); // sps_ppS_tag ->frame_type = aw_flv_v_frame_type_key; Sps_pps_tag ->h264_package_type = aw_FLv_v_h264_packet_type_seq_header; Sps_pps_tag -> h264_comPOSItion_time = 0; Sps_pps_tag -> config_record_DATA = copy_AW_data (sps_pps_data); Sps_pps_tag ->common_tag.timestamp = 0; Sps_pps_tag ->common_tag.data_size = sps_pps_data->size + 11 + SPs_ppS_tag ->common_tag.header_size; / / returnreturn sps_pps_tag;
}

Copy the code

#AudioSpecificConfig

After the AAC soft coding library FAAC is initialized, AudioSpecificConfig data can be obtained directly in aw_faac.c.

Static void aw_open_faac_enc_handler(aw_faac_context *faac_ctx){// Faac_ctx ->faac_handler = faacEncOpen(faac_ctx->config.sample_rate, faac_ctx->config.channel_count, &faac_ctx->max_input_sample_count, &faac_ctx->max_output_byte_count); . . . . // Set faac faacEncSetConfiguration(faac_ctx->faac_handler, faac_config); Uint8_t *audio_specific_data = NULL; // Use this method to obtain AudioSpecificConfig. unsigned long audio_specific_data_len = 0; faacEncGetDecoderSpecificInfo(faac_ctx->faac_handler, &audio_specific_data, &audio_specific_data_len); . . }Copy the code

In addition, AudioSpecificConfig data structures are very simple and you can easily construct one yourself. Refer to AWHWAACEncoder createAudioSpecificConfigFlvTag function of m.

- (aw_flv_audio_tag *) createAudioSpecificConfigFlvTag {/ / AudioSpecificConfig contains three elements: Profile, sampleRate, channelCount // Structure: Profile (5bit)-sampleRate(4bit)-channelCount(4bit)- Empty (3bit) Uint8_t profile = kMPEG4Object_AAC_LC; uint8_t sampleRate = 4; uint8_t chanCfg = 1;
    uint8_t config1 = (profile << 3) | ((sampleRate & 0xe) >> 1);
    uint8_t config2 = ((sampleRate & 0x1) << 7) | (chanCfg< < 3); Config_data aw_data *config_data = NULL; data_writer.write_uint8(&config_data, config1); data_writer.write_uint8(&config_data, config2); . . . . }Copy the code

After you have AudioSpecificConfig data, you can use aw_encoder_create_audio_specific_config_TAG in aw_sw_FAac_encoder. C to create the corresponding FLV audio tag. It can then be sent as a normal Audio tag.

extern aw_flv_audio_tag *aw_encoder_create_audio_specific_config_tag(aw_data *audio_specific_config_data, Aw_faac_config *faac_config){// Create normal audio tag aw_flv_audio_tag *audio_tag = aw_sw_encoder_create_flv_audio_tag(faac_config); //AudioSpecificConfig data at the same location as normal audio tags audio_TAG ->config_record_data = copy_AW_data (audio_specific_config_data); // Timestamp 0 audio_tag->common_tag.timestamp = 0; Audio_tag ->common_tag.data_size = Audio_specific_config_data ->size + 11 + Audio_TAG ->common_tag.header_size;return audio_tag;
}
Copy the code

After the RTMP connection is successful, you must send the tags corresponding to spS&PPS and AudioSpecificConfig first; otherwise, videos cannot be played.

The article lists

  1. 1 hour to learn: the simplest iOS live push stream (A) project introduction
  2. 1 hour learning: The simplest iOS Live Streams (II) Code Architecture Overview
  3. 1 hour to learn: the simplest iOS live push stream (3) Using the system interface capture audio and video
  4. 1 hour learning: the simplest iOS live push stream (4) how to use GPUImage, how to beauty
  5. 1 hour learning: the simplest iOS live stream push (5) yuV, PCM data introduction and acquisition
  6. 1 hour to learn: the simplest iOS live push stream (6) H264, AAC, FLV introduction
  7. Learn in 1 hour: The simplest iOS Live Stream push (7) H264 / AAC hard coding
  8. 1 hour to learn: The simplest iOS live Push stream (8) H264 / AAC soft coding
  9. Learn in 1 hour: The simplest iOS live stream push (nine) FLV encoding with audio and video timestamp synchronization
  10. 1 hour learning: The simplest iOS live push stream (10) Librtmp usage introduction
  11. Introduction to SPS&PPS and AudioSpecificConfig (End)