As a continuous force in the audio and video industry for many years of video service manufacturers, TSINGSEE video developed the open source platform EasyDarwin, and a number of audio and video streaming platform, we develop streaming platform basically use ffmpeg, in ffmpeg, H264 must be converted into YUV420P before coding, This article will share how to convert H264 to YUV420P.

Here is yuV420:

The eight pixels are: [Y0 U0 V0] [Y1 U1 V1] [Y2 U2 V2] [Y3 U3 V3][Y5 U5 V5] [Y6 U6 V6] [Y7U7 V7] [Y8 U8 V8] The bit stream is: The pixels mapped by Y0 U0 Y1 Y2 U2 Y3 Y5 V5 Y6 Y7 V7 Y8 are: [Y0 U0 V5] [Y1 U0 V5] [Y2 U2 V7] [Y3 U2 V7][Y5 U0 V5] [Y6 U0 V5] [Y7U2 V7] [Y8 U2 V7]

Note: 12 bytes of code stream represent 8 pixels

Understanding requires drawing the matrix, as follows:

Bit stream data :(4:2:0 ~ 4:0:2)

Y0 U0
Y1
Y2 U2
Y3

Y5     V5
Y6
Y7	   V7
Y8
Copy the code

Mapping pixels:

Y0 U0 V5
Y1 U0 V5
Y2 U2 V7
Y3 U2 V7

Y5 U0 V5
Y6 U0 V5
Y7 U2 V7
Y8 U2 V7
Copy the code

YUV 4:2:0 sampling, each four Y’s share a set of UV components. So we need to decode H264 to YUV420. First, we need to initialize ffmpeg:

typedef struct __DECODER_OBJ { AVCodec *pVideoCodec; AVCodecContext *pVideoCodecCtx; AVFrame *mVideoFrame420; ///< video frame AVPicture pYuvFrame; struct SwsContext *pSws_ctx; uint8_t *pBuffYuv420; int codec; int width; int height; int outputFormat; int frameType; int numBytes; bool isInit; }DECODER_OBJ; avfilter_register_all(); avcodec_register_all(); /* register all codecs */ av_register_all(); // // register all decoder types decoderobj. pVideoCodec = avCOdec_find_decoder (avCodecId); //H264 if (NULL == decoderObj.pVideoCodec) { ReleaseVideoDecoder(); return -3; } decoderObj.pVideoCodecCtx = avcodec_alloc_context3(decoderObj.pVideoCodec); if (NULL == decoderObj.pVideoCodecCtx) { ReleaseVideoDecoder(); return -4; } AVDictionary *opts = NULL; int ret = avcodec_open2(decoderObj.pVideoCodecCtx, decoderObj.pVideoCodec, &opts); if (ret < 0) { ReleaseVideoDecoder(); return -5; } decoderObj.mVideoFrame420 = av_frame_alloc(); if (NULL == decoderObj.mVideoFrame420) { ReleaseVideoDecoder(); return -6; } avpicture_alloc(&decoderObj.pYuvFrame, AV_PIX_FMT_YUV420P, width, height); decoderObj.numBytes = av_image_get_buffer_size(AV_PIX_FMT_YUV420P, width, height, 1);Copy the code

YUV420: YUV420: YUV420: YUV420: YUV420: YUV420: YUV420

AVPacket pAvPacket = { 0 }; decoderObj.mVideoFrame420->pict_type = picType; pAvPacket.data = buf; pAvPacket.size = size; int res = 0; int gotPic = 0; res = avcodec_decode_video2(decoderObj.pVideoCodecCtx, decoderObj.mVideoFrame420, &gotPic, &pAvPacket); if (! gotPic) return -9; decoderObj.pSws_ctx = sws_getContext(decoderObj.pVideoCodecCtx->width, decoderObj.pVideoCodecCtx->height, decoderObj.pVideoCodecCtx->pix_fmt, decoderObj.pVideoCodecCtx->width, decoderObj.pVideoCodecCtx->height, AV_PIX_FMT_YUV420P, SWS_FAST_BILINEAR, NULL, NULL, NULL); sws_scale(decoderObj.pSws_ctx, decoderObj.mVideoFrame420->data, decoderObj.mVideoFrame420->linesize, 0, decoderObj.mVideoFrame420->height, decoderObj.pYuvFrame.data, decoderObj.pYuvFrame.linesize);Copy the code

Get decoderObj. PYuvFrame. Data [0] is YUV420 data. Finally, don’t forget to free memory. The code is as follows:

if (NULL ! = decoderObj.mVideoFrame420) { av_frame_free(&decoderObj.mVideoFrame420); decoderObj.mVideoFrame420 = NULL; } if (NULL ! = decoderObj.pVideoCodecCtx) { avcodec_close(decoderObj.pVideoCodecCtx); if (NULL ! = decoderObj.pVideoCodecCtx->priv_data) free(decoderObj.pVideoCodecCtx->priv_data); if (NULL ! = decoderObj.pVideoCodecCtx->extradata) free(decoderObj.pVideoCodecCtx->extradata); avcodec_free_context(&decoderObj.pVideoCodecCtx); decoderObj.pVideoCodecCtx = NULL; } if (NULL ! = &decoderObj.pYuvFrame) { avpicture_free(&decoderObj.pYuvFrame); //decoderObj.pYuvFrame = NULL; } if (NULL ! = decoderObj.pSws_ctx) { sws_freeContext(decoderObj.pSws_ctx); decoderObj.pSws_ctx = NULL; } if (NULL ! = decoderObj.pVideoCodec) { decoderObj.pVideoCodec = NULL; } if (NULL ! = decoderObj.pBuffYuv420) { av_free(decoderObj.pBuffYuv420); decoderObj.pBuffYuv420 = NULL; } if (decoderObj.pSws_ctx) { sws_freeContext(decoderObj.pSws_ctx); decoderObj.pSws_ctx = NULL; }Copy the code

End result: Use ffplay command to play yuV one frame of data

ffplay -i -video_size 700*700 $FILE
Copy the code

Among the streaming media platforms developed by TSINGSEE video, EasyNVR and EasyDSS are mature and stable video streaming media platforms, which can be directly downloaded and tested. The remake of EasyRTC is still under development, and its architecture has a new direction.