FFmpeg supports calling platform hardware for decoding since version 3.1, which means that MediaCodec on Android can now be called from FFmpeg’S C code.
There are corresponding instructions on the official website, and the address is as follows:
Trac.ffmpeg.org/wiki/HWAcce…
As you can see from the picture, MediaCodec is not only supported on Android, VideoToolbox is also supported on iOS and Direct3D 11 is supported on Windows.
Note: Android MediaCodec currently only supports decoding, not encoding.
However, to see if it works, do a simple demo, and the complete code will be shown at the end.
The first is FFmpeg compilation. It is compiled with a number of switching options. Make sure mediacodec is enabled, as follows:
--enable-mediacodec
--enable-decoder=h264_mediacodec
--enable-decoder=hevc_mediacodec
--enable-decoder=mpeg4_mediacodec
--enable-hwaccel=h264_mediacodec
Copy the codeIt can be seen that Mediacodec supports three encoding formats h264, HEVC, MPEG4 options, not within the scope or consider soft solution.
On how to compile, I will not elaborate, and then write a special article to introduce.
After compiling the corresponding SO, you can print a list of AVCodec supported formats to see if mediacodec is available.
The specific code is as follows:
char info[40000] = {0};
AVCodec *c_temp = av_codec_next(NULL);
while(c_temp ! =NULL) {
if(c_temp->decode ! =NULL) {
sprintf(info, "%s[Dec]", info);
} else {
sprintf(info, "%s[Enc]", info);
}
switch (c_temp->type) {
case AVMEDIA_TYPE_VIDEO:
sprintf(info, "%s[Video]", info);
break;
case AVMEDIA_TYPE_AUDIO:
sprintf(info, "%s[Audio]", info);
break;
default:
sprintf(info, "%s[Other]", info);
break;
}
sprintf(info, "%s %10s\n", info, c_temp->name);
c_temp = c_temp->next;
}
Copy the codeIterate through AVCodec’s next pointer, then print out the result, and see the following to indicate that the compilation was successful.
H264_mediacodec and MPEG4_mediacodec are already supported.
And then we decode it. About FFmpeg decoding API call, in the public number previously published in the article said many times, will not explain the process in detail, a simple overview:
-
First open the file with the avformat_open_input method to get the AVFormatContext.
-
The video stream information for the file is then found by avformat_find_stream_info.
-
Get file related information and video stream information, mainly to get the encoding format information, and then find the corresponding decoder. You can also find the decoder directly using the avcodec_find_decoder_by_name method.
-
With the decoder, you can create a decoding context, AVCodecContext, and open the decoder with the avCoDEC_open2 method
-
Then read the contents of the file through av_read_frame for further decoding.
-
Next, the familiar AVCOdec_send_packet is sent to the decoder, and avCODEC_receive_frame gets the decoded data back from the decoder.
Focus on some differences between calling hardware decoding and ordinary decoding:
The first step is to set the JavaVM to FFmpeg in the JNI_OnLoad method loaded by so.
jint JNI_OnLoad(JavaVM *vm, void *res) {
av_jni_set_java_vm(vm, 0);
return JNI_VERSION_1_4;
}
Copy the codeWithout this step, there is no reflection to call Java methods.
Next, determine whether the hardware decoding type is supported or not, the above is determined by AVCodec, in fact, FFmpeg gives the definition of the hardware type, in the AVHWDeviceType enumeration variable.
enum AVHWDeviceType {
AV_HWDEVICE_TYPE_NONE,
AV_HWDEVICE_TYPE_VDPAU,
AV_HWDEVICE_TYPE_CUDA,
AV_HWDEVICE_TYPE_VAAPI,
AV_HWDEVICE_TYPE_DXVA2,
AV_HWDEVICE_TYPE_QSV,
AV_HWDEVICE_TYPE_VIDEOTOOLBOX,
AV_HWDEVICE_TYPE_D3D11VA,
AV_HWDEVICE_TYPE_DRM,
AV_HWDEVICE_TYPE_OPENCL,
AV_HWDEVICE_TYPE_MEDIACODEC,
AV_HWDEVICE_TYPE_VULKAN,
};
Copy the codeThe av_hwdevice_get_type_name method converts these enumerations to the corresponding string, such as AV_HWDEVICE_TYPE_MEDIACODEC. The corresponding string is mediacodec, which is also available in the source code:
static const char *const hw_type_names[] = {
[AV_HWDEVICE_TYPE_CUDA] = "cuda",
[AV_HWDEVICE_TYPE_DRM] = "drm",
[AV_HWDEVICE_TYPE_DXVA2] = "dxva2",
[AV_HWDEVICE_TYPE_D3D11VA] = "d3d11va",
[AV_HWDEVICE_TYPE_OPENCL] = "opencl",
[AV_HWDEVICE_TYPE_QSV] = "qsv",
[AV_HWDEVICE_TYPE_VAAPI] = "vaapi",
[AV_HWDEVICE_TYPE_VDPAU] = "vdpau",
[AV_HWDEVICE_TYPE_VIDEOTOOLBOX] = "videotoolbox",
[AV_HWDEVICE_TYPE_MEDIACODEC] = "mediacodec",
[AV_HWDEVICE_TYPE_VULKAN] = "vulkan"};Copy the codeAs with AVCodec, FFmpeg supports Mediacodec.
type = av_hwdevice_find_type_by_name(mediacodec);
if (type == AV_HWDEVICE_TYPE_NONE) {
LOGE("Device type %s is not supported.\n", mediacodec);
LOGE("Available device types:");
while((type = av_hwdevice_iterate_types(type)) ! = AV_HWDEVICE_TYPE_NONE)LOGE(" %s".av_hwdevice_get_type_name(type));
LOGE("\n");
return - 1;
}
Copy the codeMake sure mediacodec is supported, then the decoder is ready to use. As mentioned above, the main purpose of obtaining the file information is to open the decoder, but for example, the file encoding format H.264, and the decoder supporting H.264 in addition to soft solution, but also mediacodec how to choose?
For convenience, just go avcodec_find_decoder_by_name and find the mediacodec decoder.
if(! (decoder =avcodec_find_decoder_by_name("h264_mediacodec"))) {
LOGE("avcodec_find_decoder_by_name failed.\n");
return - 1;
}
Copy the codeAfter finding the decoder, we also need to get some configuration information of the decoder, such as what is the format decoded? Mediacodec decoding is NV21.
for (i = 0;; i++) {
// Decoder configuration
const AVCodecHWConfig *config = avcodec_get_hw_config(decoder, i);
if(! config) {LOGE("Decoder %s does not support device type %s.\n",
decoder->name, av_hwdevice_get_type_name(type));
return - 1;
}
if (config->methods & AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX &&
config->device_type == type) {
// Hard solution format
hw_pix_fmt = config->pix_fmt;
break; }}Copy the codeCurrently mediacodec decodes only in buffer mode, not directly resolving textures.
The next step is to add some hardware decoding contexts to the decoding context AVCodecContext.
static int hw_decoder_init(AVCodecContext *ctx, const enum AVHWDeviceType type)
{
int err = 0;
if ((err = av_hwdevice_ctx_create(&hw_device_ctx, type,
NULL.NULL.0))"0) {
LOGE("Failed to create specified HW device.\n");
return err;
}
// Hard decrypt the decoding context
ctx->hw_device_ctx = av_buffer_ref(hw_device_ctx);
return err;
}
Copy the codeAfter completing this series of operations, it is normal to decode, and get the contents of the decoded AVFrame.
If the AVFrame format is the same as the hardware decoding configuration, then it is converted to the normal YUV format using the AV_hwFrame_transfer_data method.
if (frame->format == hw_pix_fmt) {
/* retrieve data from GPU to CPU */
if ((ret = av_hwframe_transfer_data(sw_frame, frame, 0))"0) {
LOGE("Error transferring the data to system memory\n");
goto fail;
}
tmp_frame = sw_frame;
} else
tmp_frame = frame;
Copy the codeAfter completing these operations, the decoder has been successfully decoded and the actual operation is OK.
Welcome to follow the progress of wechat public account audio and video development, read more audio and video development articles ~~