Douyin and Kuaishou quickly became popular in China, which also led to the boom of short videos in China. Short video recording, editing and other functions are a systematic and professional field. Over time, there were many ways to get to Rome, but not all of them were easy. This article will focus on introducing the pits encountered in the process of hard knitting and hard solving, and also provide some reference for readers when they construct the scheme.

The pit used by Camera

Although it has nothing to do with hard programming, it is impossible for short video clients to bypass this. Here are some of the problems I have encountered.

Some pit in API usage

Camera API is the embodiment of serious fragmentation of Android, different from iOS series, there are only several models. There are many Android vendors, and ROM is also a hundred flowers. The same API can be withdrawn on different models, which we need to pay attention to in the development process. Here are two simple examples of apis that can cause graphical anomalies.

  1. setRecordingHint

Sets recording mode hint. This tells the camera that the intent of the application is to record videos MediaRecorder.start(), not to take still pictures Camera.takePicture(Camera.ShutterCallback, Camera.PictureCallback, Camera.PictureCallback, Camera.PictureCallback). Using this hint can allow MediaRecorder.start() to start faster or with fewer glitches on output. This should be called before starting preview for the best result, but can be changed while the preview is active. The default value is false. The app can still call takePicture() when the hint is true or call MediaRecorder.start() when the hint is false. But the performance may be worse.

The official note above, the English is very simple, will not translate. It looks like a harmless API, but it’s actually deadly. This method requires you to set the video-size. If you do not set this, it will cause stretching on some models (OPPO A37f, MI 2 and some LG phones).

RecordingHint is intended to help record faster. Some ROMs are implemented without considering the difference between Preview Size and Video Size, such as recording at 480 864 resolution and output file sizes of 400 800. Then it is bound to make the overall picture stretch.

  1. setVideoStabilization

Enables and disables video stabilization. Use isVideoStabilizationSupported() to determine if calling this method is valid. Video stabilization reduces the shaking due to the motion of the camera in both the preview stream and in recorded videos, including data received from the preview callback. It does not reduce motion blur in images captured with takePicture. Video stabilization can be enabled and disabled while preview or recording is active, but toggling it may cause a jump in the video stream that may be undesirable in a recorded video.

According to the official documentation, this method can help improve the stability of recorded video. Use of time by # isVideoStabilizationSupported to determine whether can use. The PreviewCallback will have a black screen and no callbacks. Both of these apis are mentioned in the official documentation, but there is no hidden danger at all. The pit of Camera API call can be seen.

The Camera effect is inconsistent

If UI girls are visually demanding and need to be consistent across all mobile brands, it’s a disaster for Android Developer. For example, the difference between Google’s flagship Pixel 2XL and Huawei Honor is obvious for the same white balance effect. In general, on Android, we can’t force consistency, but rather give users enough freedom to produce satisfactory results within the range of capabilities provided by the machine.

Some of Oppo’s models have built-in beauty effects, while other older models don’t, so we can have different strategies for the default beauty treatment on these models.

Not all models add Exif information to the photos they take, so we can’t assume that all photos are vertical. If there is Exif in the photo, use this to determine the direction of the photo.

In summary, when using the Android Camera API, we need to have an understanding of the basic concepts of exposure, color temperature, and shutter release so that we can find the right solution for all kinds of strange situations.

Several pits of hard knitting and hard solving

MediaCodec may not support this

The first is that MediaCodec doesn’t necessarily support it! Especially some of the older models stood out, before the introduction of CTS testing, it was just (manual smile). It is recommended to judge the degree of hardware-hardware-unhardware-support where appropriate.

Here are links to test MediaCodec support in CTS, Tests/tests/media/SRC/android/media/CTS/EncodeDecodeTest. Java platform/CTS – Git at Google, you can refer to the code to determine the degree of support user model. If the model does not support Encoder, it will most likely crash when creating Encoder.

Surface doesn’t necessarily support it either

One of the data types MediaCodec supports is Surface. The Surface directly uses the Buffer of the native layer, which avoids mapping and copying and greatly improves the efficiency. Not every model supports the Surface format, however.

We can catch an exception when creating inputSurface. If there is no problem with the code, it is true that Surface is not supported on the machine.

Fake a shot of API

Note: the author used Google Pixel 2XL (Version 8.1) for local testing.

Codec itself is relatively inflexible, with few configurable places, but even in this case, the interface that looks good is very likely to be just a mirror, water, dream.

  1. According to the BitMode document, three kinds of bitrate modes are supported. CQ means to ensure frame rate and picture quality as far as possible. CBR tries to ensure the bit rate of each frame, but in some videos with large movements, it is easy to blur; VBR is a middle way between the first two. However, most models support only VRB. (If there are mistakes, welcome to correct)

  2. CodecLevel and CodecProfile can be set to change the encoding level. Different encoding levels have different effects. The higher the encoding level, the better the effect. We can use FFmPEG, or mediaInfo to see the coding level of the video. The author conducted tests on the level and profile supported by Pixel 2XL, and found that the effect encoded by all videos is Base Media/ Version 2. Another API that doesn’t really work, and I can only assume that this is Google foreshadowing for the future. Similarly, getComplexityRange API can have low and upper levels, but they are both 0.

There are many implicit restrictions

MediaCodec allows you to set a variety of color formats during Configure. However, the supported types are very limited, so you must check with getCapabilitiesForType before calling.

In general, most Camera Preview output supports NV21, so we can safely use NV21 in Camera#Parameters setPreviewFormat when using the Camera API. However, MediaCodec does not necessarily support NV21. These supported formats, the performance of different manufacturers is not the same oh, qualcomm support other companies really do not support. Bottom in the use of time, must make a good judgment. I forgot to mention that the getCapabilitiesForType API also handles exceptions (manual fan face X3).

One extra point I need to point out here is the 16-bit alignment thing. Soft soft solution, you can rest assured that the use of 540 960 such resolution, but in the hard hard solution, must use 544 960, if there is no 16-bit alignment, just don’t say what MOTO models directly spend the screen to show you things.

conclusion

In addition to the above problems, there are many other API limitations, such as MediaMuxer is basically tied to MPEG-4. In that case, hardcoding doesn’t work on Android. In fact, it is not so bad, in the speed of coding and decoding than soft coding soft solution on many fast, the author simply compared, can speed up 50%(no reference, the model is better, the gap is less). Here is just to write an article, share the pit encountered in the actual development, to each reader a little bit more reference.

Document information

  • Copyright Notice: Freely reproduced – Non-commercial – Non-derivative – Remain signed (Creative Commons 3.0 License)

  • Published date: September 1, 2018

  • Social media: weibo.com/woaitqs

  • Feed subscribe: www.woaitqs.cc/feed.xml