This article mainly summarizes the Android terminal camera related development experience.

As we all know, the Android platform is not only highly fragmented, but also different hardware configurations of different phones lead to a lot of problems in the development of some modules, including the camera module. Why is that? First of all, the Android system currently provides two sets of Camera APIS, among which Camera 2 API is provided from Android 5.0(API Level 21). The Camera 2 API is now available on many models. However, this is not the case because of the second point below. Many Android phones do not support the Camera 2 API properly, and even many new phones still only support the old Camera API! As a result, Camera development has to switch different Camera APIS according to the actual situation of mobile phones.

Cameraview is an independent library provided by Google. If you need to preview the Camera, switch the front and rear cameras, switch the flash, switch the scale of the preview image, and take a picture, it is very difficult to build such a Camera module from scratch. Then this small library is a good choice. CameraView is a project to help Android developers quickly create a camera interface that can adapt to different Android systems and different Android devices, and contains a variety of basic features. Everything else related to Camera is handled by it.

Now that we have the cameraView wheel, is that the end of this article? Tucson broken! As mentioned earlier, this library is an unofficial library, so it has not been updated for a long time. There are a lot of known bugs in issues and no one has solved them! But what? You have to forgive it, or do you have to masturbate? Cameraview (Camera 2) cameraView (Camera 2) CameraView (Camera 2) CameraView (Camera 2) cameraView (Camera 2)

Back to the point, I investigated several wheels before developing the requirements of the camera module, and finally decided to use the library cameraView, because it is small and simple, without unnecessary waste code or functions, and it is also convenient for me to customize the camera interface. There are also several Camera module packages with high star on Github, but I think they are a little complicated and may not be easy to customize.

This paper mainly tells me what problems I encountered in the process of making camera module requirements or using CameraView and the corresponding solutions. Finally, I made some enhancement of CameraView. If you are interested, you can take a look at the library CameraView. The major improvements are explained in the README documentation, but probably the most useful ones are relogging important paths and fixing a few post-launch crash bugs.

1. Describe the design of cameraView components

By reading the source code for the CameraView component, the internal design is shown in the diagram below:

The core class is the custom CameraView component, which supports XML to set the camera, aspect ratio, flash, and other attributes. CameraViewImpl and CameraViewImpl are two abstract classes to do all the camera-related work.

PreviewImpl is used to implement camera preview. It may be implemented internally using SurfaceView or TextureView, so it has two subclasses, TextureViewPreview and SurfaceViewPreview. Since TextureView has only been available since Android 4.0(API Level 14) (TexturView is an enhanced version of SurfaceView), TextureViewPreview is used after Android 4.0. Prior to Android 4.0 only SurfaceViewPreview was available.

CameraViewImpl CameraViewImpl CameraViewImpl CameraViewImpl CameraViewImpl CameraViewImpl CameraViewImpl Camera2 is provided for Android 5.0(API Level 21) and above, Camera2Api23 is inherited from Camera2, is provided for Android 6.0(API Level 23) and above.

Create PreviewImpl and CameraViewImpl as follows:

It will be much clearer to read the source code of CameraView after you understand the previous diagram. The other classes are built around CameraView.

Size describes the width and height, such as 800×600, 400×300 or 640×360. AspectRatio describes the AspectRatio of Size. For example, 800×600 and 400×300 are both 4:3, but 640×360 is 16:9. SizeMap maintains a list of mappings between AspectRatio and Size, for example, {” 4:3 “: {800×600, 400×300},” 16:9 “: {640×360}}. DisplayOrientationDetector is used to monitor the camera interface screen rotation, and then inform related components respond to the screen rotation changes, such as adjust the preview picture.

2. Choose between Camera1 and Camera2

CameraViewImpl: CameraViewImpl: CameraViewImpl: CameraViewImpl: CameraViewImpl: CameraViewImpl: CameraViewImpl There is one more small detail here. What if I selected Camera2 and found that Camera2 support was weak when I started the camera? From the source, at this point CameraView will automatically demote it to Camera1, and then try to restart the camera using the camera parameters you set earlier. This is the case on many phones, from the models I tested, mi 5/4C, Vivo X7, Meizu MX6/Pro6, Galaxy S4, Huawei H60-L11, etc. (the table below records this data).

When I saw this code, I was shocked, whew, and this operation, 666, and then I thought, and I smiled, because I realized that this code was obviously optimizable. First of all, the PreviewImpl was created before, so there is no need to change the PreviewImpl by switching the CameraViewImpl, so there is no need to call createPreviewImpl again; Secondly, for a phone, what if it is Android 5.0 or higher, but the Camera 2 API support is poor? If you follow the logic of this code, it will cause the phone to try Camera2 every time it starts the camera, and then try Camera1 again, which will obviously slow the camera down. CameraViewImpl: CameraViewImpl: CameraViewImpl: CameraViewImpl: CameraViewImpl: CameraViewImpl: CameraViewImpl: CameraViewImpl: CameraViewImpl Ha ha, really witty as I 😎

The corresponding modifications have been reflected in my improved CameraView library, and the approximate code is as follows:

3. AspectRatio selection

Let’s look at AspectRatio, which is essentially the AspectRatio of the image, maybe 4:3, maybe 16:9, maybe some other ratio. In addition, we also need to know that the camera module here has several places to set the aspect ratio, here we recommend reading the Android camera development pit article, which detailed analysis of the following three dimensions between the relationship:

SurfaceView/TextureView Size: The size of the View used to display the camera’s preview image in the custom camera app, which is the size of the screen when it fills the full screen. The preview images shown here by SurfaceView/TextureView will be referred to as mobile preview images. AdjustViewBounds (adjustViewBounds) There is a property in the source for CameraView. If it is set to false, adjustViewBounds will flush the adjustaView space. The preview image will be displayed at this AspectRatio based on the AspectRatio setting.

Previewsize: The data size of the preview frame provided by the camera hardware. The preview frame data is passed to SurfaceView to display the preview image. The preview image corresponding to the preview frame data here is tentatively called the camera preview image.

Picturesize: The size of the frame data provided by the camera hardware. Frame data can be generated into bitmap files, which can be saved as.jpg or.png images. The image corresponding to the frame data captured here is called the camera image.

In order to ensure the normal display and operation of the camera module, it is usually recommended to have the same aspect ratio of the above three sizes. If the ratio is inconsistent, it may lead to image distortion, and the ratio should be 4:3 or 16:9, which are generally supported, otherwise the output results will be strange. For example, Huawei H60-L11, It doesn’t support 16:9 images, but it does support 4:3 images.

After perusing the original AspectRatio, Previewsize, and Picturesize selection code of CameraView, I feel that this code is not strict enough. For example, the output image size defaults to the maximum output size at this scale.

To be honest, this code is really difficult to write because different applications have different requirements. For example, my product requires that the output image should be 1920×1080 size (16:9), so I would prefer 16:9 ratio to the default 4:3 ratio in CameraView. So I’ve modified the original AspectRatio selection and Previewsize/Picturesize selection code for CameraView to use 16:9 ratio in preference to 4:3 ratio if not supported. The output image size of 1920×1080 is preferred when the ratio of 16:9 is supported. If not, try another size. The logic is similar under the ratio of 4:3, and the general code is as follows :(different applications should be modified according to their own needs)

The following table is the data collected by me using some test phones. It is not difficult to see from the table data that, except for Pixel, the latest son of Google, other phones have weak support for Camera 2 API, which leads to switching to Camera1. In addition, most phones support 16:9 image ratio, and most phones also support 1920×1080 image size, but some phones do not support 1280×720 output size, even choose 4:3 ratio 2048×1536 output size.

Note 1: At that time, I did not pay attention to Preview Picture Size when collecting data, so this column is basically empty. Why did the Meizu MX 6 change from one size to another? Because the size of the preview image was 960×540 due to my scale and size selection strategy, the preview image was very blurred due to this size. Later debug found this problem and tried to adjust the strategy to make it 1920×1080, after adjustment, the display is no longer blurred.]

Note 2: However, even if the proportions of the three sizes are consistent, there are still some strange phenomena on some phones, such as this and this in the Issues list of CameraView, which means that the saved images are not the same as the ones seen in the preview! This phenomenon must occur on a Huawei Honor mobile phone. There is no good solution for the moment. Fortunately, there are not many models with the problem, so the solution can be postponed.]

4. Take pictures

There are also a number of potential pitfalls with camera photography, which we’ll discuss below. The following code fragment is the implementation of camera photography in Camera1 class. The general process is as follows: When the camera is on, if the camera can autoFocus, then call the autoFocus method first. After the autoFocus method is finished, call the takePictureInternal method to take the photo. If autofocus is not available, the takePictureInternal method is called to take the photo. TakePictureInternal method implementation is to see whether isPictureCaptureInProgress is false, if so then it is set to true, then immediately call takePicture photograph, After successful then isPictureCaptureInProgress set to false.

What’s wrong with this code? From my test, there are three main problems:

1. Invoking the autoFocus method on some mobile phones may be time-consuming: I tested a Meizu MX6 mobile phone and found that the focus was very slow. The interface displayed that the camera button was clicked, and the autoFocus lasted about 5-8 seconds, which was a very bad experience. To solve this problem, I set a minimum focusing time. If the mobile phone fails to focus within the minimum focusing time, it will directly call takePictureInternal to take photos. In other words, the picture effect may be sacrificed to obtain a better shooting experience.

2. IsPictureCaptureInProgress this variable problem: Because the debug another problem that I find a new problems caused by isPictureCaptureInProgress variables, scenario is that if users click on the photo, the photo results could appear immediately before press the Home button quit to the desktop, the timing is very difficult to control, but there are ways of repetition, Once reappeared, then the isPictureCaptureInProgress. Set (false) this sentence is not invoked, this will lead to couldn’t call takePicture again after taking pictures. The solution is in the stop method that Camera1 isPictureCaptureInProgress reset to false.

3. On some mobile phones, the autoFocus method will crash: this problem was found after applying gray scale, maybe there is something wrong with the autoFocus process. My processing here is to temporarily catch it, if there is an exception, I will directly call the takePictureInternal method. Before executing camera.autofocus or Camera. takepicture in the camera button event response, be sure to verify that the camera has the preview Surfaceview set and camera Preview enabled. Here is a method to judge the preview status: Camera. SetPreviewCallback is preview frame data callback function, it will be received in the SurfaceView Camera preview is invoked when the frame data, therefore the af and take pictures inside you can set whether to allow the symbol.”

The code for the improved takePicture process is shown below

5. Camera permissions

As you know, since Android 6.0, Android has introduced dynamic permissions, so if your app’s targetSDK is set to 23 or above, you’ll need to check whether camera permissions are granted at runtime, and apply for them if not. For systems under Android 6.0, simply declare camera permissions in the Androidmanifest.xml file.

Is it for real this time? You know in your heart that the answer must be no. Domestic mobile phone now customized after the system basically has its own rights management mechanism, often there is a system application “security center” to help manage rights, so also compatible with these different rights management mechanism.

The following checkCameraPermission method is used to check the camera permissions and start the camera process when permissions are granted. This method is called in the onResume method of the Activity (which contains the CameraView).

If the targetSDK is set below 23, then only the first if branch will be used. Let’s focus on this branch. The else branch below can be analyzed by referring to other documentation, such as everything developers need to know about Android M’s new runtime permissions.

Under the Android 6.0 system, ContextCompat. CheckSelfPermission this method returns the results must be true, if is the primary system, it is really has the permissions. But in many domestic systems, in fact, there is no, when the above code is executed to McAmeraviet.start (), the system will intercept this operation, and then pop up the system custom permission application dialog box, different, such as Xiaomi mobile phone, VIVO mobile phone and Huawei mobile phone has a 20-second countdown, There is no countdown on meizu’s phone.

If the countdown is over and you haven’t clicked permission, it means that you have refused, and then the camera will fail or be abnormal to open. If you fail to start the camera because the permissions are not granted, consider popping up a dialog box to inform the user, and then directing the user to the application’s corresponding permissions grant interface to enable the permissions. For details on where to jump, see this code, which handles the logic for different custom systems to jump to the corresponding permission granting interface.

It should be noted that there is an “Application” option in the Settings of the native system. After entering the interface, you can find the details of the corresponding application. However, only some systems support the permission to directly manage the application here, so it is not possible to let the user jump to here. It is worth noting that there is a bug in the Mi system. The MI system seems to support direct modification of permissions in the details of this application, but it is useless after modification of permissions, only to change permissions in the security center of the system is effective!

6. Output images

You think that’s all there is to different phones? NO! Samsung is telling you you’re still too young! One day, a test student comes up to you with a Samsung phone and asks you, “Why did I take a picture vertically, but when I upload it to the server and click on it, the picture is horizontally?” “At this time you take the phone, open the file management to find the save path of this picture, and then look at this picture, and find it is clearly vertical, at this time you will think that the pan must be thrown out, back,” this must be the bug of the background development students! He must have rotated the picture!” . He said, “I don’t rotate pictures. It’s not my pot.” And didn’t get back to you. That’s when you realize that Samsung has a long-standing bug that automatically rotates the image by 90! Cameraview is not compatible with this situation.

However, if you take a closer look at the cameraView code, you will see that it is not cameraView. The onPictureTaken method will eventually be called with byte[] data, Normally we just store this byte array in some file to get the picture taken. However, we did not check the EXIF information of this picture, because most of the time, the degree metadata is 0, but on Samsung mobile phone, whether you take a picture vertically or horizontally, the value is 90!

At this point you might be wondering, why is this picture that you see in file Management vertical? Apparently, Samsung’s built-in album (or file manager) takes into account the EXIF information when displaying an image, which is actually horizontal, and rotates back to vertical when shown to you. So what to do? Is there a special treatment for Samsung phones that take portrait shots?

What I do here is to save the data data to the picture, and then read its EXIF information. If its degree is not 0, then rotate the picture according to the degree information and save it again. (Note: this does not change degree to 0)

[Note: You can check out this issue about Samsung phones]

7. Manually focus

I didn’t know that at first, but when the interaction came out, I found that the cameraView component didn’t have the manual focusing function, but fortunately, some warm-heartened developers enhanced cameraView to support manual focusing, and even raised PR for the official CameraView. However, this manual focus code is basically available. The corresponding code submission record can be seen in lin18/ CameraView submission. Maybe you just need to modify the focus style a little bit like me to see the effect.

As mentioned above, calling the autoFocus method will cause crash in some cases on some mobile phones. Therefore, for the sake of safety, the calls of the autoFocus method in the manual focus code I will introduce are protected, and there is one point worth mentioning. Lin18 / CameraView: lin18/ CameraView

The code above crashed when calling setParameters on some mobile phones. I guess the reason is that FOCUS_MODE_CONTINUOUS_PICTURE focusing mode may not be supported on this phone. In the code before LIN18, FocusMode was set to determine whether the Camera supported or not, but it was not determined this time. Maybe this is the reason why there was a crash in setParameters.

The resetFocus method has been improved to include support for a judgment logic and try-catch protection

OK, the above is the summary of my requirement development of Android custom camera module. It is over and I hope it can have some effect

At last, it can be seen from the previous content that there are a lot of problems in the informal component CameraView officially launched. There are a lot of mobile phone compatibility problems and abnormal crash problems in issues. Use it At your own risk. This library is not suitable for all custom camera scene development, but it is a good library if it meets your basic needs. Finally, if you decide to use CameraView, I recommend using my improved versionCameraView 😎

Additional information

Android TextureView tutorial 2: The difference between TextureView and SurfaceView Android camera development pits: Android camera development pits 3. The official documentation for the use of the Camera API: Camera API 4. About the use of Camera API: Android Camera development details 5. About runtime Permissions: Everything developers need to know about Android M’s new runtime permissions