A Practical guide to Developing Camera for Android

About the author

Guo Xiaoxing is a programmer and guitarist. He is mainly engaged in the infrastructure work of The Android platform. Welcome to exchange technical questions.

The article directories

• A Camera practice guide
  • 1.1 Turning on the Camera
  • 1.2 Closing the Camera
  • 1.3 Enabling the Preview function
  • 1.4 Closing the Preview function
  • 1.5 take photos
  • 1.6 Starting video Recording
  • 1.7 Ending a Video Recording
• Camera2 practical guide
  • 2.1 Turning on the Camera
  • 2.2 Closing the Camera
  • 2.3 Enabling the Preview function
  • 2.4 Closing preview
  • 2.5 take photos
  • 2.6 Starting video Recording
  • 2.7 Ending a Video Recording

The apis related to Android Camera are also the most seriously fragmented part of the Android ecosystem. First of all, Android itself has two sets of apis, Camera below Android 5.0 and Camera2 above Android 5.0. Moreover, when more serious, Camera2 support varies from phone manufacturer to phone manufacturer, which led us to spend a lot of time in developing the camera dealing with compatibility issues.

What is the general process of camera development? 🤔

1. Detect and access camera resources Check whether the phone has camera resources. If so, request to access camera resources.
2. Create a preview interface and create a preview class that inherits from SurfaceView and implements the SurfaceHolder interface. With the Shoot Preview class, you can create a layout file that combines the preview with the designed user interface controls to display the camera preview in real time.
3. Set the photo listener and bind it to the user interface control so that it responds to user action and starts the photo taking process.
4. Take a picture and save the file, convert the image to bitmap file, and save the final output into a variety of commonly used formats of pictures.
5. Release camera resources. The camera is a shared resource. When the camera is used up, it must be released correctly to avoid conflicts when other programs access it.

What issues should be paid attention to in camera development? 🤔

1. Version compatibility problem, Camera below Android 5.0 and above Android 5.0 use Camera2, SurfaceView below Android 4.0 and above Android 4.0 TextureView, Android 6.0 above to do camera and other runtime permissions compatible.
2. Device compatibility issues, various features in Camera/Camera2 are implemented in different ways and support levels on some handset manufacturers’ devices, this requires compatibility testing, a little bit of a pitfall.
3. The most common life cycle changes in various scenarios are background scenes and lock screen scenes. In these two scenarios, the application and release of camera resources, and the creation and destruction of Surface will bring some problems

We’ll look at that in more detail.

About Camera/Camear2

If you want to solve this compatibility problem, you need to use both sets of cameras. Should you choose Camera based on the version of Android 5.0 or above? 🤔

In fact, this is not desirable. As I mentioned earlier, Camera2 support varies from phone manufacturer to phone manufacturer. Even with Android 5.0 and above, Camera2 support can be very poor. At this point, you need to downgrade the Camera.

About SurfaceView/TextureView

• SurfaceView is a View that has its own Surface. Interface rendering can be placed in a separate thread instead of the main thread. It is more like a Window and cannot transform or animate itself.
• TextureView also has its own Surface. But it can only be drawn in Windows with hardware acceleration layers. It is more like a normal View, and can be morphed and animated.

For more information on the differences between SurfaceView and TextureView, see SurfaceTexture, TextureView, SurfaceView and GLSurfaceView in Android 5.0(Lollipop).

So how do you choose a solution for a version? 🤔

The official open source library CameraView gives the solution:

If you want to use both, you need to define a unified interface, provide different implementations for different scenarios, and create different instances for different scenarios.

As we can see, this interface generally needs to define the following functions:

• Open the camera
• Close the camera
• Open the preview
• Close the preview
• Taking pictures
• Starting video recording
• End Video Recording

Defined interfaces, we have the idea, based on the specific characteristics of the camera to realize the corresponding solution, then another question came out, the camera in the day-to-day development generally exist as a SDK for various business calls, so how to design a functional and UI phase separation, highly customizable camera SDK? 🤔

The answer is to use Fragment to encapsulate the corresponding functions of various click events (click to take a photo, click to switch the camera, click to switch the flash mode, etc.) in Fragment. Business parties can mask a layer of UI on the Fragment when using (of course, we also need to provide the default implementation). This separates functionality from the UI and makes it easy to integrate.

The camera SDK frame is as follows:

• CameraActivity: Camera interface, mainly used to achieve UI customization, the actual function (click events) by CameraFragment to complete.
• CameraFragment: Provides a functional interface to the CameraActivity to complete the click events in the CameraActivity, such as: taking photos, recording, etc.
• CameraLifecycle: Handles camera changes as the Activity life cycle, holds CameraManager internally, handles camera initialization and release, preview creation and destruction, etc.
• CameraManager: The actual manager of the camera, who calls the camera API to operate the camera, take pictures, record videos, and so on.
• Camera/Camera2: Camera API.

This scheme has been implemented in phoenix project, and the renderings are shown as follows:

With the overall architecture understood, let’s analyze how Camera/Camera2 should be implemented respectively for this architecture.

A Camera practice guide

The following key classes are involved in the Camera API:

• Camera: Operates and manages Camera resources, supports switching between Camera resources, sets preview and shooting size, and sets aperture and exposure parameters.
• SurfaceView: Used to draw camera preview images, providing real-time preview of the images.
• SurfaceHolder: An abstract interface for controlling the Surface. It can control the size, format and pixels of the Surface, and can monitor changes in the Surface.
• Surfaceholder. Callback: The interface used to listen for Surface status changes.

What’s the difference between a SurfaceView and a normal View? 🤔

Normal views share a Surface, and all drawing takes place in the UI thread. Because the UI thread has other logic to handle, there is no guarantee that the View can be updated quickly or drawn at a frame rate. This is obviously not suitable for camera real-time

To preview this situation, the SurfaceView holds a separate Surface, which manages the format, size, and display position of the Surface. It also draws the Surface in a separate thread, resulting in higher drawing efficiency and frame rate.

The surfaceHolder.callback interface defines three functions:

• surfaceCreated(SurfaceHolder holder); Call camera.open() and camera.setpreviewDisplay () in this method when the Surface is first created to open the camera and connect the camera to the Surface

Such operations.

• surfaceChanged(SurfaceHolder holder, int format, int width, int height); Call camera.startpreview () in this method when the Surface size, format, and so on have changed.
• surfaceDestroyed(SurfaceHolder holder); Called when the Surface is destroyed, you can call camera.stoppreview (), camera.release() and other methods in this method to end the preview and release

1.1 Turning on the Camera

Before we open the camera, we need to get the relevant information about the system camera.

// How many cameras are there
numberOfCameras = Camera.getNumberOfCameras();

for (int i = 0; i < numberOfCameras; ++i) {
    final Camera.CameraInfo cameraInfo = new Camera.CameraInfo();

    Camera.getCameraInfo(i, cameraInfo);
    // Rear camera
    if (cameraInfo.facing == Camera.CameraInfo.CAMERA_FACING_BACK) {
        faceBackCameraId = i;
        faceBackCameraOrientation = cameraInfo.orientation;
    } 
    // Front camera
    else if(cameraInfo.facing == Camera.CameraInfo.CAMERA_FACING_FRONT) { faceFrontCameraId = i; faceFrontCameraOrientation = cameraInfo.orientation; }}Copy the code

Once you know the information about the camera, you can open the camera using the camera ID.

camera = Camera.open(cameraId);
Copy the code

In addition, when you open the Camera, you get a Camera object from which you can get and set various parameters for the Camera.

// Get the camera parameters
camera.getParameters();
// Set camera parameters
camera.getParameters();
Copy the code

Common parameters are as follows:

Flash configuration Parameters can be obtained through the parameters.getFlashMode () interface.

• Camera.parameters.FLASH_MODE_AUTO Auto mode: automatically turns on the flash when the light is low.
• Camera.parameters. FLASH_MODE_OFF Turns off the flash.
• Camera.parameters.FLASH_MODE_ON Flash when taking a photo;
• Camera.parameters.FLASH_MODE_RED_EYE Specifies the flash parameter, which is red eye proof.

The Parameters of the focus mode can be obtained through the Parameters.getFocusMode() interface.

• Camera.parameters.FOCUS_MODE_AUTO auto focus mode, photography small white special mode;
• Camera.parameters.FOCUS_MODE_FIXED fixed focus mode, old driver mode;
• Camera.parameters.FOCUS_MODE_EDOF depth of field mode, the most popular mode for literary women;
• Camera.parameters.FOCUS_MODE_INFINITY perspective mode, the mode of taking large scenery scenes;
• Camera.parameters.FOCUS_MODE_MACRO micro focus mode, shooting flowers, grass and ants special mode;

This parameter can be obtained through the Parameters.getscenemode () interface.

• SCENE_MODE_BARCODE Scanning bar code scene. NextQRCode project will identify and set this scene.
• SCENE_MODE_ACTION: Camera.Parameters.
• Camera.parameters. SCENE_MODE_AUTO Automatically selects a scene;
• Camera.parameters. SCENE_MODE_HDR A scene with high dynamic contrast, usually used to take pictures with clear light and shade, such as sunset clouds.
• Camera.parameters.SCENE_MODE_NIGHT Night scene;

1.2 Closing the Camera

Turning off the camera is as simple as releasing the camera.

camera.release();
Copy the code

1.3 Enabling the Preview function

A Camera preview is performed via the SurfaceHolder of the SurfaceView.

private void startPreview(SurfaceHolder surfaceHolder) {
    try {
        final Camera.CameraInfo cameraInfo = new Camera.CameraInfo();
        Camera.getCameraInfo(currentCameraId, cameraInfo);
        int cameraRotationOffset = cameraInfo.orientation;

        // Get the camera parameters
        final Camera.Parameters parameters = camera.getParameters();
        // Set the focus mode
        setAutoFocus(camera, parameters);
        // Set the flash mode
        setFlashMode(mCameraConfigProvider.getFlashMode());

        if (mCameraConfigProvider.getMediaAction() == CameraConfig.MEDIA_ACTION_PHOTO
                || mCameraConfigProvider.getMediaAction() == CameraConfig.MEDIA_ACTION_UNSPECIFIED)
            turnPhotoCameraFeaturesOn(camera, parameters);
        else if (mCameraConfigProvider.getMediaAction() == CameraConfig.MEDIA_ACTION_PHOTO)
            turnVideoCameraFeaturesOn(camera, parameters);

        final int rotation = ((WindowManager) mContext.getSystemService(Context.WINDOW_SERVICE)).getDefaultDisplay().getRotation();
        int degrees = 0;
        switch (rotation) {
            case Surface.ROTATION_0:
                degrees = 0;
                break; // Natural orientation
            case Surface.ROTATION_90:
                degrees = 90;
                break; // Landscape left
            case Surface.ROTATION_180:
                degrees = 180;
                break;// Upside down
            case Surface.ROTATION_270:
                degrees = 270;
                break;// Landscape right
        }

        // Set the preview direction according to the front and rear cameras, otherwise the preview image will be inverted.
        if (cameraInfo.facing == Camera.CameraInfo.CAMERA_FACING_FRONT) {
            displayRotation = (cameraRotationOffset + degrees) % 360;
            displayRotation = (360 - displayRotation) % 360; // compensate
        } else {
            displayRotation = (cameraRotationOffset - degrees + 360) % 360;
        }
        this.camera.setDisplayOrientation(displayRotation);

        if (Build.VERSION.SDK_INT > 13
                && (mCameraConfigProvider.getMediaAction() == CameraConfig.MEDIA_ACTION_VIDEO
                || mCameraConfigProvider.getMediaAction() == CameraConfig.MEDIA_ACTION_UNSPECIFIED)) {
// parameters.setRecordingHint(true);
        }

        if (Build.VERSION.SDK_INT > 14
                && parameters.isVideoStabilizationSupported()
                && (mCameraConfigProvider.getMediaAction() == CameraConfig.MEDIA_ACTION_VIDEO
                || mCameraConfigProvider.getMediaAction() == CameraConfig.MEDIA_ACTION_UNSPECIFIED)) {
            parameters.setVideoStabilization(true);
        }

        // Set the preview size
        parameters.setPreviewSize(previewSize.getWidth(), previewSize.getHeight());
        parameters.setPictureSize(photoSize.getWidth(), photoSize.getHeight());

        // Set camera parameters
        camera.setParameters(parameters);
        / / set the surfaceHolder
        camera.setPreviewDisplay(surfaceHolder);
        // Open preview
        camera.startPreview();

    } catch (IOException error) {
        Log.d(TAG, "Error setting camera preview: " + error.getMessage());
    } catch (Exception ignore) {
        Log.d(TAG, "Error starting camera preview: "+ ignore.getMessage()); }}Copy the code

1.4 Closing the Preview function

Closing the preview is easy, just call camera.stoppreview ().

camera.stopPreview();
Copy the code

1.5 take photos

This is done by calling the Camera’s takePicture() method,

takePicture(ShutterCallback shutter, PictureCallback raw, PictureCallback postview, PictureCallback jpeg)
Copy the code

This method takes three arguments:

• ShutterCallback Shutter: Is called back the moment the photo is taken. This is usually where you can play a photo sound like “click”.
• PictureCallback RAW: Returns uncompressed image data.
• PictureCallback PostView: Returns the image data of type PostView
• PictureCallback JPEG: Returns jpeg compressed image data.

We usually use the last one, just implement the last PictureCallback.

camera.takePicture(null.null.new Camera.PictureCallback() {
        @Override
        public void onPictureTaken(byte[] bytes, Camera camera) {
            // Store the returned image data
            final File pictureFile = outputPath;
            if (pictureFile == null) {
                Log.d(TAG, "Error creating media file, check storage permissions.");
                return;
            }
            try {
                FileOutputStream fileOutputStream = new FileOutputStream(pictureFile);
                fileOutputStream.write(bytes);
                fileOutputStream.close();
            } catch (FileNotFoundException error) {
                Log.e(TAG, "File not found: " + error.getMessage());
            } catch (IOException error) {
                Log.e(TAG, "Error accessing file: " + error.getMessage());
            } catch (Throwable error) {
                Log.e(TAG, "Error saving file: "+ error.getMessage()); }}});Copy the code

If you still want to take the photo after the photo is taken, call camera.startPreview() to continue to turn on the preview, otherwise close the preview and release the camera resources.

1.6 Starting video Recording

Video recording is done through MediaRecorder.

if (prepareVideoRecorder()) {
            mediaRecorder.start();
            isVideoRecording = true;
            uiHandler.post(new Runnable() {
                @Override
                public void run(a) { videoListener.onVideoRecordStarted(videoSize); }}); }Copy the code

MediaRecorder is mainly used to record audio and video, before the use of initialization and related parameters of the setting, as shown below:

protected boolean preparemediaRecorder(a) {
    mediaRecorder = new MediaRecorder();
    try {
        mediaRecorder.setAudioSource(MediaRecorder.AudioSource.CAMCORDER);
        mediaRecorder.setVideoSource(MediaRecorder.VideoSource.SURFACE);
        
        // Output format
        mediaRecorder.setOutputFormat(camcorderProfile.fileFormat);
        // Video frame rate
        mediaRecorder.setVideoFrameRate(camcorderProfile.videoFrameRate);
        // Video size
        mediaRecorder.setVideoSize(videoSize.getWidth(), videoSize.getHeight());
        // Video bit rate
        mediaRecorder.setVideoEncodingBitRate(camcorderProfile.videoBitRate);
        // Video encoder
        mediaRecorder.setVideoEncoder(camcorderProfile.videoCodec);
        
        // Audio coding rate
        mediaRecorder.setAudioEncodingBitRate(camcorderProfile.audioBitRate);
        // Audio channel
        mediaRecorder.setAudioChannels(camcorderProfile.audioChannels);
        // Audio sampling rate
        mediaRecorder.setAudioSamplingRate(camcorderProfile.audioSampleRate);
        // Audio encoder
        mediaRecorder.setAudioEncoder(camcorderProfile.audioCodec);
        
        File outputFile = outputPath;
        String outputFilePath = outputFile.toString();
        // Output path
        mediaRecorder.setOutputFile(outputFilePath);
        
        // Set the maximum size of the video output
        if (mCameraConfigProvider.getVideoFileSize() > 0) {
            mediaRecorder.setMaxFileSize(mCameraConfigProvider.getVideoFileSize());
            mediaRecorder.setOnInfoListener(this);
        }
        
        // Set the maximum length of video output
        if (mCameraConfigProvider.getVideoDuration() > 0) {
            mediaRecorder.setMaxDuration(mCameraConfigProvider.getVideoDuration());
            mediaRecorder.setOnInfoListener(this);
        }
        mediaRecorder.setOrientationHint(getVideoOrientation(mCameraConfigProvider.getSensorPosition()));
        
        / / to
        mediaRecorder.prepare();

        return true;
    } catch (IllegalStateException error) {
        Log.e(TAG, "IllegalStateException preparing MediaRecorder: " + error.getMessage());
    } catch (IOException error) {
        Log.e(TAG, "IOException preparing MediaRecorder: " + error.getMessage());
    } catch (Throwable error) {
        Log.e(TAG, "Error during preparing MediaRecorder: " + error.getMessage());
    }
    releasemediaRecorder();
    return false;
}
Copy the code

Be worth what carry is, often in daily business to shoot video length or size requirements, this can be achieved by the mediaRecorder. SetOnInfoListener () to handle, OnInfoListener will listen is to record video, and then we can processing in its callback methods.

   @Override
public void onInfo(MediaRecorder mediaRecorder, int what, int extra) {
    if (MediaRecorder.MEDIA_RECORDER_INFO_MAX_DURATION_REACHED == what) {
        // Reach the maximum duration
    } else if (MediaRecorder.MEDIA_RECORDER_INFO_MAX_FILESIZE_REACHED == what) {
        // Reach the maximum size}}Copy the code

For more information about MediaRecorder, please refer to the official MediaRecorder documentation.

1.7 Ending a Video Recording

Ending a video recording is as simple as calling the mediaRecorder.stop() method.

mediaRecorder.stop();
Copy the code

Also, be careful to release camera resources if you are no longer using it.

This is the whole content of Camera, or relatively simple, let’s move on to the related content of Camera2, pay attention to the difference between the two.

Camera2 practical guide

• Android Camera2 official video
• Android Camera2 official documentation
• Android Camera2 official use case

The following key classes are mainly involved in the Camera2 API:

• CameraManager: CameraManager for turning the system camera on and off
• CameraCharacteristics: Describe the various features of the camera. We can get them from the CameraManager’s getCameraCharacteristics(@nonnull String cameraId) method.
• CameraDevice: Describes the system Camera, similar to the early Camera.
• CameraCaptureSession: Session class. When you need to take photos, preview, etc., you need to create an instance of this class and then control it using the methods in the instance (for example, capture()).
• CaptureRequest: describes a request for an operation. Any operation that takes a photo, previews, etc., must first pass the CaptureRequest parameter, which is also controlled by the CameraRequest member variable.
• CaptureResult: Describes the result after the photo is taken.

Camera2 photography process is as follows:

Developers create CaptureRequest to make Capture requests to the camera. These requests are queued for the camera to process, and the camera returns the results wrapped in CaptureMetadata to the developer. The entire process is set up in a session of CameraCaptureSession.

2.1 Turning on the Camera

Before we open the camera, we first get the CameraManager, then the list of cameras, and then the parameters for each camera (mainly the front and rear cameras).

mCameraManager = (CameraManager) mContext.getSystemService(Context.CAMERA_SERVICE);
try {
    final String[] ids = mCameraManager.getCameraIdList();
    numberOfCameras = ids.length;
    for (String id : ids) {
        final CameraCharacteristics characteristics = mCameraManager.getCameraCharacteristics(id);

        final int orientation = characteristics.get(CameraCharacteristics.LENS_FACING);
        if (orientation == CameraCharacteristics.LENS_FACING_FRONT) {
            faceFrontCameraId = id;
            faceFrontCameraOrientation = characteristics.get(CameraCharacteristics.SENSOR_ORIENTATION);
            frontCameraCharacteristics = characteristics;
        } else{ faceBackCameraId = id; faceBackCameraOrientation = characteristics.get(CameraCharacteristics.SENSOR_ORIENTATION); backCameraCharacteristics = characteristics; }}}catch (Exception e) {
    Log.e(TAG, "Error during camera initialize");
}
Copy the code

Camera2 like Camera also have cameraId concept, through our mCameraManager. GetCameraIdList () to obtain cameraId list, Then through mCameraManager. GetCameraCharacteristics (id) for each id corresponding to the parameters of the camera.

CameraCharacteristics parameters include:

• LENS_FACING: Front camera (LENS_FACING_FRONT) or rear camera (LENS_FACING_BACK).
• SENSOR_ORIENTATION: Camera orientation.
• FLASH_INFO_AVAILABLE: Indicates whether the flash is supported.
• CameraCharacteristics. INFO_SUPPORTED_HARDWARE_LEVEL: get the current camera characteristics of equipment support.

Note: In fact, not all features of Camera2 are available on Android devices from various vendors, Through characteristics. The get (CameraCharacteristics. INFO_SUPPORTED_HARDWARE_LEVEL) method to according to the return value to obtain the support level, specifically:

• INFO_SUPPORTED_HARDWARE_LEVEL_FULL: Full hardware support, allowing manual control of full HD cameras, support for continuous shooting mode, and other new features.
• INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED: Support for this file is limited. This file needs to be queried separately.
• INFO_SUPPORTED_HARDWARE_LEVEL_LEGACY: Supported by all devices, that is, the same features supported by the outdated Camera API.

Using the INFO_SUPPORTED_HARDWARE_LEVEL parameter, we can determine whether to use Camera or Camera2. The specific method is as follows:

@TargetApi(Build.VERSION_CODES.LOLLIPOP)
public static boolean hasCamera2(Context mContext) {
    if (mContext == null) return false;
    if (Build.VERSION.SDK_INT < Build.VERSION_CODES.LOLLIPOP) return false;
    try {
        CameraManager manager = (CameraManager) mContext.getSystemService(Context.CAMERA_SERVICE);
        String[] idList = manager.getCameraIdList();
        boolean notFull = true;
        if (idList.length == 0) {
            notFull = false;
        } else {
            for (final String str : idList) {
                if (str == null || str.trim().isEmpty()) {
                    notFull = false;
                    break;
                }
                final CameraCharacteristics characteristics = manager.getCameraCharacteristics(str);

                final int supportLevel = characteristics.get(CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL);
                if (supportLevel == CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LEGACY) {
                    notFull = false;
                    break; }}}return notFull;
    } catch (Throwable ignore) {
        return false; }}Copy the code

For more details on ameraCharacteristics parameters, see the CameraCharacteristics official document.

Open the camera is the chief called mCameraManager. OpenCamera (currentCameraId stateCallback, backgroundHandler) method, as you can see, there are three parameters:

• String cameraId: unique ID of the camera.
• CameraDevice. StateCallback callback: camera open callback.
• Handler Handler: The Handler that StateCallback needs to call. We can generally use the Handler of the current thread.

 mCameraManager.openCamera(currentCameraId, stateCallback, backgroundHandler);
Copy the code

We mentioned above CameraDevice StateCallback, it is a camera to open a callback, defines the open, close and error correction methods, we can do it in the callback method corresponding to the operation.

private CameraDevice.StateCallback stateCallback = new CameraDevice.StateCallback() {
    @Override
    public void onOpened(@NonNull CameraDevice cameraDevice) {
        / / get CameraDevice
        mcameraDevice = cameraDevice;
    }

    @Override
    public void onDisconnected(@NonNull CameraDevice cameraDevice) {
        / / close the CameraDevice
        cameraDevice.close();

    }

    @Override
    public void onError(@NonNull CameraDevice cameraDevice, int error) {
        / / close the CameraDevicecameraDevice.close(); }};Copy the code

2.2 Closing the Camera

With the above description, closing is simple.

/ / close the CameraDevice
cameraDevice.close();
Copy the code

2.3 Enabling the Preview function

Camera2 is called by creating a request session, specifically:

1. Call the mCameraDevice. CreateCaptureRequest (CameraDevice. TEMPLATE_PREVIEW) method to create CaptureRequest, call
2. Create CaptureSession mCameraDevice. CreateCaptureSession () method.

CaptureRequest.Builder createCaptureRequest(@RequestTemplate int templateType)
Copy the code

The templateType parameter in the createCaptureRequest() method represents the request type. There are six types of request:

• TEMPLATE_PREVIEW: Creates a preview request
• TEMPLATE_STILL_CAPTURE: Create a request suitable for still image capture, image quality over frame rate.
• TEMPLATE_RECORD: Request to create a video record
• TEMPLATE_VIDEO_SNAPSHOT: Snapshot request for creating a video recording
• TEMPLATE_ZERO_SHUTTER_LAG: Creates a request for zero shutter delay. Maximize image quality without affecting the preview frame rate.
• TEMPLATE_MANUAL: Create a basic capture request in which all auto controls are disabled (auto exposure, auto white balance, auto focus).

createCaptureSession(@NonNull List<Surface> outputs, @NonNull CameraCaptureSession.StateCallback callback, @Nullable Handler handler)
Copy the code

The createCaptureSession() method takes three parameters:

• List outputs: We need to output the Surface.
• CameraCaptureSession. StateCallback callback: session state related callback.
• This Handler is used to identify which callback should be called. It is usually written to the Handler of the current thread.

About CameraCaptureSession. StateCallback callback method:

• onConfigured(@NonNull CameraCaptureSession session); The camera is configured and ready to process Capture requests.
• onConfigureFailed(@NonNull CameraCaptureSession session); Camera configuration failed
• onReady(@NonNull CameraCaptureSession session); The camera is in the ready state and no requests need to be processed.
• onActive(@NonNull CameraCaptureSession session); The camera is processing the request.
• onClosed(@NonNull CameraCaptureSession session); Session closed
• onSurfacePrepared(@NonNull CameraCaptureSession session, @NonNull Surface surface); Surface is ready

With this in mind, creating a preview request is simple.

previewRequestBuilder = mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);
previewRequestBuilder.addTarget(workingSurface);

// Note that in addition to the Surface preview, we also added imagereader.getSurface (), which is used to retrieve data after the photo is taken
mCameraDevice.createCaptureSession(Arrays.asList(workingSurface, imageReader.getSurface()),
        new CameraCaptureSession.StateCallback() {
            @Override
            public void onConfigured(@NonNull CameraCaptureSession cameraCaptureSession) {
                cameraCaptureSession.setRepeatingRequest(previewRequest, captureCallback, backgroundHandler);
            }

            @Override
            public void onConfigureFailed(@NonNull CameraCaptureSession cameraCaptureSession) {
                Log.d(TAG, "Fail while starting preview: "); }},null);
Copy the code

Can be found in the onConfigured () calls the cameraCaptureSession. SetRepeatingRequest (previewRequest captureCallback, backgroundHandler), So we can continue to preview.

Note: As mentioned above, we added imagereader.getSurface (), which is responsible for obtaining data after taking photos. The specific operation is to set an OnImageAvailableListener for imageReader. And then get it in its onImageAvailable() method.

mImageReader.setOnImageAvailableListener(mOnImageAvailableListener, mBackgroundHandler);

private final ImageReader.OnImageAvailableListener mOnImageAvailableListener
            = new ImageReader.OnImageAvailableListener() {

        @Override
        public void onImageAvailable(ImageReader reader) {
            // Retrieve the image when it is available and save it
            mBackgroundHandler.post(newImageSaver(reader.acquireNextImage(), mFile)); }};Copy the code

2.4 Closing preview

Closing preview is to close the current preview session. Combined with the content of opening preview above, the specific implementation is as follows:

if(captureSession ! =null) {
    captureSession.close();
    try {
        captureSession.abortCaptures();
    } catch (Exception ignore) {
    } finally {
        captureSession = null; }}Copy the code

2.5 take photos

There are three steps to taking photos:

1. af

try {
    // Focus the camera
    previewRequestBuilder.set(CaptureRequest.CONTROL_AF_TRIGGER, CameraMetadata.CONTROL_AF_TRIGGER_START);
    // Modify the status
    previewState = STATE_WAITING_LOCK;
    // Send the focus request
    captureSession.capture(previewRequestBuilder.build(), captureCallback, backgroundHandler);
} catch (Exception ignore) {
}
Copy the code

We define a CameraCaptureSession. CaptureCallback handles requests to focus the results returned.

private CameraCaptureSession.CaptureCallback captureCallback = new CameraCaptureSession.CaptureCallback() {

    @Override
    public void onCaptureProgressed(@NonNull CameraCaptureSession session,
                                    @NonNull CaptureRequest request,
                                    @NonNull CaptureResult partialResult) {}@Override
    public void onCaptureCompleted(@NonNull CameraCaptureSession session,
                                   @NonNull CaptureRequest request,
                                   @NonNull TotalCaptureResult result) {
            // Wait to focus
            final Integer afState = result.get(CaptureResult.CONTROL_AF_STATE);
            if (afState == null) {
                // Failed to focus and shot directly
                captureStillPicture();
            } else if(CaptureResult.CONTROL_AF_STATE_FOCUSED_LOCKED == afState || CaptureResult.CONTROL_AF_STATE_NOT_FOCUSED_LOCKED == afState || CaptureResult.CONTROL_AF_STATE_INACTIVE == afState || CaptureResult.CONTROL_AF_STATE_PASSIVE_SCAN == afState)  { Integer aeState = result.get(CaptureResult.CONTROL_AE_STATE);if (aeState == null ||
                        aeState == CaptureResult.CONTROL_AE_STATE_CONVERGED) {
                    previewState = STATE_PICTURE_TAKEN;
                    // After focusing, take a photo
                    captureStillPicture();
                } else{ runPreCaptureSequence(); }}}};Copy the code

2. Taking pictures

We defined a captureStillPicture() to take a picture.

private void captureStillPicture(a) {
    try {
        if (null == mCameraDevice) {
            return;
        }
        
        // Build a CaptureRequest to take photos
        final CaptureRequest.Builder captureBuilder =
                mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_STILL_CAPTURE);
        captureBuilder.addTarget(imageReader.getSurface());

        // Use the same AR and AF modes as previews
        captureBuilder.set(CaptureRequest.CONTROL_AF_MODE, CaptureRequest.CONTROL_AF_MODE_CONTINUOUS_PICTURE);
        // Set the direction
        captureBuilder.set(CaptureRequest.JPEG_ORIENTATION, getPhotoOrientation(mCameraConfigProvider.getSensorPosition()));

        // Create a session
        CameraCaptureSession.CaptureCallback CaptureCallback = new CameraCaptureSession.CaptureCallback() {
            @Override
            public void onCaptureCompleted(@NonNull CameraCaptureSession session,
                                           @NonNull CaptureRequest request,
                                           @NonNull TotalCaptureResult result) {
                Log.d(TAG, "onCaptureCompleted: "); }};// Stop the continuous view
        captureSession.stopRepeating();
        // Capture the photo
        captureSession.capture(captureBuilder.build(), CaptureCallback, null);

    } catch (CameraAccessException e) {
        Log.e(TAG, "Error during capturing picture"); }}Copy the code

3. Cancel the focus

After taking the photo, we also need to unlock the camera focus and return the camera to preview.

try {
    // Reset autofocus
    previewRequestBuilder.set(CaptureRequest.CONTROL_AF_TRIGGER, CameraMetadata.CONTROL_AF_TRIGGER_CANCEL);
    captureSession.capture(previewRequestBuilder.build(), captureCallback, backgroundHandler);
    // Restore the camera to its normal preview state
    previewState = STATE_PREVIEW;
    // Open the continuous view mode
    captureSession.setRepeatingRequest(previewRequest, captureCallback, backgroundHandler);
} catch (Exception e) {
    Log.e(TAG, "Error during focus unlocking");
}
Copy the code

2.6 Starting video Recording

// Close the preview first, because you need to add a preview of the output Surface, which is mediaRecorder. GetSurface ()
closePreviewSession();

// Initialize MediaRecorder and set parameters
if (preparemediaRecorder()) {

    final SurfaceTexture texture = Camera2Manager.this.texture;
    texture.setDefaultBufferSize(videoSize.getWidth(), videoSize.getHeight());

    try {
        // Build aptureRequest for video recording
        previewRequestBuilder = mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_RECORD);
        final List<Surface> surfaces = new ArrayList<>();

        // Set the Surface preview
        final Surface previewSurface = workingSurface;
        surfaces.add(previewSurface);
        previewRequestBuilder.addTarget(previewSurface);

        // Set the preview to output the Surface
        workingSurface = mediaRecorder.getSurface();
        surfaces.add(workingSurface);
        previewRequestBuilder.addTarget(workingSurface);

        mCameraDevice.createCaptureSession(surfaces, new CameraCaptureSession.StateCallback() {
            @Override
            public void onConfigured(@NonNull CameraCaptureSession cameraCaptureSession) {
                captureSession = cameraCaptureSession;

                previewRequestBuilder.set(CaptureRequest.CONTROL_MODE, CameraMetadata.CONTROL_MODE_AUTO);
                try {
                    // Keep sending Capture requests for real-time preview.
                    captureSession.setRepeatingRequest(previewRequestBuilder.build(), null, backgroundHandler);
                } catch (Exception e) {
                }

                try {
                    // Start recording
                    mediaRecorder.start();
                } catch (Exception ignore) {
                    Log.e(TAG, "mediaRecorder.start(): ", ignore);
                }

                isVideoRecording = true;

                uiHandler.post(new Runnable() {
                    @Override
                    public void run(a) { cameraVideoListener.onVideoRecordStarted(videoSize); }}); }@Override
            public void onConfigureFailed(@NonNull CameraCaptureSession cameraCaptureSession) {
                Log.d(TAG, "onConfigureFailed");
            }
        }, backgroundHandler);
    } catch (Exception e) {
        Log.e(TAG, "startVideoRecord: ", e); }}Copy the code

About MediaRecorder above talk Camera we have already said, here will not repeat.

So that’s all the video recording is about, just simple API usage, pretty simple.

2.7 Ending a Video Recording

Ending a video recording is to close the session and release some resources. Specifically:

1. Closing the Preview session
2. Stop the mediaRecorder
3. To release the mediaRecorder

// Close the preview session
if(captureSession ! =null) {
    captureSession.close();
    try {
        captureSession.abortCaptures();
    } catch (Exception ignore) {
    } finally {
        captureSession = null; }}/ / stop the mediaRecorder
if(mediaRecorder ! =null) {
    try {
        mediaRecorder.stop();
    } catch (Exception ignore) {
    }
}

/ / to release the mediaRecorder
try {
    if(mediaRecorder ! =null) { mediaRecorder.reset(); mediaRecorder.release(); }}catch (Exception ignore) {

} finally {
    mediaRecorder = null;
}
Copy the code

That’s the Camera/Camera2 practice. More on image and video processing can be found in the Phoenix project.