Jetpack CameraX library

The introduction of PreviewView

PreviewView is a custom View that displays images from the camera. It was built to make it easier for developers to set up and manipulate the Preview surface used by the camera.

If you need to provide the basic function of displaying camera images in your application, using PreviewView is the most recommended approach, which has several advantages:

  • Easy to use: PreviewView is a View that manages the Surface used by the Preview case to display all the captured images in the interface layout.
  • Code light: PreviewView is only focused on implementing the camera preview function. All of its internal resources are devoted to displaying the camera’s preview images and managing the Preview surface while the camera is in use. This separation of concerns keeps PreviewView code clean;
  • Full support: PreviewView handles the most difficult parts of displaying a camera’s image on screen, including aspect ratio, zooming, and rotation. Different devices will lead to inconsistent behavior, including equipment, screen size, camera hardware support levels, will also need to lock adapter such as split screen mode, directions and dynamically adjust the size of the display window display mode, in order to solve these problems and provide the seamless experience on a variety of equipment, PreviewView also did some compatibility processing.

PreviewView implementation mode

PreviewView is a subclass of FrameLayout that uses SurfaceView or TextureView to display images captured from the camera. Once the camera is ready, a Preview Surface instance is created and held as long as the camera is in use. If the camera is still working and the Preview Surface instance is released early, a new preview surface instance is created.

When it comes to key metrics like power consumption and response time, SurfaceView generally performs better than TextureView, which is why PreviewView makes SurfaceView its default implementation mode. However, some devices (mostly older devices) will flash back when the Preview surface is released too early. Unfortunately, with the SurfaceView, there is no control over when the surface is released, as this is controlled by the View hierarchy. So on these devices, PreviewView can only use TextureView as the implementation mode. You should also force The PreviewView to use TextureView when you need to rotate the camera preview screen, change transparency, or animate it.

You can call PreviewView. SetPreferredImplementationMode (ImplementationMode) and set for SURFACE_VIEW or ImplementationMode parameters TEXTURE_VIEW to change the implementation mode of the PreviewView. When the preferred mode is set to SURFACE_VIEW, the PreviewView will follow your Settings as much as possible (using the SurfaceView); When the preferred mode is set to TEXTURE_VIEW, PreviewView ensures that TEXTURE_VIEW is always used.

⚠ ️ before start using PreviewView, please by calling the Preview. SetSurfaceProvider (PreviewView. CreateSurfaceProvider ()) to set you want to the implementation of the model.

Here is how to set the implementation mode of PreviewView:

// Before the camera preview, Set to the implementation of the model previewView. PreferredImplementationMode = ImplementationMode. SURFACE_VIEW / / set the previewView to preview Use cases to begin the camera picture preview. Preview setSurfaceProvider (previewView. CreateSurfaceProvider (cameraInfo))Copy the code

PreviewView – Preview

PreviewView starts a data stream of Preview images by processing the SurfaceProvider needed to create the Preview use case. The SurfaceProvider prepares the Surface that needs to be provided to the camera, displays the data stream from the preview screen, and recreates the Surface if necessary. PreviewView. CreateSurfaceProvider (CameraInfo) receives a nullable CameraInfo instance. The PreviewView determines how to implement the functionality internally, taking into account the CameraInfo parameter passed in, the implementation mode you set, and the current camera functionality. If the CameraInfo you pass in is null, the PreviewView will use TextureView as its implementation mode because it cannot determine whether the selected camera will work with SurfaceView.

Once you have created the Preview use case and some other required instances, bind them to LifecycleOwner, create the SurfaceProvider using the CameraInfo of the bound camera, bind it to the Preview use case, Call Preview. SetSurfaceProvider (SurfaceProvider) to start the Preview picture data flow.

The following example shows how to bind PreviewView to Preview to start the Preview screen data stream:

Lifecycle val Camera = = lifecycle val Camera = = lifecycle val Camera = = = lifecycle val Camera = = = = lifecycle val Camera = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = cameraProvider.bindToLifecycle(lifecycleOwner, cameraSelector, preview, imageAnalysis, ImageCapture) // Create the surfaceProvider using the cameraInfo of the bound camera PreviewView. CreateSurfaceProvider (camera. CameraInfo) / / will surfaceProvider bound to preview case to start the preview preview.setSurfaceProvider(surfaceProvider)Copy the code

Previewview-scale type

PreviewView provides an API that lets you control how and where the preview looks in the parent view:

  • How determines whether to FIT or FILL the preview in the parent view;
  • Where determines whether the preview screen is aligned in the upper left (START), CENTER (CENTER), or bottom right (END) relative to the parent view.

The combination of “how” and “where” represents the scale types supported by PreviewView, The options are FIT_START, FIT_CENTER, FIT_END, FILL_START, FILL_CENTER, and FILL_END. The most common ones are FIT_CENTER and FILL_CENTER. The former scales the preview screen with a guaranteed aspect ratio and then centers it, while the latter does not scale and keeps it centered but may cause the screen to be cropped.

There are two ways to set the scale type:

  • This is done by setting the scaleType property of the PreviewView in the XML layout file, as shown in the following example:
<androidx.camera.view.PreviewView
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    app:scaleType="fitEnd" />
Copy the code
  • This is done in code by calling previewView.setScaleType (ScaleType), as shown in the following example:
previewView.setScaleType(ScaleType.FIT_CENTER)
Copy the code

To get the scale type used by the current PreviewView, call previewView.getScaleType ().

PreviewView – Camera control operation

PreviewView may scale, rotate, and transform the preview frames received from the camera to properly display them in the UI, depending on the orientation of the camera’s camera sensor, the rotation direction of the device, and the display mode and preview scale. This is why it is important to convert UI coordinates to camera sensor coordinates. In CameraX, this transformation is done by MeteringPointFactory, which can be created using the API provided by PreviewView: PreviewView. CreateMeteringPointFactory (cameraSelector), including cameraSelector parameters representing the incoming stream data camera.

When you need tap-to-focus, PreviewView’s MeteringPointFactor makes it easy. Although autofocus is enabled by default in the camera preview (camera support is required), you can still control the focus target when you click on the PreviewView. The MeteringPointFactory converts the coordinates of the focus target to the coordinates of the camera sensor, and then uses the camera to focus the area.

The following example shows how to use a touch Listener to focus on a PreviewView:

fun onTouch(x: Float, y: Float) {// Create MeteringPoint, Named factory val factory = previewView. CreateMeteringPointFactory (cameraSelector) / / converts the coordinates of the UI interface of the camera sensor coordinate val point = factory.createPoint(x, Y) / / to create focus action val action = FocusMeteringAction. Builder (point). The build () / / execution created by the focusing action cameraControl.startFocusAndMetering(action) }Copy the code

Another feature commonly used in camera preview screens is pinch-to-zoom, which allows you to zoom in and out by pinching in the preview screen. To implement it on a PreviewView, add a touch listener on top of it and bind it to a Scale Gesture Listener. This will allow you to intercept pinching gestures and update the camera’s zoom scale accordingly.

The following example shows how pinch-to-zoom can be implemented on a PreviewView:

Val Listener = object: ScaleGestureDetector creates a callback called listener that is called when a gesture event occurs.SimpleOnScaleGestureListener() {override Fun onScale(Detector: ScaleGestureDetector): Boolean {// Get the current camera scale val currentZoomRatio: Float = cameraInfo.zoomRatio.value ? : 1 f / / get the user to knead pulled gestures to change the scaling val delta = detector, scaleFactor / / update the scaling of cameras cameraControl. SetZoomRatio (currentZoomRatio * delta)return true}} val scaleGestureDetector = scaleGestureDetector (context, The listener) / / pass the PreviewView touch events to zoom gestures on the listener PreviewView. SetOnTouchListener {_, event -> scaleGestureDetector.onTouchEvent(event)return@setOnTouchListener true
}
Copy the code

PreviewView – How to test

PreviewView provides consistent camera processing behavior across a wide variety of Android devices, thanks to the investment in PreviewView and its other apis in CameraX’s automated test lab. These tests fall into two main categories:

  • Unit tests can verify PreviewView behavior with the current implementation mode, scale type, and MeteringPointFactor. Unit testing also ensures that the PreviewView adjusts the preview screen correctly when appropriate, when the parent view changes size, when the layout of the display changes, or when it is bound to the Window.
  • Integration testing ensures that the PreviewView is integrated into the application and can properly display or stop displaying the picture data stream from the camera. These tests verify preview’s status in various situations, including closing and re-opening the application multiple times while it is running, switching the front and rear cameras, and re-creating the application after its life cycle is destroyed. The main area covered by these tests is using TextureView as the implementation mode for PreviewView, because it can be very difficult to capture the start and end of the camera preview using SurfaceView.

conclusion

To sum up:

  • PreviewView is a custom View that can easily display the camera’s preview images.
  • PreviewView uses SurfaceView as its preview surface implementation by default, but will switch to TextureView when needed;
  • Bind use cases such as ImageCapture and ImageAnalysis to LifecycleOwner, create a surfaceProvider, and bind it to the Preview use case to launch the camera Preview;
  • Control the display mode of preview screen by defining the zoom type of PreviewView;
  • Focus by creating a MeteringPointFactory for the PreviewView;
  • Pinch/zoom is implemented by setting a gesture listener to the PreviewView.

Want to learn more about the great features of CameraX? Please refer to the following information and recommended reading:

  • An overview of the Android developer documentation | CameraX

Developer. The android. Google. Cn/training/ca… Codelab | CameraX

  • Codelab | CameraX use guide

Codelabs.developers.google.com/codelabs/ca…

  • Community | CameraX online developer community

Groups.google.com/a/android.c…

  • The sample code | use CameraX in building a camera application

Github.com/android/cam…

If you have questions about PreviewView or Preview, feel free to leave them in the comments section below. Thank you for reading!

Click here to learn more about CameraX