Welcome to follow the public account: Sumsmile/focus on image processing mobile development veteran
Mobile terminal graphics, audio and video processing, image acquisition, need to have a certain understanding of Camera. I plan to use two or three articles to sort out the basic knowledge related to camera shooting and openglES rendering.
Contents:
1.1 Get Camera information :cameraId, Orientation 1.2 Open Camera hardware 1.3 Set preview properties, size and encoding format 1.4 Set preview surface That is, the container for receiving and displaying images 1.5 Start preview 2. Camera preview direction correction 2.1 Mobile phone natural direction and local coordinate system 2.2 Display Direction 2.3 Camera sensor Direction 2.4 Rear camera picture correction 2.5 Front camera picture correction 3. Switch camera 4Copy the codeImplementation effect
I. Basic operation of camera preview
1.1 Obtaining camera information :cameraId, Orientation (Camera hardware orientation)
int numberOfCameras = Camera.getNumberOfCameras();// Obtain the number of cameras
for (int cameraId = 0; cameraId < numberOfCameras; cameraId++) {
Camera.CameraInfo cameraInfo = new Camera.CameraInfo();
Camera.getCameraInfo(cameraId, cameraInfo);
if (cameraInfo.facing == Camera.CameraInfo.CAMERA_FACING_BACK) {
// Rear camera information
mBackCameraId = cameraId;
mBackCameraInfo = cameraInfo;
} else if (cameraInfo.facing == Camera.CameraInfo.CAMERA_FACING_FRONT){
// Front camera informationmFrontCameraId = cameraId; mFrontCameraInfo = cameraInfo; }}Copy the code1.2 Opening the Camera hardware
Configure camera and storage permissions
<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
Copy the codeSet SurfaceCallback, callback, and start preview. Note that surfaceChanged calls back at least once
SurfaceView cameraPreview = findViewById(R.id.camera_preview);
cameraPreview.getHolder().addCallback(new PreviewSurfaceCallback());
private class PreviewSurfaceCallback implements SurfaceHolder.Callback{
@Override
public void surfaceCreated(SurfaceHolder holder) {}@Override
public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
mPreviewSurface = holder;
mPreviewSurfaceWidth = width;
mPreviewSurfaceHeight = height;
if(mCameraHandler ! =null){ mCameraHandler.obtainMessage(MSG_SET_PREVIEW_SIZE, width, height).sendToTarget(); mCameraHandler.obtainMessage(MSG_SET_PICTURE_SIZE).sendToTarget(); mCameraHandler.obtainMessage(MSG_SET_PREVIEW_SURFACE, holder).sendToTarget(); mCameraHandler.sendEmptyMessage(MSG_START_PREVIEW); }}@Override
public void surfaceDestroyed(SurfaceHolder holder) {
mPreviewSurface = null;
mPreviewSurfaceWidth = 0;
mPreviewSurfaceHeight = 0; }}Copy the codeOpen the camera with the parameter camerAID
private void openCamera(int cameraId) {
Camera camera = mCamera;
if (ContextCompat.checkSelfPermission(this, Manifest.permission.CAMERA) == PackageManager.PERMISSION_GRANTED){
// Turn on the camera
mCamera = Camera.open(cameraId);
mCameraId = cameraId;
mCameraInfo = cameraId == mFrontCameraId ? mFrontCameraInfo : mBackCameraInfo;
// Set the camera orientation, as described in section 2.1mCamera.setDisplayOrientation(getCameraDisplayOrientation(mCameraInfo)); }}Copy the code1.3 Set preview properties, size and coding format
Query the supported preview size and encoding format and set it as required.
private void setPreviewSize(int shortSide, int longSide) {
if(mCamera ! =null&& shortSide ! =0&& longSide ! =0) {float aspectRatio = (float)longSide / shortSide;
Camera.Parameters parameters = mCamera.getParameters();
List<Camera.Size> supportedPreviewSizes = parameters.getSupportedPreviewSizes();
for (Camera.Size previewSize : supportedPreviewSizes) {
//1. Set preview size
if((float)previewSize.width / previewSize.height == aspectRatio && previewSize.height <= shortSide && previewSize.width <= longSide) {
parameters.setPreviewSize(previewSize.width, previewSize.height);
//2. Set the PREVIEW_FORMAT = imageformat.nv21
/ / NV21 YUV
if(isPreviewFormatSupported(parameters, PREVIEW_FORMAT)){
parameters.setPreviewFormat(PREVIEW_FORMAT);
int frameWidth = previewSize.width;
int frameHeight = previewSize.height;
int previewFormat = parameters.getPreviewFormat();
PixelFormat pixelFormat = new PixelFormat();
PixelFormat.getPixelFormatInfo(previewFormat, pixelFormat);
int bufferSize = (frameWidth * frameHeight * pixelFormat.bitsPerPixel) / 8;
//3. Set the buffer array for preview
mCamera.addCallbackBuffer(new byte[bufferSize]);
mCamera.addCallbackBuffer(new byte[bufferSize]);
mCamera.addCallbackBuffer(new byte[bufferSize]); } mCamera.setParameters(parameters); }}}}Copy the code1.4 Setting up preview surface, which is the container for receiving and displaying images
The actual setup is the surfaceHolder
private void setPreviewSurface(SurfaceHolder previewSurface) {
if(mCamera ! =null&& previewSurface ! =null) {
try {
mCamera.setPreviewDisplay(previewSurface);
} catch(IOException e) { e.printStackTrace(); }}}Copy the code1.5 Starting preview
private void startPreview(a) {
if(mCamera ! =null&& mPreviewSurface ! =null) {
// Add callback to facilitate buffer reuse
mCamera.setPreviewCallbackWithBuffer(new Camera.PreviewCallback() {
@Override
public void onPreviewFrame(byte[] data, Camera camera) {
// Recycle the buffercamera.addCallbackBuffer(data); }}); mCamera.startPreview(); }}Copy the codeSecond, camera preview direction correction
Due to the hardware design of the mobile phone camera, without additional processing, the image Angle of the camera preview is wrong. Exactly speaking, it is 90° counterclockwise in portrait mode.
Set the rotation direction of the camera preview.
mCamera.setDisplayOrientation(getCameraDisplayOrientation(mCameraInfo));
Copy the codeThe full code is from the Official Google /Android documentation
private int getCameraDisplayOrientation(Camera.CameraInfo cameraInfo) {
int roration = getWindowManager().getDefaultDisplay().getRotation();
// Screen display direction Angle (relative to local coordinate Y positive direction Angle)
int degrees = 0;
switch (roration) {
case Surface.ROTATION_0:
degrees = 0;
break;
case Surface.ROTATION_90:
degrees = 90;
break;
case Surface.ROTATION_180:
degrees = 180;
break;
case Surface.ROTATION_270:
degrees = 270;
break;
}
int result;
if (cameraInfo.facing == Camera.CameraInfo.CAMERA_FACING_FRONT){
result = (cameraInfo.orientation + degrees) % 360;
result = (360 - result) % 360;
} else {
result = (cameraInfo.orientation - degrees + 360) %360;
}
// The Angle the camera needs to be corrected
return result;
}
Copy the codeCamera preview direction correction is relatively complex, looked up a lot of information, mostly copied Google code, speak ambiguous. I believe many friends have seen the above code, but do not necessarily understand the final result calculation.
To clarify camera orientation correction, we will introduce a few important concepts
- Mobile phone Natural Direction
- Local coordinate system
- According to the direction
- Camera Sensor Direction
2.1 Mobile phone natural direction and local coordinate system
Phones default to portrait, with the short side facing up in the natural direction, and tablets default to landscape, with the wide side facing up in the natural direction.
The local coordinate system is related to the natural state of the mobile phone, and the Y-axis is aligned with the upward direction of the mobile phone in the natural state. In the following figure, the local coordinate system of the mobile phone is Y-axis upward:
For the sake of illustration, each direction is referred to as the positive direction of the local coordinate Y-axis
2.2 Display Direction
The display direction depends on the horizontal and vertical screen states. In vertical screen, the display direction is up and the display direction is consistent with the local coordinate Y axis. In landscape screen, the display direction is up and the local coordinate X axis is aligned.
Note that when the horizontal screen is rotated to the left, the display direction is upward, and the included Angle relative to the local coordinate Y axis is 90°, that is, the Y axis can be rotated 90° clockwise to align the display direction; when the horizontal screen is rotated to the right, the included Angle is 270°.
It is important to understand this concept, which will be used later in the calculation of camera Angle correction.
2.3 Camera Sensor Direction
Take the rear camera as an example
During development, in portrait mode, the coordinate system of window View is y axis with short side and X axis with long side
When the camera hardware is installed, it rotates 90° clockwise relative to the natural orientation of the phone. The short side is the X axis, and the long side is the Y axis. It looks like it was designed specifically for pad landscape. (according to? I’m also a hen.
When the phone is facing left landscape, the two coordinate systems are just aligned, so it is also right not to adapt the display in development.
In the above code, camerainfo. orientation captures the orientation of the camera (relative to the local Y-axis).
2.4 Rear camera picture correction
As mentioned above, due to the change of display direction caused by camera installation Angle and mobile phone horizontal and vertical screen status switch, the image captured by the camera may be displayed on the screen with deflection Angle. In actual development, we need to calculate this Angle for correction.
If nothing is done, the preview is skewed with degree 0 and orientation 90°.
Again, the Angle is based on the positive direction of the local coordinate y axis
How do you understand that? You can imagine that your head is a camera, and if you tilt your head 90 degrees to the right, you’ll see a skewed image. Then you feed the image to the monitor, which doesn’t know you’re tilting your head to collect the data.
Correction needs to call mCamera. SetDisplayOrientation (int arg), set up a point of view, the acquisition of image clockwise arg Angle, to compensate for the Angle of the camera.
Take left landscape as an example:
arg = orientation - degree // So if it's left landscape
arg = 90 - 90 = 0 // It happens that the display is correct
Copy the codeIt can be understood as: the data collected by the camera is 90 degrees ahead of the local coordinate system, and the left landscape screen causes the display direction to be 90 degrees ahead, so that the camera direction and the display direction are even.
To the right landscape:
// Camera relative local coordinates Y axis unchanged: orientation = 90
// Display direction up, relative to the local coordinate Y axis clockwise rotation degree = 270
arg = orientation - degree + 360 = 90 - 270 + 360 = 180
Copy the codeWe need to compensate 180 degrees, where +360 is to keep the rotation direction clockwise, so that arG is not negative, in fact, -180 and +180 are the same. And in the demo, it was upside down, so it had to compensate for 180 degrees.
2.5 Front camera picture correction
The front is a little bit more difficult, but the difference is
1. When taking a selfie, the rotation Angle you see is opposite to the real Angle obtained by the camera, that is, you see counterclockwise, but the real Angle is clockwise. The camera system processes front-facing shots by mirroring left and right to simulate the effect of a person looking in a mirror, so the pixels on the display are already switched left and right. These two things make the front camera correction a little more convoluted to understand, but the code looks similar. Here is a bit around, the author in the evening when taking a bath to think this problem absent-minded, in the toilet daze for more than an hour, the wife still thinks I take a bath have an accident.
So, just taking into account the opposite selfie Angle,
arg = orientation - (-degree) = orientation + degree
Copy the codeConsidering the mirror image, the final compensation Angle is:
result = 360- arg,Copy the codeThis is consistent with the template code provided in the official Google documentation. Friends, can see this all understand, for their own studious point praise.
If you look at the following figure, after the left and right mirror image, A is mirror IMAGE B, point A is turned to the positive Y-axis at an Angle of A, and point B is turned to the positive Y-axis at an Angle of B, A + B = 360, so after the mirror image, the Angle that really needs to be compensated is 360-arg
Addendum: finally take the module 360, also very well understood, to ensure that the Angle within a period.
result = (360 - result) % 360;
Copy the codeCode reference camera demo
Three, switch the camera
The main logic is the same as opening the preview for the first time, but the difference is to close the preview first. The flow is as follows:
- Stop preview (in practice, no preview stops and no errors are reported)
- Turn off the current Camera
- Reopen the camera and use another cameraId
- Set properties such as preview size
- Set the preview surfaceHolder
// Demo uses a button click to switch camera
Button switchCameraButton = findViewById(R.id.switch_camera);
switchCameraButton.setOnClickListener(new OnSwitchCameraButtonClickListener());
private class OnSwitchCameraButtonClickListener implements View.OnClickListener {
@Override
public void onClick(View v) {
if(mCameraHandler ! =null&& mPreviewSurface ! =null) {
int cameraId = switchCameraId();// Switch camera ID
mCameraHandler.sendEmptyMessage(MSG_STOP_PREVIEW);// Stop preview
mCameraHandler.sendEmptyMessage(MSG_CLOSE_CAMERA);// Turn off the current camera
mCameraHandler.obtainMessage(MSG_OPEN_CAMERA, cameraId, 0).sendToTarget();// Start a new camera
mCameraHandler.obtainMessage(MSG_SET_PREVIEW_SIZE, mPreviewSurfaceWidth, mPreviewSurfaceHeight).sendToTarget();// Configure the preview size
mCameraHandler.obtainMessage(MSG_SET_PICTURE_SIZE, mPreviewSurfaceWidth, mPreviewSurfaceHeight).sendToTarget();// Configure the photo size
mCameraHandler.obtainMessage(MSG_SET_PREVIEW_SURFACE, mPreviewSurface).sendToTarget();// Configure preview Surface
mCameraHandler.sendEmptyMessage(MSG_START_PREVIEW);// Enable preview}}}// Stop preview
private void stopPreview(a) {
Camera camera = mCamera;
if(camera ! =null) {
camera.stopPreview();
Log.d(TAG, "stopPreview() called"); }}// Close the camera
private void closeCamera(a) {
if(mCamera ! =null) {
mCamera.release();
mCamera = null; }}Copy the codeFourth, take pictures
It’s easier to take photos based on the logic of preview. Camera provides an API for taking pictures.
- Set takePicture size and other properties, (if not set, there may be a default size, I did not verify here)
- Camera.takePicture
Setting the takePicture size is similar to the setup logic of preview
/** * Sets the photo size according to the specified size requirements, considers the scale of the specified size, and goes to the maximum size that matches the scale as the photo size. * *@paramThe length of the shortSide *@paramThe length of the longSide */
private void setPictureSize(int shortSide, int longSide) {
Camera camera = mCamera;
if(camera ! =null&& shortSide ! =0&& longSide ! =0) {
float aspectRatio = (float) longSide / shortSide;
Camera.Parameters parameters = camera.getParameters();
List<Camera.Size> supportedPictureSizes = parameters.getSupportedPictureSizes();
for (Camera.Size pictureSize : supportedPictureSizes) {
if ((float) pictureSize.width / pictureSize.height == aspectRatio) {
parameters.setPictureSize(pictureSize.width, pictureSize.height);
camera.setParameters(parameters);
break; }}}}Copy the codeThe button to trigger takePicture
Button takePictureButton = findViewById(R.id.take_picture);
takePictureButton.setOnClickListener(new OnTakePictureButtonClickListener());
Copy the codeTaking pictures
private class OnTakePictureButtonClickListener implements View.OnClickListener {
@Override
public void onClick(View v) {
takePicture();
// Preview is stopped every time the photo is taken and needs to be restartPreviewrestartPreview(); }}/ / photo
private void takePicture(a) {
if(mCamera ! =null) {
Camera.Parameters parameters = mCamera.getParameters();
mCamera.setParameters(parameters);
// takePicture can be set to multiple callbacks, you can view the source code, not described here
mCamera.takePicture(new ShutterCallback(), new RawCallback(), new PostviewCallback(), newJpegCallback()); }}Copy the codeMatters needing attention
This method is only valid when preview is active (after{@link #startPreview()}). Preview will be stopped after the image is taken; callers must call {@link #startPreview()} again if they want to re-start preview or take more pictures. This should not StartPreview () again if you want to continue taking pictures, otherwise call takePicture again and crash
The next post:
Android Camera open practice (2)OpenGL ES use
Welcome to follow the public account: Sumsmile/focus on image processing mobile development veteran
The resources
[1] the Android developer: developer.android.com/reference/a…
[2] Camera Preview demo: github.com/darylgo/Cam…
[3] Android Platform Camera development Practice Guide: juejin.cn/post/684490…
[4] to understand the Android camera preview direction and direction: www.jianshu.com/p/7d88ec134…
[5] Android Camera1 tutorial preview: www.jianshu.com/p/705d4792e…
[6] camera getRotation: developer.android.com/reference/a…
[7] android camera API: developer.android.com/guide/topic…