preface
CameraX is a Jetpack support library designed to help you simplify camera application development. Although there are a lot of information about CameraX on the Internet, many of them are basically copies of the information on the official website. The value of learning is not as high as that of reading the official website directly.
Also introduced some bloggers CameraX combined with examples of OpenGL rendering, but seems to be based on the Preview class setOnPreviewOutputUpdateListener this method for processing, But the author found that setOnPreviewOutputUpdateListener update CameraX version after this method directly, out came…
Of course the methods described in this article will be outdated as CameraX evolves, but hopefully it will be a bit of a primer…
show me the code
Create an OpenGL render View from GLSurfaceView, glcameraView.java:
public class GLCameraView extends GLSurfaceView implements GLSurfaceView.Renderer, SurfaceTexture.OnFrameAvailableListener { private static final String LOG_TAG = "OpenGLCameraX"; private Executor executor = Executors.newSingleThreadExecutor(); private int textureId; private SurfaceTexture surfaceTexture; private int vPosition; private int vCoord; private int programId; private int textureMatrixId; private float[] textureMatrix = new float[16]; protected FloatBuffer mGLVertexBuffer; protected FloatBuffer mGLTextureBuffer; public GLCameraView(Context context) { this(context, null); } public GLCameraView(Context context, AttributeSet attrs) { super(context, attrs); setEGLContextClientVersion(2); setRenderer(this); // Set setRenderMode(glsurfaceView.rendermode_when_dirty); } @SuppressLint("UnsafeExperimentalUsageError") public void attachPreview(Preview preview) { preview.setSurfaceProvider(new Preview.SurfaceProvider() { @Override public void onSurfaceRequested(@NonNull SurfaceRequest request) { Surface surface = new Surface(surfaceTexture); request.provideSurface(surface, executor, new Consumer<SurfaceRequest.Result>() { @Override public void accept(SurfaceRequest.Result result) { surface.release(); surfaceTexture.release(); Log.v(LOG_TAG, "--accept------"); }}); }}); } @Override public void onSurfaceCreated(GL10 gl, EGLConfig config) { int[] ids = new int[1]; // OpenGL related gles20.glgentextures (1, ids, 0); textureId = ids[0]; surfaceTexture = new SurfaceTexture(textureId); surfaceTexture.setOnFrameAvailableListener(this::onFrameAvailable); String vertexShader = OpenGLUtils.readRawTextFile(getContext(), R.raw.camera_vertex); String fragmentShader = OpenGLUtils.readRawTextFile(getContext(), R.raw.camera_frag); programId = OpenGLUtils.loadProgram(vertexShader, fragmentShader); vPosition = GLES20.glGetAttribLocation(programId, "vPosition"); vCoord = GLES20.glGetAttribLocation(programId, "vCoord"); textureMatrixId = GLES20.glGetUniformLocation(programId, "textureMatrix"); // 4 vertices, each vertex has two floating-point types, Each float of 4 bytes mGLVertexBuffer = ByteBuffer. AllocateDirect (4 * 4 * 2). The order (ByteOrder. NativeOrder ()). AsFloatBuffer (); mGLVertexBuffer.clear(); / / VERTEX coordinates float VERTEX [] = {1.0 f, 1.0 f, f 1.0, 1.0, f to 1.0 f, f 1.0, 1.0 f, 1.0 f}; mGLVertexBuffer.put(VERTEX); / / texture coordinates mGLTextureBuffer = ByteBuffer. AllocateDirect (4 * 2 * 4). The order (ByteOrder. NativeOrder ()). AsFloatBuffer (); mGLTextureBuffer.clear(); // Normal texture map coordinates, but the posted map is upside down, So you need to revise the / / float [] TEXTURE = {/ f / 0.0, 1.0, f/f / 1.0, 1.0, f/f / 0.0, 0.0, f/f / 1.0, f / 0.0 /}; Float [] TEXTURE = {0.0f, 0.0f, 1.0f, 0.0f, 1.0f, 1.0f, 1.0f}; float[] TEXTURE = {0.0f, 0.0f, 1.0f, 1.0f, 1.0f}; float[] TEXTURE = {0.0f, 0.0f, 1.0f, 1.0f}; mGLTextureBuffer.put(TEXTURE); } @Override public void onSurfaceChanged(GL10 gl, int width, int height) { GLES20.glViewport(0, 0, width, height); } @override public void onDrawFrame(GL10 gl) {gles20. glClearColor(1, 0, 0, 0); GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT); / / update the texture surfaceTexture updateTexImage (); surfaceTexture.getTransformMatrix(textureMatrix); GLES20.glUseProgram(programId); / / transformation matrix GLES20. GlUniformMatrix4fv (textureMatrixId, 1, false, textureMatrix, 0). // Pass coordinate data mglVerTexBuffer. position(0); GLES20.glVertexAttribPointer(vPosition, 2, GLES20.GL_FLOAT, false, 0, mGLVertexBuffer); GLES20.glEnableVertexAttribArray(vPosition); / / pass texture coordinates mGLTextureBuffer. Position (0); GLES20.glVertexAttribPointer(vCoord, 2, GLES20.GL_FLOAT, false, 0, mGLTextureBuffer); GLES20.glEnableVertexAttribArray(vCoord); GlActiveTexture (gles20.gl_texture0); GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, textureId); GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, 0, 4); Gles20. glBindTexture(gles11ext.gl_texture_external_oes, 0); } @Override public void onFrameAvailable(SurfaceTexture surfaceTexture) { requestRender(); }}Copy the codeCamera_vertex. GLSL:
attribute vec4 vPosition;
attribute vec4 vCoord;
varying vec2 aCoord;
uniform mat4 textureMatrix;
void main(){
gl_Position = vPosition;
aCoord = (textureMatrix * vCoord).xy;
}
Copy the codeWrite the fragment shader camera_frag.glsl:
#extension GL_OES_EGL_image_external : require
//SurfaceTexture比较特殊
//float数据是什么精度的
precision mediump float;
//采样点的坐标
varying vec2 aCoord;
//采样器
uniform samplerExternalOES vTexture;
void main(){
//变量 接收像素值
// texture2D：采样器 采集 aCoord的像素
//赋值给 gl_FragColor 就可以了
gl_FragColor = texture2D(vTexture,aCoord);
}
Copy the codeLoad and compile the shader openglutils.java:
public static String readRawTextFile(Context context, int rawId) { InputStream is = context.getResources().openRawResource(rawId); BufferedReader br = new BufferedReader(new InputStreamReader(is)); String line; StringBuilder sb = new StringBuilder(); try { while ((line = br.readLine()) ! = null) { sb.append(line); sb.append("\n"); } } catch (Exception e) { e.printStackTrace(); } try { br.close(); } catch (IOException e) { e.printStackTrace(); } return sb.toString(); } @param fSource @param fSource @return */ public static int loadProgram(String vSource, String fSource){/** * vertex shader */ int vShader = gles20.glcreateshader (gles20.gl_vertex_shader); // Load the shader code gles20.glshaderSource (vShader,vSource); // Compile (configure) gles20.glcompileshader (vShader); Int [] status = new int[1]; GLES20.glGetShaderiv(vShader, GLES20.GL_COMPILE_STATUS,status,0); if(status[0] ! = GLES20. GL_TRUE) {/ / failed throw new an IllegalStateException (" load vertex shader: "+ GLES20. GlGetShaderInfoLog (vShader)); } /** * fShader = gles20.glcreateshader (gles20.gl_fragment_shader); // Load the shader code gles20.glshaderSource (fShader,fSource); // Compile (configure) gles20.glcompileshader (fShader); Gles20.glgetshaderiv (fShader, gles20.gl_compile_status,status,0); // Check whether the configuration is successful. if(status[0] ! = GLES20. GL_TRUE) {/ / failed throw new an IllegalStateException (" load fragment shader: "+ GLES20. GlGetShaderInfoLog (vShader)); } /** * create shader program */ int program = gles20.glCreateProgram (); GlAttachShader (program,vShader); GLES20.glAttachShader(program,fShader); // link shader program gles20.gllinkProgram (program); Gles20.glgetprogramiv (program, gles20.gl_link_status,status,0); if(status[0] ! = GLES20.GL_TRUE){ throw new IllegalStateException("link program:"+ GLES20.glGetProgramInfoLog(program)); } GLES20.glDeleteShader(vShader); GLES20.glDeleteShader(fShader); return program; }Copy the codeUse mainactivity.java with CameraX:
public class MainActivity extends AppCompatActivity { private GLCameraView camera_preview; static { System.loadLibrary("native-lib"); } @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); camera_preview = findViewById(R.id.camera_preview); if (allPermissionsGranted()) { startCamera(); } else { ActivityCompat.requestPermissions( this, new String[]{Manifest.permission.CAMERA}, 100); } } @Override public void onRequestPermissionsResult(int requestCode, @NonNull String[] permissions, @NonNull int[] grantResults) { super.onRequestPermissionsResult(requestCode, permissions, grantResults); if (requestCode == 100) { if (allPermissionsGranted()) { startCamera(); } else {toast.maketext (this, "no camera permission ", toast.length_long).show(); } } } private boolean allPermissionsGranted() { return ContextCompat.checkSelfPermission(this, Manifest.permission.CAMERA) == PackageManager.PERMISSION_GRANTED; } private void startCamera() { Executor executor = Executors.newSingleThreadExecutor(); ListenableFuture<ProcessCameraProvider> processCameraProvider = ProcessCameraProvider.getInstance(this); processCameraProvider.addListener(new Runnable() { @Override public void run() { try { ProcessCameraProvider cameraProvider = processCameraProvider.get(); Preview preview = new Preview.Builder() .build(); camera_preview.attachPreview(preview); cameraProvider.unbindAll(); cameraProvider.bindToLifecycle(MainActivity.this, CameraSelector.DEFAULT_BACK_CAMERA,preview); } catch (ExecutionException e) { e.printStackTrace(); } catch (InterruptedException e) { e.printStackTrace(); } } }, ContextCompat.getMainExecutor(this)); } /** * A native method that is implemented by the 'native-lib' native library, * which is packaged with this application. */ public native String stringFromJNI(); }Copy the codeAdd some comments to key code points and call it a day.
The lines
1. The current portrait preview looks normal, but the preview interface is obviously deformed when it is landscape. How to solve this problem? Interested children can understand the matrix transformation of OpenGL, using matrix transformation to solve this problem.
2. The default low resolution is used for preview. What should I do if I need to preview high resolution?
3. I tested the frame rate during the preview and it was about 26 frames per second. How to change it to 60 frames per second?
4. OpenGL beginners should know about VBO, VAO, FBO and other related concepts. If you want to further study OpenGL, you can also combine VBO, VAO, FBO and CameraX to do a practice.
Beep beep beep
CameraX has been around for over two years, but there has been no official release. It seems that a beat version has been released recently, and the API has been changing as I’ve learned. So I think CameraX is the future, but not the present.
While CameraX is still unstable and may even have problems, the odds are in favor of those who plan ahead, and continuing to learn from the evolution of CameraX is like learning from the engineers at Google.
Reference: Google Official
Pay attention to me, progress together, life is more than coding!!
