Overview of Graphic Rendering

Android platform provides a wealth of official controls for developers to achieve interface UI development, but in the actual business often encounter a variety of customization requirements, which must be achieved by developers through the way of self-drawing controls. Generally, Android provides Canvas and OpenGL ES two ways to achieve Canvas, which is implemented by the Skia 2D vector graphics processing function library at the bottom of Android. So how do you draw with Canvas and OpenGL? This must rely on the View class provided by Android to implement, the following combination of common application methods, as shown below:

Canvas

  • View + Canvas
  • SurfaceView + Canvas
  • TextureView + Canvas

OpenGL ES

  • SurfaceView + OpenGL ES
  • GLSurfaceView + OpenGL ES
  • TextureView + OpenGL ES

View + Canvas

This is a common way to draw your own controls by overriding the View class’s onDraw(Canvas Canvas) method. When you need to refresh the drawing, call the invalidate() method and let the View object refresh itself. This scheme is relatively simple and involves less custom logic. The disadvantage is that the drawing logic is carried out in the UI thread, the refresh efficiency is not high, and 3D rendering is not supported.

public class CustomView extends View {
    public CustomView(Context context) {
        super(context);
    }
 
    public CustomView(Context context, @Nullable AttributeSet attrs) {
        super(context, attrs);
    }
 
    public CustomView(Context context, @Nullable AttributeSet attrs, int defStyleAttr) {
        super(context, attrs, defStyleAttr);
    }
 
    @Override
    protected void onDraw(Canvas canvas) {
        // draw whatever.}}Copy the code

SurfaceView + Canvas

Compared with View + Canvas, this method uses SurfaceView, so it creates a Surface of its own on the WMS system of Android for rendering. Its drawing logic can be carried out in an independent thread, so its performance is more efficient than View + Canvas. However, it is usually necessary to create a drawing thread and implement the SurfaceHolder-callback interface to manage the life cycle of the SurfaceView. The implementation logic is slightly more complex than View + Canvas. In addition, it still does not support 3D rendering, and since Surface is not in hierachy View, its display is not controlled by View attributes, so it cannot be translated, zooming and other transformations, nor can it be placed in other viewgroups, and SurfaceView cannot be nested.

public class CustomSurfaceView extends SurfaceView implements SurfaceHolder.Callback.Runnable {
 
    private boolean mRunning = false;
    private SurfaceHolder mSurfaceHolder;
 
    public CustomSurfaceView(Context context) {
        super(context);
        initView();
    }
 
    public CustomSurfaceView(Context context, AttributeSet attrs) {
        super(context, attrs);
        initView();
    }
 
    public CustomSurfaceView(Context context, AttributeSet attrs, int defStyleAttr) {
        super(context, attrs, defStyleAttr);
        initView();
    }
 
    private void initView(a) {
        getHolder().addCallback(this);
    }
 
    @Override
    public void surfaceCreated(SurfaceHolder holder) {
        mSurfaceHolder = holder;
        new Thread(this).start();
    }
 
    @Override
    public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
        mSurfaceHolder = holder;
    }
 
    @Override
    public void surfaceDestroyed(SurfaceHolder holder) {
        mRunning = false;
    }
 
    @Override
    public void run(a) {
        mRunning = true;
        while (mRunning) {
            SystemClock.sleep(333);
            Canvas canvas = mSurfaceHolder.lockCanvas();
            if(canvas ! =null) {
                try {
                    synchronized(mSurfaceHolder) { onRender(canvas); }}finally{ mSurfaceHolder.unlockCanvasAndPost(canvas); }}}}private void onRender(Canvas canvas) {
        // draw whatever.}}Copy the code

TextureView + Canvas

This method is similar to SurfaceView + Canvas method, but because it is implemented through TextureView, it can eliminate the defect that Surface is not in View hierachy. TextureView does not create a separate window in WMS. It is a normal View in the hierachy of the View, so it can be moved, rotated, scaled, animated and changed like any other normal View. This approach has its drawbacks. It must be used in hardware-accelerated Windows, takes up more memory than SurfaceView, and renders in the main UI thread before 5.0, and a separate render thread after 5.0.

public class CustomTextureView extends TextureView implements TextureView.SurfaceTextureListener.Runnable {
 
    private boolean mRunning = false;
    private SurfaceTexture mSurfaceTexture;
    private Surface mSurface;
    private Rect mRect;
 
    public CustomTextureView(Context context) {
        super(context);
        initView();
    }
 
    public CustomTextureView(Context context, AttributeSet attrs) {
        super(context, attrs);
        initView();
    }
 
    public CustomTextureView(Context context, AttributeSet attrs, int defStyleAttr) {
        super(context, attrs, defStyleAttr);
        initView();
    }
 
    private void initView(a) {
        setSurfaceTextureListener(this);
    }
 
    @Override
    public void onSurfaceTextureAvailable(SurfaceTexture surface, int width, int height) {
        mSurfaceTexture = surface;
        mRect = new Rect(0.0, width, height);
        mSurface = new Surface(mSurfaceTexture);
        new Thread(this).start();
    }
 
    @Override
    public void onSurfaceTextureSizeChanged(SurfaceTexture surface, int width, int height) {
        mSurfaceTexture = surface;
        mRect = new Rect(0.0, width, height);
        mSurface = new Surface(mSurfaceTexture);
    }
 
    @Override
    public boolean onSurfaceTextureDestroyed(SurfaceTexture surface) {
        mRunning = false;
        return false;
    }
 
    @Override
    public void onSurfaceTextureUpdated(SurfaceTexture surface) {}@Override
    public void run(a) {
        mRunning = true;
        while (mRunning) {
            SystemClock.sleep(333);
            Canvas canvas = mSurface.lockCanvas(mRect);
            if(canvas ! =null) {
                try {
                    synchronized(mSurface) { onRender(canvas); }}finally{ mSurface.unlockCanvasAndPost(canvas); }}}}private void onRender(Canvas canvas) {
        canvas.drawColor(Color.RED);
        // draw whatever.}}Copy the code

All of the above are common ways of 2D graphics rendering. If you want to carry out 3D graphics rendering or advanced image processing (such as filter, AR and other effects), you must introduce OpenGL ES to achieve it. OpenGL ES (OpenGL for Embedded Systems) is a subset of OpenGL 3D graphics API. It is designed for mobile phones, PDAs, game consoles and other Embedded devices. It is a design standard of graphics rendering API. Different software and hardware developers may have different implementations within the OpenGL API. Here is how to render OpenGL ES on The Android platform. There are usually three ways:

SurfaceView + OpenGL ES

EGL is the interface between OpenGL API and native window system. The platform independence of OpenGL ES is realized by EGL, and EGL shields the differences between different platforms. If you use the OpenGL API to draw graphics, you must first build the EGL environment.

The usual steps for EGL rendering are:

– Obtain the EGLDisplay object, establish a connection with the local window system call the eglGetDisplay method to get the EGLDisplay.

– Initialize the EGL method. After opening the connection, call the eglInitialize method to initialize it.

– Get the EGLConfig object, determine the configuration information of the render surface and call the eglChooseConfig method to get EGLConfig.

Using EGLDisplay and EGLConfig, call eglCreateWindowSurface or eglCreatePbufferSurface to create a rendering surface to get the EGLSurface.

Using EGLDisplay and EGLConfig, call the eglCreateContext method to create the EGLContext.

– Binding context Binds EGLSurface, EGLContext, and EGLDisplay with eglMakeCurrent method. After binding, OpenGLES environment is created, and then rendering can be performed.

– Swap buffers After OpenGLES are drawn, use the eglSwapBuffers method to swap before and after buffers to display the drawn content to the screen, and do not call this method for off-screen rendering.

– Release EGL When EGL is no longer needed, unbind eglMakeCurrent and destroy EGLDisplay, EGLSurface, and EGLContext.

The above EGL environment construction is quite complex, so I will not explain too much here. The following code can refer to its specific implementation:

public class OpenGLSurfaceView extends SurfaceView implements SurfaceHolder.Callback.Runnable {
    private boolean mRunning = false;
    private SurfaceHolder mSurfaceHolder;
 
    public OpenGLSurfaceView(Context context) {
        super(context);
        initView();
    }
 
    public OpenGLSurfaceView(Context context, AttributeSet attrs) {
        super(context, attrs);
        initView();
    }
 
    public OpenGLSurfaceView(Context context, AttributeSet attrs, int defStyleAttr) {
        super(context, attrs, defStyleAttr);
        initView();
    }
 
    private void initView(a) {
        getHolder().addCallback(this);
    }
 
    @Override
    public void surfaceCreated(SurfaceHolder holder) {
        mSurfaceHolder = holder;
        new Thread(this).start();
    }
 
    @Override
    public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
        mSurfaceHolder = holder;
    }
 
    @Override
    public void surfaceDestroyed(SurfaceHolder holder) {
        mRunning = false;
    }
 
    @Override
    public void run(a) {
        // Create an EGL instance
        EGL10 egl = (EGL10) EGLContext.getEGL();
        //
        EGLDisplay dpy = egl.eglGetDisplay(EGL10.EGL_DEFAULT_DISPLAY);
        // Initialize EGLDisplay
        int[] version = new int[2];
        egl.eglInitialize(dpy, version);
 
        int[] configSpec = {
                EGL10.EGL_RED_SIZE,      5,
                EGL10.EGL_GREEN_SIZE,    6,
                EGL10.EGL_BLUE_SIZE,     5,
                EGL10.EGL_DEPTH_SIZE,   16,
                EGL10.EGL_NONE
        };
 
        EGLConfig[] configs = new EGLConfig[1];
        int[] num_config = new int[1];
        // Select config to create the OpengL runtime environment
        egl.eglChooseConfig(dpy, configSpec, configs, 1, num_config);
        EGLConfig config = configs[0];
 
        EGLContext context = egl.eglCreateContext(dpy, config,
                EGL10.EGL_NO_CONTEXT, null);
        // Create a new surface
        EGLSurface surface = egl.eglCreateWindowSurface(dpy, config, mSurfaceHolder, null);
        // Set the Opengles environment to current
        egl.eglMakeCurrent(dpy, surface, surface, context);
        // Get the current Opengles canvas
        GL10 gl = (GL10)context.getGL();
 
        mRunning = true;
        while (mRunning) {
            SystemClock.sleep(333);
            synchronized (mSurfaceHolder) {
                onRender(gl);
 
                // Displays the drawing result on the screen
                egl.eglSwapBuffers(dpy, surface);
            }
        }
 
        egl.eglMakeCurrent(dpy, EGL10.EGL_NO_SURFACE, EGL10.EGL_NO_SURFACE, EGL10.EGL_NO_CONTEXT);
        egl.eglDestroySurface(dpy, surface);
        egl.eglDestroyContext(dpy, context);
        egl.eglTerminate(dpy);
    }
 
    private void onRender(GL10 gl) {
        gl.glClearColor(1.0 F.0.0 F.0.0 F.1.0 F);
        // Clears the screen and depth buffer.gl.glClear(GL10.GL_COLOR_BUFFER_BIT | GL10.GL_DEPTH_BUFFER_BIT); }}Copy the code

As can be seen from the above code, there are mainly the following two changes in the drawing method of SurfaceView + Canvas:

  • Added EGL environment construction code before and after the while(true) loop
  • The onRender() method uses GL10 instead of Canvas

GLSurfaceView + OpenGL ES

Due to the complexity of building an EGL environment and the need to robustly maintain a thread, direct OpenGL drawing using SurfaceView is not convenient. Fortunately, the Android platform provides the GLSurfaceView class, which encapsulates this logic well and enables developers to quickly develop OpenGL rendering. To render graphics using the GLSurfaceView class, you need to implement the GLSurfaceView.renderer interface, which provides an onDrawFrame(GL10 GL) method within which to implement the specific rendering logic.

public class OpenGLGLSurfaceView extends GLSurfaceView implements GLSurfaceView.Renderer {
    public OpenGLGLSurfaceView(Context context) {
        super(context);
        setRenderer(this);
    }
 
    public OpenGLGLSurfaceView(Context context, AttributeSet attrs) {
        super(context, attrs);
        setRenderer(this);
    }
 
    @Override
    public void onSurfaceCreated(GL10 gl, EGLConfig config) {
        // pass through
    }
 
    @Override
    public void onSurfaceChanged(GL10 gl, int width, int height) {
        gl.glViewport(0.0, width, height);
    }
 
    @Override
    public void onDrawFrame(GL10 gl) {
        gl.glClearColor(1.0 F.0.0 F.0.0 F.1.0 F);
        // Clears the screen and depth buffer.gl.glClear(GL10.GL_COLOR_BUFFER_BIT | GL10.GL_DEPTH_BUFFER_BIT); }}Copy the code

TextureView + OpenGL ES

This method is similar to SurfaceView + OpenGL ES. One advantage of using this method is that it is implemented through TextureView, so it can eliminate the defect that Surface is not in View hierachy. TextureView does not create a separate window in WMS, but as a normal View within the View hierachy, so it can be moved, rotated, scaled, animated, etc., just like any other normal View. When building an EGL environment using the TextureView class here, note that the argument passed to eglCreateWindowSurface() is a SurfaceTexture instance.

public class OpenGLTextureView extends TextureView implements TextureView.SurfaceTextureListener.Runnable {
    private boolean mRunning = false;
    private SurfaceTexture mSurfaceTexture;
 
    public OpenGLTextureView(Context context) {
        super(context);
        initView();
    }
 
    public OpenGLTextureView(Context context, AttributeSet attrs) {
        super(context, attrs);
        initView();
    }
 
    public OpenGLTextureView(Context context, AttributeSet attrs, int defStyleAttr) {
        super(context, attrs, defStyleAttr);
        initView();
    }
 
    private void initView(a) {
        setSurfaceTextureListener(this);
    }
 
    @Override
    public void onSurfaceTextureAvailable(SurfaceTexture surface, int width, int height) {
        mSurfaceTexture = surface;
        new Thread(this).start();
    }
 
    @Override
    public void onSurfaceTextureSizeChanged(SurfaceTexture surface, int width, int height) {
        mSurfaceTexture = surface;
    }
 
    @Override
    public boolean onSurfaceTextureDestroyed(SurfaceTexture surface) {
        mRunning = false;
        return false;
    }
 
    @Override
    public void onSurfaceTextureUpdated(SurfaceTexture surface) {}@Override
    public void run(a) {
        // Create an EGL instance
        EGL10 egl = (EGL10) EGLContext.getEGL();
        //
        EGLDisplay dpy = egl.eglGetDisplay(EGL10.EGL_DEFAULT_DISPLAY);
        // Initialize EGLDisplay
        int[] version = new int[2];
        egl.eglInitialize(dpy, version);
 
        int[] configSpec = {
                EGL10.EGL_RED_SIZE,      5,
                EGL10.EGL_GREEN_SIZE,    6,
                EGL10.EGL_BLUE_SIZE,     5,
                EGL10.EGL_DEPTH_SIZE,   16,
                EGL10.EGL_NONE
        };
 
        EGLConfig[] configs = new EGLConfig[1];
        int[] num_config = new int[1];
        // Select config to create the OpengL runtime environment
        egl.eglChooseConfig(dpy, configSpec, configs, 1, num_config);
        EGLConfig config = configs[0];
 
        EGLContext context = egl.eglCreateContext(dpy, config,
                EGL10.EGL_NO_CONTEXT, null);
        // Create a new surface
        EGLSurface surface = egl.eglCreateWindowSurface(dpy, config, mSurfaceTexture, null);
        // Set the Opengles environment to current
        egl.eglMakeCurrent(dpy, surface, surface, context);
        // Get the current Opengles canvas
        GL10 gl = (GL10)context.getGL();
 
        mRunning = true;
        while (mRunning) {
            SystemClock.sleep(333);
            synchronized (mSurfaceTexture) {
                onRender(gl);
 
                // Displays the drawing result on the screen
                egl.eglSwapBuffers(dpy, surface);
            }
        }
 
        egl.eglMakeCurrent(dpy, EGL10.EGL_NO_SURFACE, EGL10.EGL_NO_SURFACE, EGL10.EGL_NO_CONTEXT);
        egl.eglDestroySurface(dpy, surface);
        egl.eglDestroyContext(dpy, context);
        egl.eglTerminate(dpy);
    }
 
    private void onRender(GL10 gl) {
        gl.glClearColor(1.0 F.0.0 F.1.0 F.1.0 F);
        // Clears the screen and depth buffer.gl.glClear(GL10.GL_COLOR_BUFFER_BIT | GL10.GL_DEPTH_BUFFER_BIT); }}Copy the code

Code Sample Reference

Github.com/sunjinbo/hi…