How to draw a picture on Android platform

【Android audio and video learning road (two) 】AudioRecord API details and applications

【Android audio and video learning road (three) 】AudioTrack use and detailed explanation

【Android audio and video learning road (4) 】 the use of Camera

【Android audio and video learning road (five) 】MediaExtractor and MediaMuxer explained

【Android audio and Video learning road (6) 】 MediaCodec simple explanation

[Android Audio and video learning road (7)] Audio and video recording process practice

[Android audio and video learning road (eight)] YUV format exploration

OpenGL ES

“This is the fourth day of my participation in the August Gwen Challenge.

preface

This article mainly explains Android device screen related OpenGL ES coordinate system, define shape, shape surface basic knowledge, and define triangle and square.

Define a triangle

OpenGL ES allows you to define the image you’re drawing using three dimensional coordinates, so you have to define the coordinates of a triangle before you can draw it. In OpenGL, the typical way to do this is to define a vertex array of floating-point numbers for coordinates. For maximum efficiency, these coordinates can be written to ByteBuffer and passed to the OpenGL ES graphics pipe for processing.

class Triangle { private val vertexBuffer: FloatBuffer // Set color with red, green, blue and alpha (opacity) values var color = floatArrayOf(0.63671875f, 0.22265625 0.76953125 f, f, // initialize vertex byte buffer for shape coordinates // (number of coordinate values * 4 bytes per float)  val bb: ByteBuffer = ByteBuffer.allocateDirect(triangleCoords.size * 4) // use the device hardware's native byte order bb.order(ByteOrder.nativeOrder()) // create a floating point buffer from the ByteBuffer vertexBuffer = bb.asFloatBuffer() // add the coordinates to the FloatBuffer vertexBuffer.put(triangleCoords) // set the buffer to read the first coordinate vertexBuffer.position(0) } companion object { // number of coordinates per vertex in this array const val COORDS_PER_VERTEX = 3 var triangleCoords = floatArrayOf( // in counterclockwise order: Bottom left 0.0f, 0.0326427f, 0.0f, // top 0.0326427f, 0.0f, 0.0f, // bottom left 0.0326427f, Bottom right)}}Copy the code

By default, OpenGL ES uses a coordinate system. [0,0,0] (X, Y, Z) specifies the center of the GLSurfaceView frame, [1,1,0] is the upper right corner of the frame, and [-1, -1,0] is the lower left corner of the frame.

Note that the coordinates of this graph are defined in counterclockwise order. The drawing order is important because it defines which side is the front side of the graph you normally want to draw, and the back side.

Define a square

As you can see, it’s very easy to define a triangle in OpenGL. But what if you want something a little more complicated? Like a square? There’s a lot of ways to do this, but the way to draw this graph in OpenGL is to draw two triangles together

Similarly, you should define vertices for the two triangles representing the shape in counterclockwise order and place these values in a ByteBuffer. To avoid defining two coordinates shared by each triangle twice, use the drawing list to tell the OpenGL ES graphics pipeline how to draw these vertices. Here is the code for this shape:

class Square { private val vertexBuffer: FloatBuffer private val drawListBuffer: ShortBuffer private val drawOrder = shortArrayOf(0, 1, 2, 0, 2, 3) // order to draw vertices init { // initialize vertex byte buffer for shape coordinates // (# of coordinate values * 4 bytes per float) val bb: ByteBuffer = ByteBuffer.allocateDirect(squareCoords.size * 4) bb.order(ByteOrder.nativeOrder()) vertexBuffer = bb.asFloatBuffer() vertexBuffer.put(squareCoords) vertexBuffer.position(0) // initialize byte buffer for the draw list // (# of coordinate values * 2 bytes per short) val dlb: ByteBuffer = ByteBuffer.allocateDirect(drawOrder.size * 2) dlb.order(ByteOrder.nativeOrder()) drawListBuffer = dlb.asShortBuffer() drawListBuffer.put(drawOrder) drawListBuffer.position(0) } companion object { // number of Coordinates per vertex in this array const val COORDS_PER_VERTEX = 3 val squareCoords = floatArrayOf(-0.5f, 0.5f, F, // top left-0.5f, -0.5f, 0.0f, // bottom left 0.5f, -0.5f, 0.0f, // bottom right 0.5f, 0.0f, 0.0f) // top right}}Copy the code

This example gives you an idea of how to create more complex shapes with OpenGL. In general, you use a collection of triangles to draw objects. Now I’ll show you how to use the OpenGL ES 2.0 API to draw the shapes we defined in the previous section.

Initialize the shape

Before you can do any drawing, you must initialize and load the shape you want to draw. Unless the structure of the shape (i.e. the original coordinates) changes during execution, you should always initialize memory and efficiency in your Renderer’s method onSurfaceCreated().

private lateinit var triangle: Triangle private lateinit var square: Square override fun onSurfaceCreated(gl10: GL10? , config: EGLConfig?) { triangle = Triangle() square = Square() }Copy the code

4. Draw shapes

Drawing a defined shape using OpenGLES 2.0 requires a lot of code because you have to provide a lot of information for the graphics rendering pipeline. In particular, you must define the following:

  • Vertex Shader – OpenGLES graphic code for rendering the vertices of shapes.
  • Fragment Shader – OpenGLES code for rendering the look (color or texture) of a shape.
  • Program – An OpenGLES object that contains shaders that you want to use to draw one or more shapes.

You need at least one VertexShader to draw a shape and one FragmentShader to color the shape. These shapes must be compiled and added to an OpenGLES Program, which is then used to draw the shapes. Here is an example showing how to define a basic shader that can be used to draw shapes:

class Triangle {
    private val vertexShaderCode =
        "attribute v4 vPosition;" +
                "void main() {" +
                "   gl_position = vPosition" +
                "}"

    private val fragmentShaderCode =
        "precision mediump float;" +
                "uniform vec4 vColor;" +
                "void main() {" +
                " gl_FragColor = vColor;" +
                "}"

    ...
}
Copy the code

The Shaders include OpenGLShading Language (GLSL) code, which must be compiled before use. To compile this code, create a utility class method in your Renderer class:

private class MyGLRenderer : Renderer {
    ...
    companion object {
        fun loadShader(type: Int, shaderCode: String): Int {
            // create a vertex shader type (GLES20.GL_VERTEX_SHADER)
            // or a fragment shader type (GLES20.GL_FRAGMENT_SHADER
            val shader = GLES20.glCreateShader(type)

            // add the source code to the shader and compile it
            GLES20.glShaderSource(shader, shaderCode)
            GLES20.glCompileShader(shader)

            return shader
        }
    }

}
Copy the code

To draw your shapes, you must compile the shader code, add them to an OpenGLES Program object and then link to the Program. Do these things in the constructor of the Renderer object so that you only need to do them once.

Note: Compiling OpenGLES shaders and link programs is CPU intensive, so you should avoid doing this multiple times. If you do not know the contents of the shader at runtime, you should only create the code once and then cache them to avoid creating them multiple times.

class Triangle {

    ...
    private val program: Int

    init {
     
        val vertexShader = MyGLRenderer.loadShader(GLES20.GL_VERTEX_SHADER, vertexShaderCode)
        val fragmentShader = MyGLRenderer.loadShader(GLES20.GL_FRAGMENT_SHADER, fragmentShaderCode)

        // create empty OpenGL ES Program
        program = GLES20.glCreateProgram()

        // add the vertex shader to program
        GLES20.glAttachShader(program, vertexShader)

        // add the fragment shader to program
        GLES20.glAttachShader(program, fragmentShader)

        // creates OpenGL ES program executables
        GLES20.glLinkProgram(program)
    }
}
Copy the code

At this point, you are ready to add the actual draw call. You need to specify many parameters for the render pipeline to tell it what you want to draw and how. Because drawing operations vary from shape to shape, it’s a good idea to have your shape classes include their own drawing logic. Create a draw() method that draws the shape. The following code sets the position and color values to the shape’s VertexShader and FragmentShader, and then performs the draw function:

. private val vertexCount = triangleCoords.size / COORDS_PER_VERTEX private val vertexStride = COORDS_PER_VERTEX * 4 // 4 bytes per vertex fun draw() { // Add program to OpenGL ES environment GLES20.glUseProgram(program) // get handle to vertex shader's vPosition member val positionHandle = GLES20.glGetAttribLocation(program, "vPosition") // Enable a handle to the triangle vertices GLES20.glEnableVertexAttribArray(positionHandle) // Prepare the  triangle coordinate data GLES20.glVertexAttribPointer(positionHandle, COORDS_PER_VERTEX, GLES20.GL_FLOAT, false, vertexStride, vertexBuffer) // get handle to fragment shader's vColor member val colorHandle = GLES20.glGetUniformLocation(program, "vColor") // Set color for drawing the triangle GLES20.glUniform4fv(colorHandle, 1, color, 0) // Draw the triangl GLES20.glDrawArrays(GLES20.GL_TRIANGLES, 0, vertexCount) // Disable vertex array GLES20.glDisableVertexAttribArray(positionHandle) }Copy the code

Once all this code is done, drawing the object simply calls the draw() method in the renderer’s onDrawFrame() method:

override fun onDrawFrame(gl10: GL10?) {
    triangle.draw()
}
Copy the code

When you run the application, you should see the following: