[sound Ming]

First of all, this series of articles are based on their own understanding and practice, there may be wrong places, welcome to correct.

Secondly, this is an introductory series, covering only enough knowledge, and there are many blog posts on the Internet for in-depth knowledge. Finally, in the process of writing the article, I will refer to the articles shared by others and list them at the end of the article, thanking these authors for their sharing.

Code word is not easy, reproduced please indicate the source!

Tutorial code: [Making portal】

directory

First, Android audio and video hard decoding:
  • 1. Basic knowledge of audio and video
  • 2. Audio and video hard decoding process: packaging basic decoding framework
  • 3. Audio and video playback: audio and video synchronization
  • 4, audio and video unencapsulation and packaging: generate an MP4
2. Use OpenGL to render video frames
  • 1. Preliminary understanding of OpenGL ES
  • 2. Use OpenGL to render video images
  • 3, OpenGL rendering multi-video, picture-in-picture
  • 4. Learn more about EGL of OpenGL
  • 5, OpenGL FBO data buffer
  • 6, Android audio and video hardcoding: generate an MP4
Android FFmpeg audio and video decoding
  • 1, FFmpeg SO library compilation
  • 2. Android introduces FFmpeg
  • 3, Android FFmpeg video decoding playback
  • 4, Android FFmpeg+OpenSL ES audio decoding playback
  • 5, Android FFmpeg+OpenGL ES play video
  • Android FFmpeg Simple Synthesis MP4: Video unencapsulation and Reencapsulation
  • 7, Android FFmpeg video encoding

You can read about it in this article

EGL serves as a bridge between OpenGL and local window rendering, and is often ignored or ignored by developers who are new to OpenGL. With the further study, EGL will be something we have to face. This article explains what EGL is, what it does, and how to use it.

What is EGL

As an Android developer, EGL seems like a strange place to be. Why?

Blame it on Android’s GLSurfaceView encapsulation. Ha ha ha ~

1. Why does onDrawFrame keep getting called back?

As mentioned in the previous article, OpenGL is thread-based. Until now, we haven’t fully understood this problem, but we do know that when we inherit glSurfaceView.renderer, the system calls back the following methods:

override fun onSurfaceCreated(gl: GL10? , config:EGLConfig?).{}override fun onSurfaceChanged(gl: GL10? , width:Int, height: Int){}override fun onDrawFrame(gl: GL10?).{}Copy the code

And the onDrawerFrame method is constantly called, which is where we implement the Drawing process of OpenGL.

So we can guess, is it possible that the thread that gets called over and over again is a while loop?

The answer is: Yes.

If you look at the GLSurfaceView source, you’ll find a thread called GLThread that initializes the EGL content. And when appropriate, three methods in the Renderer are called.

So, what exactly is EGL?

2. What is EGL?

We know that OpenGL is a set of apis that can manipulate gpus, but just being able to manipulate gpus doesn’t render images to the device’s display window. Then you need an intermediate layer that connects OpenGL to the device window, preferably cross-platform.

Thus came EGL, a set of platform-independent apis provided by the Khronos Group.

3. Some basic knowledge of EGL
  • EGLDisplay

EGL defines an abstract system display class for manipulating device Windows.

  • EGLConfig

EGL configurations, such as RGBA bits

  • EGLSurface

Render cache, a piece of memory where all the image data to be rendered on the screen is cached on the EGLSurface.

  • EGLContext

OpenGL context, used to store OpenGL drawing state information, data.

The process of initializing EGL is actually the process of configuring the above information.

How to use EGL

Just by looking at the above, it is difficult to understand exactly what EGL does or how it should be used.

Let me ask you a question

If there are two GlSurfaceViews rendering video at the same time, why can OpenGL correctly draw the video into two GLSurfaceViews?

If you think about every API in OpenGL ES, is there any API that specifies which GLSurfaceView the current image should be rendered to?

No!

Please read the following with this question in mind.

1. Encapsulate the EGL core API

First, the core of EGL initialization (the four described in section 1) is wrapped and named EGLCore

const val FLAG_RECORDABLE = 0x01

private const val EGL_RECORDABLE_ANDROID = 0x3142

class EGLCore {

    private val TAG = "EGLCore"

    // EGL related variables
    private var mEGLDisplay: EGLDisplay = EGL14.EGL_NO_DISPLAY
    private var mEGLContext = EGL14.EGL_NO_CONTEXT
    private var mEGLConfig: EGLConfig? = null

    /** * Initialize EGLDisplay *@paramEglContext Shares the context */
    fun init(eglContext: EGLContext? , flags:Int) {
        if(mEGLDisplay ! == EGL14.EGL_NO_DISPLAY) {throw RuntimeException("EGL already set up")}valsharedContext = eglContext ? : EGL14.EGL_NO_CONTEXT// 1, create EGLDisplay
        mEGLDisplay = EGL14.eglGetDisplay(EGL14.EGL_DEFAULT_DISPLAY)
        if (mEGLDisplay === EGL14.EGL_NO_DISPLAY) {
            throw RuntimeException("Unable to get EGL14 display")}// 2, initialize EGLDisplay
        val version = IntArray(2)
        if(! EGL14.eglInitialize(mEGLDisplay, version,0, version, 1)) {
            mEGLDisplay = EGL14.EGL_NO_DISPLAY
            throw RuntimeException("unable to initialize EGL14")}// 3, initialize EGLConfig, EGLContext
        if (mEGLContext === EGL14.EGL_NO_CONTEXT) {
            val config = getConfig(flags, 2) ?: throw RuntimeException("Unable to find a suitable EGLConfig")
            val attr2List = intArrayOf(EGL14.EGL_CONTEXT_CLIENT_VERSION, 2, EGL14.EGL_NONE)
            val context = EGL14.eglCreateContext(
                mEGLDisplay, config, sharedContext,
                attr2List, 0
            )
            mEGLConfig = config
            mEGLContext = context
        }
    }

    /** * Get EGL configuration information *@paramFlags Initialization flag *@paramVersion EGL version */
    private fun getConfig(flags: Int, version: Int): EGLConfig? {
        var renderableType = EGL14.EGL_OPENGL_ES2_BIT
        if (version >= 3) {
            // Configure EGL 3
            renderableType = renderableType or EGLExt.EGL_OPENGL_ES3_BIT_KHR
        }

        // Configure the array, mainly to configure RAGA bits and depth bits
        // A pair of keys followed by a value
        // The array must end with egl14.egl_none
        val attrList = intArrayOf(
            EGL14.EGL_RED_SIZE, 8,
            EGL14.EGL_GREEN_SIZE, 8,
            EGL14.EGL_BLUE_SIZE, 8,
            EGL14.EGL_ALPHA_SIZE, 8.//EGL14.EGL_DEPTH_SIZE, 16,
            //EGL14.EGL_STENCIL_SIZE, 8,
            EGL14.EGL_RENDERABLE_TYPE, renderableType,
            EGL14.EGL_NONE, 0.// placeholder for recordable [@-3]
            EGL14.EGL_NONE
        )
        // Configure the tag specified by Android
        if(flags and FLAG_RECORDABLE ! =0) {
            attrList[attrList.size - 3] = EGL_RECORDABLE_ANDROID
            attrList[attrList.size - 2] = 1
        }
        val configs = arrayOfNulls<EGLConfig>(1)
        val numConfigs = IntArray(1)

        // Get a list of available EGL configurations
        if(! EGL14.eglChooseConfig(mEGLDisplay, attrList,0,
                configs, 0, configs.size,
                numConfigs, 0)) {
            Log.w(TAG, "Unable to find RGB8888 / $version EGLConfig")
            return null
        }
        
        Use the first configuration recommended by the system
        return configs[0]}/** * Create a displayable render cache *@paramSurface render window surface */
    fun createWindowSurface(surface: Any): EGLSurface {
        if (surface !is Surface && surface !is SurfaceTexture) {
            throw RuntimeException("Invalid surface: $surface")}val surfaceAttr = intArrayOf(EGL14.EGL_NONE)

        val eglSurface = EGL14.eglCreateWindowSurface(
                                        mEGLDisplay, mEGLConfig, surface,
                                        surfaceAttr, 0)

        if (eglSurface == null) {
            throw RuntimeException("Surface was null")}return eglSurface
    }

    /** * Create off-screen render cache *@paramWidth Cache window width *@paramHeight Cache window height */
    fun createOffscreenSurface(width: Int, height: Int): EGLSurface {
        val surfaceAttr = intArrayOf(EGL14.EGL_WIDTH, width,
                                                 EGL14.EGL_HEIGHT, height,
                                                 EGL14.EGL_NONE)

        val eglSurface = EGL14.eglCreatePbufferSurface(
                                        mEGLDisplay, mEGLConfig,
                                        surfaceAttr, 0)

        if (eglSurface == null) {
            throw RuntimeException("Surface was null")}return eglSurface
    }

    /** * binds the current thread to the context */
    fun makeCurrent(eglSurface: EGLSurface) {
        if (mEGLDisplay === EGL14.EGL_NO_DISPLAY) {
            throw RuntimeException("EGLDisplay is null, call init first")}if(! EGL14.eglMakeCurrent(mEGLDisplay, eglSurface, eglSurface, mEGLContext)) {throw RuntimeException("makeCurrent(eglSurface) failed")}}/** * binds the current thread to the context */
    fun makeCurrent(drawSurface: EGLSurface, readSurface: EGLSurface) {
        if (mEGLDisplay === EGL14.EGL_NO_DISPLAY) {
            throw RuntimeException("EGLDisplay is null, call init first")}if(! EGL14.eglMakeCurrent(mEGLDisplay, drawSurface, readSurface, mEGLContext)) {throw RuntimeException("eglMakeCurrent(draw,read) failed")}}/** * Sends the cached image data to the device for display */
    fun swapBuffers(eglSurface: EGLSurface): Boolean {
        return EGL14.eglSwapBuffers(mEGLDisplay, eglSurface)
    }

    /** * Sets the time of the current frame in nanoseconds */
    fun setPresentationTime(eglSurface: EGLSurface, nsecs: Long) {
        EGLExt.eglPresentationTimeANDROID(mEGLDisplay, eglSurface, nsecs)
    }

    /** * Destroy the EGLSurface and unbind the context */
    fun destroySurface(elg_surface: EGLSurface) {
        EGL14.eglMakeCurrent(
            mEGLDisplay, EGL14.EGL_NO_SURFACE, EGL14.EGL_NO_SURFACE,
            EGL14.EGL_NO_CONTEXT
        )
        EGL14.eglDestroySurface(mEGLDisplay, elg_surface);
    }

    /** * Release resources */
    fun release(a) {
        if(mEGLDisplay ! == EGL14.EGL_NO_DISPLAY) {// Android is unusual in that it uses a reference-counted EGLDisplay. So for
            // every eglInitialize() we need an eglTerminate().EGL14.eglMakeCurrent( mEGLDisplay, EGL14.EGL_NO_SURFACE, EGL14.EGL_NO_SURFACE, EGL14.EGL_NO_CONTEXT ) EGL14.eglDestroyContext(mEGLDisplay, mEGLContext) EGL14.eglReleaseThread() EGL14.eglTerminate(mEGLDisplay) } mEGLDisplay = EGL14.EGL_NO_DISPLAY mEGLContext =  EGL14.EGL_NO_CONTEXT mEGLConfig =null}}Copy the code

This is the most basic, concise encapsulation of EGL initialization, and almost every method is required.

To be specific:

  • Initialize init in 3 steps:

    • Create EGLDisplay with eglGetDisplay
    • EGLDisplay is initialized with eglInitialize
    • Initialize the EGLContext with eglCreateContext

The getConfig method is called when EGLCongtext is initialized.

  • Configure context getConfig:

    • Configure the version flag based on the EGL version selected
    • Initialize the configuration list to configure a pair of rGBA and depth bits for rendering, with the first as a type and the second as a value, and must end with egl14.egl_none.
    • Configure the Android-specific attribute EGL_RECORDABLE_ANDROID.
    • According to the above configuration information, through eglChooseConfig, the system will return a list of matching configuration information, generally use to return the first configuration information.

Android specifies the flag EGL_RECORDABLE_ANDROID

Tell EGL that the surface it creates must be compatible with the video codec. Without this flag, EGL might use a Buffer that MediaCodec does not understand. This variable comes with the system after API26. For compatibility, we write the value 0x3142 ourselves.

  • Create EGLSurface, divided into two modes:

    • Displayable window, created using eglCreateWindowSurface.
    • An off-screen (invisible) window, created using eglCreatePbufferSurface.

The first, the most common, is to bind the Surface, or SurfaceTexture, held by the SurfaceView on the page. So the image data processed by OpenGL can be displayed on the screen.

The second method is for off-screen rendering, that is, the image data processed by OpenGL is stored in the cache and not displayed on the screen, but the whole rendering process is the same as normal mode, so that the user does not need to see the image data.

  • Bind the OpenGL rendering thread to the drawing context: makeCurrent

    • Use eglMakeCurrent to implement the binding.

At this point, EGL is initialized using the methods encapsulated in EGLCore. But it still doesn’t answer the above question.

The answer is in glMakeCurrent.

GlMakeCurrent this method, the realization of device display window (EGLDisplay), OpenGL context (EGLContext), image data cache (GLSurface), the current thread binding.

Note here: “Binding of current thread”.

Now to answer the question posed above: Why does OpenGL draw correctly in multiple GLSurfaceViews?

After EGL initialization, that is, after the rendering environment (EGLDisplay, EGLContext, GLSurface) is ready, glMakeCurrent needs to be explicitly called in the rendering thread (the thread drawing the image). At this point, the system layer binds the OpenGL rendering environment to the current thread.

After that, whenever you call any OpenGL ES API in the rendering thread (such as glES20.glgentextures), OpenGL will automatically switch the context (i.e. switch OpenGL’s rendering information and resources) based on the current thread.

In other words, if you call OpenGL API from a thread that does not call glMakeCurrent, the system will not find the corresponding OpenGL context, and therefore will not find the corresponding resource, which may cause an exception error.

That’s why it’s said that OpenGL rendering must be done in an OpenGL thread.

In fact, all three of the GLSurfaceView#Renderer callback methods are called from GLThread.

  • Swap cached data and display image: swapBuffers

    • EglSwapBuffers is a method provided by EGL for displaying EGLSurface data to the device screen. After drawing the image in OpenGL, call this method to actually display it.

  • Unbind the data cache surface and free resources

    • When the Surface on the page is destroyed (e.g. App to background), resources need to be unbound.
    • When the page exits, the SurfaceView is destroyed and all resources need to be freed.

The above just encapsulates the core API, and the next step is to create a new class to call it.

2. Call the EGL core method

Here, create a new EGLSurfaceHolder to manipulate the EGLCore

class EGLSurfaceHolder {

    private val TAG = "EGLSurfaceHolder"

    private lateinit var mEGLCore: EGLCore

    private var mEGLSurface: EGLSurface? = null

    fun init(shareContext: EGLContext? = null, flags: Int) {
        mEGLCore = EGLCore()
        mEGLCore.init(shareContext, flags)
    }

    fun createEGLSurface(surface: Any? , width:Int = - 1, height: Int = - 1) {
        mEGLSurface = if(surface ! =null) {
            mEGLCore.createWindowSurface(surface)
        } else {
            mEGLCore.createOffscreenSurface(width, height)
        }
    }

    fun makeCurrent(a) {
        if(mEGLSurface ! =null) { mEGLCore.makeCurrent(mEGLSurface!!) }}fun swapBuffers(a) {
        if(mEGLSurface ! =null) { mEGLCore.swapBuffers(mEGLSurface!!) }}fun destroyEGLSurface(a) {
        if(mEGLSurface ! =null) {
            mEGLCore.destroySurface(mEGLSurface!!)
            mEGLSurface = null}}fun release(a) {
        mEGLCore.release()
    }
}
Copy the code

The code is very simple, the most important thing is to hold the EGLSurface (of course, you can also put the EGLSurface in the EGLCore), and open up more concise EGL operations to external calls.

3. Simulate GLSurfaceView and use EGL to achieve rendering

To better understand EGL, here’s how to use EGL by simulating GLSurfaceView.

  • Customize a CustomerRender
class CustomerGLRenderer : SurfaceHolder.Callback {

    //OpenGL render thread
    private val mThread = RenderThread()

    // Weak reference to SurfaceView on page
    private var mSurfaceView: WeakReference<SurfaceView>? = null

    // All the renderers
    private val mDrawers = mutableListOf<IDrawer>()

    init {
        // Start the render thread
        mThread.start()
    }
    
    /** * Set SurfaceView */
    fun setSurface(surface: SurfaceView) {
        mSurfaceView = WeakReference(surface)
        surface.holder.addCallback(this)

        surface.addOnAttachStateChangeListener(object : View.OnAttachStateChangeListener{
            override fun onViewDetachedFromWindow(v: View?). {
                mThread.onSurfaceStop()
            }

            override fun onViewAttachedToWindow(v: View?).{}})}/** * add the renderer */
    fun addDrawer(drawer: IDrawer) {
        mDrawers.add(drawer)
    }

    override fun surfaceCreated(holder: SurfaceHolder?). {
        mThread.onSurfaceCreate()
    }

    override fun surfaceChanged(holder: SurfaceHolder? , format:Int, width: Int, height: Int) {
        mThread.onSurfaceChange(width, height)
    }

    override fun surfaceDestroyed(holder: SurfaceHolder?). {
        mThread.onSurfaceDestroy()
    }
}
Copy the code

Mainly as follows:

  1. A custom RenderThread named RenderThread
  2. A weak reference to the SurfaceView
  3. A list of plotters

When initialized, the render thread is started. Then it forwards the SurfaceView life cycle to the render thread, and nothing else.

  • Define render state
/** * Render state */
enum class RenderState {
    NO_SURFACE, // There is no valid surface
    FRESH_SURFACE, // Hold a new, uninitialized surface
    SURFACE_CHANGE, // Change the surface size
    RENDERING, // After initialization, you can start rendering
    SURFACE_DESTROY, / / surface destruction
    STOP // Stop drawing
}
Copy the code

Based on these states, the thread’s execution state is switched in the RenderThread.

The instructions are as follows:

  1. The thread starts, enters the while(true) loop with the state NO_SURFACE, and enters the hold on;
  2. After Surface Create, the state changes to FRESH_SURFACE.
  3. After Surface change, enter SURFACE_CHANGE state.
  4. After executing SURFACE_CHANGE, it will enter RENDERING state automatically.
  5. Render every 20ms without other interruptions;
  6. If the Surface is destroyed, re-enter the NO_SURFACE state; If there is a new surface, repeat 2-5.
  7. If the SurfaceView is destroyed, it enters the STOP state and the render thread exits, end.
  • Execute render loop
inner class RenderThread: Thread() {
    
    // Render state
    private var mState = RenderState.NO_SURFACE
    
    private var mEGLSurface: EGLSurfaceHolder? = null

    // Whether EGLSurface is bound
    private var mHaveBindEGLContext = false

    // Whether the EGL context has been created to determine whether a new texture ID needs to be produced
    private var mNeverCreateEglContext = true

    private var mWidth = 0
    private var mHeight = 0

    private val mWaitLock = Object()

//------------ Part 1: Thread waiting and unlocking -----------------

    private fun holdOn(a) {
        synchronized(mWaitLock) {
            mWaitLock.wait()
        }
    }

    private fun notifyGo(a) {
        synchronized(mWaitLock) {
            mWaitLock.notify()
        }
    }

//------------ Part 2: Surface declares the periodic forwarding function ------------

    fun onSurfaceCreate(a) {
        mState = RenderState.FRESH_SURFACE
        notifyGo()
    }

    fun onSurfaceChange(width: Int, height: Int) {
        mWidth = width
        mHeight = height
        mState = RenderState.SURFACE_CHANGE
        notifyGo()
    }

    fun onSurfaceDestroy(a) {
        mState = RenderState.SURFACE_DESTROY
        notifyGo()
    }

    fun onSurfaceStop(a) {
        mState = RenderState.STOP
        notifyGo()
    }

//------------ Part 3: OpenGL render loop ------------

    override fun run(a) {
        [1] Initialize EGL
        initEGL()
        while (true) {
            when (mState) {
                RenderState.FRESH_SURFACE -> {
                    [2] Use surface to initialize EGLSurface and bind the context
                    createEGLSurfaceFirst()
                    holdOn()
                }
                RenderState.SURFACE_CHANGE -> {
                    createEGLSurfaceFirst()
                    [3] Initialize the width and height of the OpenGL world coordinate system
                    GLES20.glViewport(0.0, mWidth, mHeight)
                    configWordSize()
                    mState = RenderState.RENDERING
                }
                RenderState.RENDERING -> {
                    // [4] Enter loop rendering
                    render()
                }
                RenderState.SURFACE_DESTROY -> {
                    [5] Destroy the EGLSurface and unbind the context
                    destroyEGLSurface()
                    mState = RenderState.NO_SURFACE
                }
                RenderState.STOP -> {
                    // [6] Release all resources
                    releaseEGL()
                    return
                }
                else -> {
                    holdOn()
                }
            }
            sleep(20)}}//------------ Part 4: EGL related operations ------------

    private fun initEGL(a){ mEGLSurface = EGLSurfaceHolder() mEGLSurface? .init(null, EGL_RECORDABLE_ANDROID)
    }

    private fun createEGLSurfaceFirst(a) {
        if(! mHaveBindEGLContext) { mHaveBindEGLContext =true
            createEGLSurface()
            if (mNeverCreateEglContext) {
                mNeverCreateEglContext = false
                generateTextureID()
            }
        }
    }

    private fun createEGLSurface(a){ mEGLSurface? .createEGLSurface(mSurfaceView? .get()? .holder? .surface) mEGLSurface? .makeCurrent() }private fun destroyEGLSurface(a){ mEGLSurface? .destroyEGLSurface() mHaveBindEGLContext =false
    }

    private fun releaseEGL(a){ mEGLSurface? .release() }//------------ Part 5: OpenGL ES operations -------------

    private fun generateTextureID(a) {
        val textureIds = OpenGLTools.createTextureIds(mDrawers.size)
        for ((idx, drawer) in mDrawers.withIndex()) {
            drawer.setTextureID(textureIds[idx])
        }
    }

    private fun configWordSize(a) {
        mDrawers.forEach { it.setWorldSize(mWidth, mHeight) }
    }

    private fun render(a){ GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT or GLES20.GL_DEPTH_BUFFER_BIT) mDrawers.forEach { it.draw() } mEGLSurface? .swapBuffers() } }Copy the code

It is mainly divided into five parts, 1-2 is very simple, I believe everyone can understand. As for 4-5, these are all methods called in run.

Focus on part 3, which is the Run method.

[1] initEGL initializes EGL with the EGLSurfaceHolder before entering while(true).

Note that initEGL is called only once, meaning that EGL is initialized only once, no matter how many times the surface is destroyed and rebuilt later.

[2] After you have available surface, go to FRESH_SURFACE state and call createEGLSurface and makeCurrent of EGLSurfaceHolder to bind thread, context and window.

[3] Set the width and height of OpenGL window according to the width and height of surface window, and then automatically enter RENDERING state. This corresponds to the onSurfaceChanged method callback in glSurfaceView. Renderer.

[4] Enter the loop render, here render the picture every 20ms. Call the onDrawFrame method in glSurfaceView. Renderer.

For comparison, here’s the SimpleRender defined in the previous article:

class SimpleRender: GLSurfaceView.Renderer {

    private val drawers = mutableListOf<IDrawer>()

    override fun onSurfaceCreated(gl: GL10? , config:EGLConfig?). {
        GLES20.glClearColor(0f.0f.0f.0f)
        // Turn on the blending, i.e. Translucent
        GLES20.glEnable(GLES20.GL_BLEND)
        GLES20.glBlendFunc(GLES20.GL_SRC_ALPHA, GLES20.GL_ONE_MINUS_SRC_ALPHA)

        val textureIds = OpenGLTools.createTextureIds(drawers.size)
        for ((idx, drawer) in drawers.withIndex()) {
            drawer.setTextureID(textureIds[idx])
        }
    }

    override fun onSurfaceChanged(gl: GL10? , width:Int, height: Int) {
        GLES20.glViewport(0.0, width, height)
        for (drawer in drawers) {
            drawer.setWorldSize(width, height)
        }
    }

    override fun onDrawFrame(gl: GL10?). {
        GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT or GLES20.GL_DEPTH_BUFFER_BIT)
        drawers.forEach {
            it.draw()
        }
    }

    fun addDrawer(drawer: IDrawer) {
        drawers.add(drawer)
    }
}
Copy the code

[5] If the Surface is destroyed (for example, in the background), call the EGLSurfaceHolder’s destroyEGLSurface destroy and unbind window.

Note: When the page comes back to the foreground, the surface will be recreated. Just re-create the EGLSurface and bind the context to the EGLSurface, and you can continue rendering without opening a new rendering thread.

[6] The SurfaceView is destroyed (for example, the page Finish), no longer needs to render, all EGL resources need to be freed, and the thread exits.

4. Use the renderer

Create a new page EGLPlayerActivity

<? xml version="1.0" encoding="utf-8"? > <android.support.constraint.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
        android:layout_width="match_parent"
        android:layout_height="match_parent" xmlns:app="http://schemas.android.com/apk/res-auto">
    <SurfaceView
            android:id="@+id/sfv"
            app:layout_constraintTop_toTopOf="parent"
            android:layout_width="match_parent"
            android:layout_height="match_parent"/>
</android.support.constraint.ConstraintLayout>
Copy the code
class EGLPlayerActivity: AppCompatActivity() {
    private val path = Environment.getExternalStorageDirectory().absolutePath + "/mvtest_2.mp4"
    private val path2 = Environment.getExternalStorageDirectory().absolutePath + "/mvtest.mp4"

    private val threadPool = Executors.newFixedThreadPool(10)

    private var mRenderer = CustomerGLRenderer()

    override fun onCreate(savedInstanceState: Bundle?). {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_egl_player)
        initFirstVideo()
        initSecondVideo()
        setRenderSurface()
    }

    private fun initFirstVideo(a) {
        val drawer = VideoDrawer()
        drawer.setVideoSize(1920.1080)
        drawer.getSurfaceTexture {
            initPlayer(path, Surface(it), true)
        }
        mRenderer.addDrawer(drawer)
    }

    private fun initSecondVideo(a) {
        val drawer = VideoDrawer()
        drawer.setAlpha(0.5 f)
        drawer.setVideoSize(1920.1080)
        drawer.getSurfaceTexture {
            initPlayer(path2, Surface(it), false)
        }
        mRenderer.addDrawer(drawer)

        Handler().postDelayed({
            drawer.scale(0.5 f.0.5 f)},1000)}private fun initPlayer(path: String, sf: Surface, withSound: Boolean) {
        val videoDecoder = VideoDecoder(path, null, sf)
        threadPool.execute(videoDecoder)
        videoDecoder.goOn()

        if (withSound) {
            val audioDecoder = AudioDecoder(path)
            threadPool.execute(audioDecoder)
            audioDecoder.goOn()
        }
    }

    private fun setRenderSurface(a) {
        mRenderer.setSurface(sfv)
    }
}
Copy the code

The entire process is almost the same as the GLSurfaceView used to render the video in the previous article.

The only difference is that you need to set the SurfaceView to the CustomerRenderer.

At this point, I’m ready to play the video. EGL basics and how to use them are basically covered.

Why learn EGL when you already have GLSurfaceView?

Just listen to me blow on the water, ha ha ha.

Use of EGL

1. Deepen your understanding of OpenGL

If you haven’t studied EGL carefully, then your OpenGL career will be incomplete, because you will never have a deep understanding of how OpenGL renders, and you will be unable to deal with some problems.

2. Android video must be hardcoded using EGL

EGL is essential if you need to use the coding capabilities of Android Mediacodec, and you’ll see how to use EGL in a future article on video coding.

3. EGL related knowledge is required for FFmpeg codec

At the JNI layer, Android does not implement a glSurfaceView-like tool to help hide EGL content. Therefore, if you need to implement FFmpeg codec in C++, you will need to implement the entire OpenGL rendering process yourself.

This is the real purpose of LEARNING EGL. GLSurfaceView is sufficient for rendering video frames only.

So, EGL, must learn!

4. Refer to the article

OpenGL EGL usage practices

Analyze the Android system EGL and GL thread from the source point of view