The background,
Android on the OpenGL package is very good, is also very hidden, general users directly use GLSurfaceView can meet the requirements. In recent projects, many functions have been relegated to the c++ layer, so it is necessary to understand the underlying logic of OpenGL. Although Android provides various versions of OpenGL So library, but there is no encapsulation of the underlying API, So if you want to use C++ to write OpenGL, the best way to learn Android source code.
How to use GLSurfaceView
Before analyzing GLSurfaceView source code, it is necessary to introduce the use of GLSurfaceView:
surfaceView = findViewById(R.id.triangle_api_surfaceView) surfaceView.setEGLContextClientVersion(3) surfaceView.setRenderer(object : GLSurfaceView.Renderer { /** * Called when the surface is created or recreated. * * * Called when the rendering thread * starts and whenever the EGL context is lost. The EGL context will typically * be lost when the Android device awakes after going to sleep. * * * Since this method is called at the beginning of rendering, as well as * every time the EGL context is lost, this method is a convenient place to put * code to create resources that need to be created when the rendering * starts, and that need to be recreated when the EGL context is lost. * Textures are an example of a resource that you might want to create * here. * * * Note that when the EGL context is lost, all OpenGL resources associated * with that context will be automatically deleted. You do not need to call * the corresponding "glDelete" methods such as glDeleteTextures to * manually delete these lost resources. * * * @param gl the GL interface. Use `instanceof` to * test if the interface supports GL11 or higher interfaces. * @param config the EGLConfig of the created surface. Can be used * to create matching pbuffers. */ override fun onSurfaceCreated(gl: GL10? , config: EGLConfig?) {TODO (" What should be done after the surface is created, ** * Called when the surface changed size. ** * Called after the surface is created and whenever ** the OpenGL ES surface size changes. * * * Typically you will set your viewport here. If your camera * is fixed then you could also set your projection matrix here: * <pre class="prettyprint"> * void onSurfaceChanged(GL10 gl, int width, int height) { * gl.glViewport(0, 0, width, height); * // for a fixed camera, set the projection too * float ratio = (float) width / height; * gl.glMatrixMode(GL10.GL_PROJECTION); * gl.glLoadIdentity(); * gl.glFrustumf(-ratio, ratio, -1, 1, 1, 10); * } </pre> * * @param gl the GL interface. Use `instanceof` to * test if the interface supports GL11 or higher interfaces. * @param width * @param height */ override fun onSurfaceChanged(gl: GL10? , width: Int, height: Int) {TODO (" Render window size changes during video playback can be used to adjust resolution changes in view window, ** * Called to draw the current frame. ** * This method is responsible for drawing the current frame. * * * The implementation of this method typically looks like this: * <pre class="prettyprint"> * void onDrawFrame(GL10 gl) { * gl.glClear(GL10.GL_COLOR_BUFFER_BIT | GL10.GL_DEPTH_BUFFER_BIT); * / /... other gl calls to render the scene ... * } </pre> * * @param gl the GL interface. Use `instanceof` to * test if the interface supports GL11 or higher interfaces. */ override fun onDrawFrame(gl: GL10?) {TODO(" Load vertex, texture data and render ")}}) surfaceView.renderMode = glsurfaceView.rendermode_continuouslyCopy the codeAs you can see, using GLSurfaceView is quite simple, as long as the APP developer implements the following three methods:
- onSurfaceCreated(gl: GL10? , config: EGLConfig?)
- onSurfaceChanged(gl: GL10? , width: Int, height: Int)
- onDrawFrame(gl: GL10?)
The purpose of these three method annotations is not described here. The next article will cover how to render using GLSurfaceView. Note here that all three methods render within the GL thread, OpenGL thread. The core logic of OpenGL is all in the GL thread.
Third, source code analysis
Entry method
Let’s start with setRenderer, which does two main things: 1. Check the environment and variable synchronization configuration:
// Check the environment
checkRenderThreadState();
// Synchronize configuration items, if not set to default (lazy loading mode)
if (mEGLConfigChooser == null) {
mEGLConfigChooser = new SimpleEGLConfigChooser(true);
}
if (mEGLContextFactory == null) {
mEGLContextFactory = new DefaultContextFactory();
}
if (mEGLWindowSurfaceFactory == null) {
mEGLWindowSurfaceFactory = new DefaultWindowSurfaceFactory();
}
mRenderer = renderer;
Copy the codeRenderthreadstate () checks if GLThread exists. If GLThread exists, an exception is thrown. If GLThread exists, setRenderer(Renderer) cannot be called on the same GLSurfaceView for multiple times. Therefore, a rendering service can only be performed in one GLThread. If a rendering service needs to be performed in the same thread, more advanced functions such as FBO and multi-target rendering need to be learned. MEGLConfigChooser, mEGLContextFactory, and mEGLWindowSurfaceFactory are methods that users can call before setRenderer to set EGL. If none is set, the default implementation is used. MEGLConfigChooser specifies OpenGL color, depth, template, and other Settings. MEGLContextFactory is used to provide EGLContext creation and destruction processing. MEGLWindowSurfaceFactory is used to provide EGLSurface creation and destruction processing. Start a GL thread:
mGLThread = new GLThread(mThisWeakRef);
mGLThread.start();
Copy the codeThis is the famous GL thread. Mainly used for interaction with OpenGL API environment and render context switch and abnormal scene processing. The input parameter mThisWeakRef is a weak reference to the GLSurfaceView itself.
GLThread – The core logic of OpenGL
Let’s take a look at the implementation of GLThread. Let’s look at the run method first:
public void run(a) {
setName("GLThread " + getId());
if (LOG_THREADS) {
Log.i("GLThread"."starting tid=" + getId());
}
try {
guardedRun();
} catch (InterruptedException e) {
// fall thru and exit normally
} finally {
sGLThreadManager.threadExiting(this); }}Copy the codeContinuing to trace the code to guardedRun(), this method is a long, endless loop, so let’s list the key methods and highlight what they do and in what order they are called
private void guardedRun(a) throws InterruptedException {
mEglHelper = new EglHelper(mGLSurfaceViewWeakRef);
mHaveEglContext = false;
mHaveEglSurface = false;
mWantRenderNotification = false;
try{...while (true) {
synchronized (sGLThreadManager) {
while (true) {
// Used for pause, push and other state recovery.if (readyToDraw()) {
...
// Create an EGLContextmEglHelper.start(); . }... }...// Create an EGLSurface, which is essentially a block of memory
if(mEglHelper.createSurface()) { ... }...// Get the GL object and wrap the OpenGL API environment, using GL10 heregl = (GL10) mEglHelper.createGL(); .// Calls back the 'onSurfaceCreated()' method of the external Renderer objectGLSurfaceView view = mGLSurfaceViewWeakRef.get(); view.mRenderer.onSurfaceCreated(gl, mEglHelper.mEglConfig); .if (sizeChanged) {
...
// Call back to the 'onSurfaceChanged()' method on the external Renderer objectview.mRenderer.onSurfaceChanged(gl, w, h); . }...// Calls back the 'onDrawFrame()' method of the external Renderer objectview.mRenderer.onDrawFrame(gl); .//Egl interactive memory, opengL uses two memory buffers, one for display, the other for background drawing, after drawing OK, interactive memory for display
intswapError = mEglHelper.swap(); . }}finally {
/* * clean-up everything... * /
synchronized(sGLThreadManager) { stopEglSurfaceLocked(); stopEglContextLocked(); }}}Copy the codeAt the beginning of the method, an EglHelper is created, which is a utility class that encapsulates some common EGL operations. 1. Look at the readyToDraw() method:
private boolean readyToDraw(a) {
return(! mPaused) && mHasSurface && (! mSurfaceIsBad) && (mWidth >0) && (mHeight > 0)
&& (mRequestRender || (mRenderMode == RENDERMODE_CONTINUOUSLY));
}
Copy the codeMPaused is used to set whether paused. MHasSurface is used to mark whether the Surface has been created or not. The Surface here is different from EGLSurface. I will dig into the differences between the two later. MSurfaceIsBad is used to indicate whether the EGLSurface is available, while mWidth and mHeight are the width and height of the Surface. MRequestRender flags whether the user requests a refresh. GLSurfaceView has two refresh modes:
/**
* The renderer only renders
* when the surface is created, or when {@link #requestRender} is called.
*
* @see #getRenderMode()
* @see #setRenderMode(int)
* @see #requestRender()
*/
public final static int RENDERMODE_WHEN_DIRTY = 0;
/**
* The renderer is called
* continuously to re-render the scene.
*
* @see #getRenderMode()
* @see #setRenderMode(int)
*/
public final static int RENDERMODE_CONTINUOUSLY = 1;
Copy the codeRENDERMODE_WHEN_DIRTY is only rendered when the Surface is created and when Renderer is called. RENDERMODE_CONTINUOUSLY This mode keeps rendering back to readyToDraw(), which can be rendered when it is not paused and the toggle Surface and EGLSurface are normal.
Meglhelper.start () :
public void start(a) {
if (LOG_EGL) {
Log.w("EglHelper"."start() tid=" + Thread.currentThread().getId());
}
/* * Get an EGL instance */
mEgl = (EGL10) EGLContext.getEGL();
/* * Get to the default display. */
mEglDisplay = mEgl.eglGetDisplay(EGL10.EGL_DEFAULT_DISPLAY);
if (mEglDisplay == EGL10.EGL_NO_DISPLAY) {
throw new RuntimeException("eglGetDisplay failed");
}
/* * We can now initialize EGL for that display */
int[] version = new int[2];
if(! mEgl.eglInitialize(mEglDisplay, version)) {throw new RuntimeException("eglInitialize failed");
}
GLSurfaceView view = mGLSurfaceViewWeakRef.get();
if (view == null) {
mEglConfig = null;
mEglContext = null;
} else {
mEglConfig = view.mEGLConfigChooser.chooseConfig(mEgl, mEglDisplay);
/* * Create an EGL context. We want to do this as rarely as we can, because an * EGL context is a somewhat heavy object. */
mEglContext = view.mEGLContextFactory.createContext(mEgl, mEglDisplay, mEglConfig);
}
if (mEglContext == null || mEglContext == EGL10.EGL_NO_CONTEXT) {
mEglContext = null;
throwEglException("createContext");
}
if (LOG_EGL) {
Log.w("EglHelper"."createContext " + mEglContext + " tid=" + Thread.currentThread().getId());
}
mEglSurface = null;
}
Copy the codeEGLContext is the context used to link the render and view Windows, and the purpose of this code is to generate the mEglContext. If mEGLContextFactory is not set externally, it will use the default DefaultContextFactory. We can look at the implementation of DefaultContextFactory
public EGLContext createContext(EGL10 egl, EGLDisplay display, EGLConfig config) {
int[] attrib_list = {EGL_CONTEXT_CLIENT_VERSION, mEGLContextClientVersion,
EGL10.EGL_NONE };
returnegl.eglCreateContext(display, config, EGL10.EGL_NO_CONTEXT, mEGLContextClientVersion ! =0 ? attrib_list : null);
}
Copy the codeEgl’s eglCreateContext() method, which is the EGL10 API, will eventually be called to the Native API if you continue to track the code. Behind the opportunity to do a source Android system can see the Framework layer implementation.
3. Take a look at the meglhelper.createsurface () code
public boolean createSurface(a) {
if (LOG_EGL) {
Log.w("EglHelper"."createSurface() tid=" + Thread.currentThread().getId());
}
/* * Check preconditions. */
if (mEgl == null) {
throw new RuntimeException("egl not initialized");
}
if (mEglDisplay == null) {
throw new RuntimeException("eglDisplay not initialized");
}
if (mEglConfig == null) {
throw new RuntimeException("mEglConfig not initialized");
}
/* * The window size has changed, so we need to create a new * surface. */
destroySurfaceImp();
/* * Create an EGL surface we can render into. */
GLSurfaceView view = mGLSurfaceViewWeakRef.get();
if(view ! =null) {
mEglSurface = view.mEGLWindowSurfaceFactory.createWindowSurface(mEgl,
mEglDisplay, mEglConfig, view.getHolder());
} else {
mEglSurface = null;
}
if (mEglSurface == null || mEglSurface == EGL10.EGL_NO_SURFACE) {
int error = mEgl.eglGetError();
if (error == EGL10.EGL_BAD_NATIVE_WINDOW) {
Log.e("EglHelper"."createWindowSurface returned EGL_BAD_NATIVE_WINDOW.");
}
return false;
}
/* * Before we can issue GL commands, we need to make sure * the context is current and bound to a surface. */
if(! mEgl.eglMakeCurrent(mEglDisplay, mEglSurface, mEglSurface, mEglContext)) {/* * Could not make the context current, probably because the underlying * SurfaceView surface has been destroyed. */
logEglErrorAsWarning("EGLHelper"."eglMakeCurrent", mEgl.eglGetError());
return false;
}
return true;
}
Copy the code**createSurface()** Does two main things:
A) to generate EGLSurface
mEglSurface = view.mEGLWindowSurfaceFactory.createWindowSurface(mEgl,
mEglDisplay, mEglConfig, view.getHolder());
Copy the codeSimilarly, if there is no set mEGLWindowSurfaceFactory, default DefaultWindowSurfaceFactory see its implementation, the call to the eglCreateWindowSurface () method
public EGLSurface createWindowSurface(EGL10 egl, EGLDisplay display, EGLConfig config, Object nativeWindow) {
EGLSurface result = null;
try {
result = egl.eglCreateWindowSurface(display, config, nativeWindow, null);
} catch (IllegalArgumentException e) {
// This exception indicates that the surface flinger surface
// is not valid. This can happen if the surface flinger surface has
// been torn down, but the application has not yet been
// notified via SurfaceHolder.Callback.surfaceDestroyed.
// In theory the application should be notified first,
// but in practice sometimes it is not. See b/4588890
Log.e(TAG, "eglCreateWindowSurface", e);
}
return result;
}
Copy the codeB) Bind EGLSurface to the previously generated EGLContext
if(! mEgl.eglMakeCurrent(mEglDisplay, mEglSurface, mEglSurface, mEglContext)) {/* * Could not make the context current, probably because the underlying * SurfaceView surface has been destroyed. */
logEglErrorAsWarning("EGLHelper"."eglMakeCurrent", mEgl.eglGetError());
return false;
}
Copy the codeAfter this step, all pre-render environment preparation is complete. Then the main character, Renderer, gets to work. The Renderer follows in three steps:
// Calls back the 'onSurfaceCreated()' method of the external Renderer objectGLSurfaceView view = mGLSurfaceViewWeakRef.get(); view.mRenderer.onSurfaceCreated(gl, mEglHelper.mEglConfig); .if (sizeChanged) {
...
// Call back to the 'onSurfaceChanged()' method on the external Renderer objectview.mRenderer.onSurfaceChanged(gl, w, h); . }...// Calls back the 'onDrawFrame()' method of the external Renderer object
view.mRenderer.onDrawFrame(gl);
Copy the codeTo display Renderer data, you must call meglhelper.swap ();
The SurfaceView uses a dual memory buffer mechanism. There are two internal frambuffers, one for foreground display and the other for background drawing. After drawing OK, interactive memory will display. The purpose of eglSwapBuffers is for the rendered FrameBuffer to interact with the foreground FrameBuffer
/**
* Display the current render surface.
* @return the EGL error code from eglSwapBuffers.
*/
public int swap(a) {
if (! mEgl.eglSwapBuffers(mEglDisplay, mEglSurface)) {
return mEgl.eglGetError();
}
return EGL10.EGL_SUCCESS;
}
Copy the codeTo sum up:
- All render logic must be in the same thread. EGLContext and EGLSurface are thread bound.
- Render performs rendering only after EGLContext and EGLSurface have been generated and bound.
- In order for the rendered data to be displayed in the view window, it must be called
eglSwapBuffersExchange Framebuffer
How do GLthreads pause and resume
In the GLThread onPause() method, mRequestPaused = true is notified to all threads.
mRequestPaused = true;
sGLThreadManager.notifyAll();
Copy the codeLet’s see where else mRequestPaused is used? In the guardedRun() method of GLThread, you can see that mRequestPaused is used when mPaused! = mRequestPaused assigns the local variable pausing. I personally understand the reason for assigning this mRequestPaused to a local variable is that it is a dead-loop thread and the mRequestPaused variable can be released after the assignment, and it takes time to destroy the EGLContext and EGLSurface.
boolean pausing = false;
if(mPaused ! = mRequestPaused) { pausing = mRequestPaused; mPaused = mRequestPaused; sGLThreadManager.notifyAll();if (LOG_PAUSE_RESUME) {
Log.i("GLThread"."mPaused is now " + mPaused + " tid="+ getId()); }}Copy the codeWhen pausing = true, Release EGLSurface and EGLContext, mHaveEglSurface and mHaveEglContext will be set to false, and readyToDraw() will always return false, The code is going to be running inside the innermost while loop and not being able to jump out of the loop, waiting
// When pausing, release the EGL surface:
if (pausing && mHaveEglSurface) {
if (LOG_SURFACE) {
Log.i("GLThread"."releasing EGL surface because paused tid=" + getId());
}
stopEglSurfaceLocked();
}
// When pausing, optionally release the EGL Context:
if (pausing && mHaveEglContext) {
GLSurfaceView view = mGLSurfaceViewWeakRef.get();
boolean preserveEglContextOnPause = view == null ? false : view.mPreserveEGLContextOnPause;
if(! preserveEglContextOnPause) { stopEglContextLocked();if (LOG_SURFACE) {
Log.i("GLThread"."releasing EGL context because paused tid="+ getId()); }}}Copy the codeIf onResume() sets mRequestPaused to false and mRequestRender to true, readyToDraw() returns true. Meglhelper.start () returns true to break out of the innermost loop. At this point, the rendering logic can be executed normally.
How to exit GLThread
When I looked at the code I was thinking about an endless loop, how do I stop it? The thread will be stopped when surfaceDestroyed() is called. The final test found that the GLThread thread may still be rendering when surfaceDestroyed() is executed. EGL_BAD_SURFACE error code 0x300D will be reported when egl.swapbuffers () is implemented. Therefore, surfaceBuffers will be destroyed. At this point, the Surface has been destroyed, while the EGLSurface still exists and has not been destroyed in time.
switch (swapError) {
....
default:
// Other errors typically mean that the current surface is bad,
// probably because the SurfaceView surface has been destroyed,
// but we haven't been notified yet.
// Log the error to help developers understand why rendering stopped.
EglHelper.logEglErrorAsWarning("GLThread"."eglSwapBuffers", swapError);
synchronized(sGLThreadManager) {
mSurfaceIsBad = true;
sGLThreadManager.notifyAll();
}
break;
}
Copy the codeSo where is the logic to actually stop the thread? It can be seen from the above logic analysis that the thread stopping logic must notify the GLThread thread before Surface destruction, or an exception will occur. After a long search, I finally found a relationship between a piece of code and a GLThread thread. Let’s first look at the logic that can stop a thread in GLThread and the only thing that can stop a thread is this variable mShouldExit
while (true) {
synchronized (sGLThreadManager) {
while (true) {
if (mShouldExit) {
return; }... }}}Copy the codeTrace the code and find that mShouldExit is in requestExitAndWait(),
public void requestExitAndWait(a) {
// don't call this from GLThread thread or it is a guaranteed
// deadlock!
synchronized(sGLThreadManager) {
mShouldExit = true;
sGLThreadManager.notifyAll();
while (! mExited) {
try {
sGLThreadManager.wait();
} catch(InterruptedException ex) { Thread.currentThread().interrupt(); }}}}Copy the codeTracking down the code, it was finally revealed that the GLThread thread had been terminated in the onDetachedFromWindow() method.
@Override
protected void onDetachedFromWindow(a) {
if (LOG_ATTACH_DETACH) {
Log.d(TAG, "onDetachedFromWindow");
}
if(mGLThread ! =null) {
mGLThread.requestExitAndWait();
}
mDetached = true;
super.onDetachedFromWindow();
}
Copy the codeconclusion
- From GLSurfaceView source code analysis to learn the role and principle of GLThread, also imitate GLThread in C++ call EGL and OpenGL ES API to achieve a time.
- I found my shortcomings from the code analysis, such as the details of SurfaceView double buffering mechanism, the relationship between Surface and EGLSurface in Framework layer, and I will continue to dig into the underlying implementation later.
Github code: github.com/JianYeung/B…