What resources can be shared when OpenGL ES shares context?

The original article was published on the wechat official account: Byte Flow

EGL concept review

EGL is the communication interface between OpenGL ES and Native Window System. Its main functions are as follows:

Communicate with the device’s native windowing system;
Query available types and configurations of drawing surfaces;
Create a drawing surface;
Synchronous rendering between OpenGL ES and other graphics rendering apis;
Manage rendering resources such as texture maps.

The platform independence of OpenGL ES is achieved through EGL, which blocks the differences between different platforms (Apple provides its own iOS implementation of the EGL API, calling itself EAGL).

The native window related API provides access to the native window system, and EGL creates the rendering surface, EGLSurface, and provides the graphics rendering context, EGLContext, for state management, on which OpenGL ES can then draw.

In the picture:

Display(EGLDisplay) is an abstraction of the actual Display device;
Surface (EGLSurface) is an abstraction of the FrameBuffer area of memory used to store images, including the Color Buffer, Stencil Buffer, and Depth Buffer;
Context (EGLContext) stores some state information of OpenGL ES drawings;

When developing OpenGL ES applications on Android platform, the GLSurfaceView class has provided us with the management of Display, Surface, Context, namely GLSurfaceView internal implementation of EGL encapsulation, Can be very convenient to use the interface GLSurfaceView.Renderer implementation, using OpenGLES API for rendering, to a large extent to improve the convenience of OpenGLES development.

Of course, we can also implement EGL encapsulation by ourselves. This paper is to implement EGL encapsulation at Native layer, realize image background rendering without GLSurfaceView, and use GPU to complete efficient image processing.

For more detailed use of EGL, see OpenGL ES 3.0 Development (6) : EGL in the article series

Which resources can be shared when the context is shared

When context is shared, what resources can be shared across threads? This is the main point of this article.

For the sake of some readers’ patience, let’s get straight to the conclusion.

Shared resources:

Texture;
Shader.
Shader program;
Buffer objects, such as VBO, EBO, and RBO.

Resources that cannot be shared:

FBO frame buffer object (not in the buffer class);
VAO vertex array object (not in the Buffer class).

FBO and VAO are resource management objects. FBO manages several buffers and does not occupy resources. VAO manages VBO or EBO and does not occupy resources.

In the next section, we will start a new rendering thread outside the main rendering thread, and then share the textures, programs and other resources generated by the main rendering thread with the new rendering thread.

Shared context multi-threaded rendering

This section will open up a new off-screen rendering thread by sharing EGLContext outside the main rendering thread, then share the texture, program and VBO resources generated by the main rendering thread to the new rendering thread for use, and finally return the texture saving (the new rendering thread) rendering results to the main thread for on-screen rendering.

Shared context

In EGL_VERSION_1_4 (Android 5.0), the context object EGLContext can be obtained directly by calling eglGetCurrentContext in the current rendering thread.

Both the C++ and Java layers have apis for obtaining context objects:

//Java
EGL14.eglGetCurrentContext();

//C++
#include "egl.h"
eglGetCurrentContext();
Copy the code

When we use EGL to create a rendering environment in a new thread, we create the context object for the new thread using the sharedContext obtained by the main rendering thread.

EGLContext context = eglCreateContext(mEGLDisplay, config,
                                              sharedContext, attrib2_list);
Copy the code

Since we are rendering to the off-screen area in the new thread, we need to create a PbufferSurface.

EGLSurface eglSurface = eglCreatePbufferSurface(mEGLDisplay, mEGLConfig, surfaceAttribs);
Copy the code

As an international convention, we encapsulate the OPERATIONS of EGL into a class EglCore for easy use. For specific code, please refer to the project at the end of this article.

Multi-threaded rendering

Like the Looper class in the Android Java layer, we implement Looper in C++ to create new threads and manage messages within them.

class Looper {

public:
    Looper(a); Looper&operator= (const Looper& ) = delete;
    Looper(Looper&) = delete;
    virtual ~Looper(a);void postMessage(int what, bool flush = false);
    void postMessage(int what, void *obj, bool flush = false);
    void postMessage(int what, int arg1, int arg2, bool flush = false);
    void postMessage(int what, int arg1, int arg2, void *obj, bool flush = false);

    void quit(a);

    virtual void handleMessage(LooperMessage *msg);

private:
    void addMessage(LooperMessage *msg, bool flush);

    static void *trampoline(void *p);

    void loop(void);

    LooperMessage *head;
    pthread_t worker;
    sem_t headWriteProtect;
    sem_t headDataAvailable;
    bool running;

};
Copy the code

OnSurfaceCreated, OnSurfaceChanged, and OnDrawFrame are defined in the GLRenderLooper class to handle the corresponding events.

enum {
    MSG_SurfaceCreated,
    MSG_SurfaceChanged,
    MSG_DrawFrame,
    MSG_SurfaceDestroyed,
};

class GLRenderLooper : public Looper {
public:
    GLRenderLooper(a);virtual ~GLRenderLooper(a);static GLRenderLooper* GetInstance(a);
    static void ReleaseInstance(a);

private:
    virtual void handleMessage(LooperMessage *msg);

    void OnSurfaceCreated(a);
    void OnSurfaceChanged(int w, int h);
    void OnDrawFrame(a);
    void OnSurfaceDestroyed(a);

    bool CreateFrameBufferObj(a);

private:
    static mutex m_Mutex;
    static GLRenderLooper* m_Instance;

    GLEnv *m_GLEnv;
    EglCore *m_EglCore = nullptr;
    OffscreenSurface *m_OffscreenSurface = nullptr;
    GLuint m_VaoId;
    GLuint m_FboTextureId;
    GLuint m_FboId;
};
Copy the code

In the function GLRenderLooper: : OnSurfaceCreated, using sharedContext create OpenGL rendering environment.

void GLRenderLooper::OnSurfaceCreated(a) {

	// Create an OpenGL off-screen rendering environment using sharedContext
    m_EglCore = new EglCore(m_GLEnv->sharedCtx, FLAG_RECORDABLE);
    SizeF imgSizeF = m_GLEnv->imgSize;
    m_OffscreenSurface = new OffscreenSurface(m_EglCore, imgSizeF.width, imgSizeF.height);
    m_OffscreenSurface->makeCurrent(a);glGenVertexArrays(1, &m_VaoId);
    glBindVertexArray(m_VaoId);

    glBindBuffer(GL_ARRAY_BUFFER, m_GLEnv->vboIds[0]);
    glEnableVertexAttribArray(0);
    glVertexAttribPointer(0.3, GL_FLOAT, GL_FALSE, 3 * sizeof(GLfloat), (const void *)0);
    glBindBuffer(GL_ARRAY_BUFFER, GL_NONE);

    glBindBuffer(GL_ARRAY_BUFFER, m_GLEnv->vboIds[1]);
    glEnableVertexAttribArray(1);
    glVertexAttribPointer(1.2, GL_FLOAT, GL_FALSE, 2 * sizeof(GLfloat), (const void *)0);
    glBindBuffer(GL_ARRAY_BUFFER, GL_NONE);

    glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, m_GLEnv->vboIds[2]);
    GO_CHECK_GL_ERROR(a);glBindVertexArray(GL_NONE);

    if (!CreateFrameBufferObj())
    {
        LOGCATE("GLRenderLooper::OnSurfaceCreated CreateFrameBufferObj fail"); }}Copy the code

GLRenderLooper: : OnDrawFrame function, mapped pay attention to the exchange buffer, and then save the texture mapping results, through DiaoHan digital handed back on the main thread to screen rendering.

void GLRenderLooper::OnDrawFrame(a) {
    LOGCATE("GLRenderLooper::OnDrawFrame");
    SizeF imgSizeF = m_GLEnv->imgSize;

    glBindFramebuffer(GL_FRAMEBUFFER, m_FboId);
    glViewport(0.0, imgSizeF.width, imgSizeF.height);
    glUseProgram(m_GLEnv->program);
    glBindVertexArray(m_VaoId);
    glActiveTexture(GL_TEXTURE0);
    glBindTexture(GL_TEXTURE_2D, m_GLEnv->inputTexId);
    GLUtils::setInt(m_GLEnv->program, "s_TextureMap".0);
    float offset = (sin(m_FrameIndex * MATH_PI / 80) + 1.0 f) / 2.0 f;
    GLUtils::setFloat(m_GLEnv->program, "u_Offset", offset);
    glDrawElements(GL_TRIANGLES, 6, GL_UNSIGNED_SHORT, (const void *)0);
    glBindVertexArray(0);
    glBindTexture(GL_TEXTURE_2D, 0);
	
	// Note the swap buffer
    m_OffscreenSurface->swapBuffers(a);glBindFramebuffer(GL_FRAMEBUFFER, 0);

	// Pass the texture m_FboTextureId to the main thread to render on screen
    m_GLEnv->renderDone(m_GLEnv->callbackCtx, m_FboTextureId);
    m_FrameIndex++;
}
Copy the code

Back to the main render thread, Init passes the textures, program, VBO resources, and EGLContext generated by the main render to the new thread.

m_GLEnv.sharedCtx     = eglGetCurrentContext(a); m_GLEnv.program = m_FboProgramObj; m_GLEnv.inputTexId = m_ImageTextureId; m_GLEnv.vboIds[0]     = m_VboIds[0];
m_GLEnv.vboIds[1]     = m_VboIds[2];
m_GLEnv.vboIds[2]     = m_VboIds[3];
m_GLEnv.imgSize       = imgSize;
m_GLEnv.renderDone    = OnAsyncRenderDone;// The main thread callback function
m_GLEnv.callbackCtx   = this;

// Send the shared resource to the new thread
GLRenderLooper::GetInstance() - >postMessage(MSG_SurfaceCreated, &m_GLEnv);

GLRenderLooper::GetInstance() - >postMessage(MSG_SurfaceChanged, m_RenderImage.width, m_RenderImage.height);
Copy the code

When the main thread renders, it first sends the render command to the new thread and then waits for it to finish rendering. When the new thread finishes rendering, it calls the OnAsyncRenderDone function to tell the main thread to render on screen.

void SharedEGLContextSample::Draw(int screenW, int screenH)
{{// Send render instructions to the new thread and wait for it to finish rendering
		unique_lock<mutex> lock(m_Mutex);
		GLRenderLooper::GetInstance() - >postMessage(MSG_DrawFrame);
		m_Cond.wait(lock);
	}

	// Render the main thread to the upper screen
	glViewport(0.0, screenW, screenH);
	glUseProgram(m_ProgramObj);
	GO_CHECK_GL_ERROR(a);glBindVertexArray(m_VaoId);
	glActiveTexture(GL_TEXTURE0);
	glBindTexture(GL_TEXTURE_2D, m_FboTextureId);
	GLUtils::setInt(m_ProgramObj, "s_TextureMap".0);
	GO_CHECK_GL_ERROR(a);glDrawElements(GL_TRIANGLES, 6, GL_UNSIGNED_SHORT, (const void *)0);
	GO_CHECK_GL_ERROR(a);glBindTexture(GL_TEXTURE_2D, GL_NONE);
	glBindVertexArray(GL_NONE);

}

void SharedEGLContextSample::OnAsyncRenderDone(void *callback, int fboTexId) {
	// When the new thread finishes rendering, it calls OnAsyncRenderDone to notify the main thread to render
	SharedEGLContextSample *ctx = static_cast<SharedEGLContextSample *>(callback);
	unique_lock<mutex> lock(ctx->m_Mutex);
	ctx->m_FboTextureId = fboTexId;
	ctx->m_Cond.notify_all(a); }Copy the code

One final note: Multi-threaded rendering ensures that shared resources such as textures are not accessed at the same time, which can result in rendering errors.

For the full code, refer to the following project and select multi-thread Render:

https://github.com/githubhaohao/NDK_OpenGLES_3_0
Copy the code

Technical communication

Technical exchange/access to the source code can be added to my wechat: byt-flow

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

OpenGL ES shared context for multi-threaded rendering

EGL concept review

Which resources can be shared when the context is shared

Shared context multi-threaded rendering

Shared context

Multi-threaded rendering

Technical communication

OpenGL ES shared context for multi-threaded rendering

EGL concept review

Which resources can be shared when the context is shared

Shared context multi-threaded rendering

Shared context

Multi-threaded rendering

Technical communication

Related Posts

Android MVVM Architecture Guide (ii) : DataBinding

Design Pattern – Command Pattern

Flutter will strike back! 1.2 version released, or will be unified rivers and lakes