OpenGL (Open Graphics Library) is a professional Graphics programming interface that defines cross-programming language and cross-platform programming interface specifications. It can be used for 3d, 2d graphics image rendering, is a powerful, easy to call the bottom graphics library. In an RTC application, because of the need of video rendering or algorithm processing, OpenGL is an efficient way to achieve rendering or processing. The efficient implementation of OpenGL is supported on Windows, Linux, and macOS. OpenGL ES (OpenGL for Embedded Systems) is available on both iOS and Android. It is very convenient to use OpenGL for cross-platform development. This article will share how to use OpenGL to achieve efficient cross-platform application rendering.

Because of the different hardware and software implementation, OpenGL supports different capabilities and specifications on different platforms.

OpenGL ES supported by mobile platforms is a subset of OpenGL 3D graphics API, which is designed for mobile phones and other embedded devices. Therefore, for cross-platform development compatible with mobile platforms, choose apis supported by OpenGL ES as much as possible.

Early versions of OpenGL and OpenGL ES were for fixed pipelines, with support for shading language and programmable pipelines starting with 2.0. Programmable pipeline apis are more flexible to use and support richer features. Most hardware now supports programmable pipelines, and the fixed pipeline API has been officially recommended to be deprecated and removed in older versions. Therefore, it is recommended to use a coloring language and a programmable pipeline API and support at least version 2.0. If the device or system you want to support is newer, you can use version 3.0 or later.

Before an application calls any OpenGL function, it first needs to create an OpenGL Context. This OpenGL context can be understood as a very large state machine, which holds various states in OpenGL, which is the basis for OpenGL to execute various instructions. To some extent, OpenGL functions operate on an object or state in a state machine, its context. Because OpenGL context is a large state machine, switching context is often expensive. Different drawing modules require different state and object management. So you can create multiple different contexts in your application, use different contexts in different threads, and share buffers, textures, and other resources between contexts. To improve processing efficiency, you should avoid repeatedly switching context and changing a large number of states.

Although OpenGL is cross-platform, the implementation and environment construction of OpenGL vary from platform to platform. The establishment and switching of OpenGL Context varies from platform to platform during development. Developers can use open source frameworks such as GLFW and GLUT to help build and manage OpenGL context, or they can use native apis of each platform to complete the task. This article focuses on using native apis for each platform. The following describes the OpenGL environment and context creation for each platform.

A, Windows,

Windows platform Because of Microsoft’s Direct3D, Microsoft’s support for OpenGL has been lukewarm. OpenGL is still supported in most Microsoft operating systems in versions 1.0 and 1.1, and only supports fixed pipeline apis, which are not friendly to modern applications developed using OpenGL. However, OpenGL’s ARB extension mechanism allows us to access OpenGL’s advanced feature interface. The Windows OpenGL implementation provides a function called wglGetProcAddress that allows us to retrieve a pointer to an OpenGL function provided by the driver. But there is a faster way. Through GLEW (GL Extension Wrangler) library to complete this tedious retrieval process. Just import the header glew.h and the Glew library and call glewInit() at the beginning of the application, after which all Pointers to OpenGL’s extensions and core features above 1.1 will be set automatically.

The call to glewInit requires the creation of an OpenGL context, which is then deleted after initialization. Then create a new OpenGL context that supports the WGL_ARB extension. Example code is as follows:

/* Register window class/WNDCLASSEX wcex; ZeroMemory(&wcex, sizeof(WNDCLASSEX)); wcex.cbSize = sizeof(WNDCLASSEX); wcex.hInstance = GetModuleHandle(NULL); wcex.lpfnWndProc = GLEW_WindowProc; wcex.lpszClassName = kszGlewInitClassName; / create a window to initialize glew/HWND HWND = CreateWindow (kszGlewInitClassName, “”, L (WS_POPUP | WS_DISABLED), CW_USEDEFAULT, CW_USEDEFAULT, 10, 10,NULL, NULL, GetModuleHandle(NULL), NULL); / Set the pixel format/HDC HDC = GetDC(HWND); PIXELFORMATDESCRIPTOR pfd; memset(&pfd, 0, sizeof(PIXELFORMATDESCRIPTOR)); pfd.nSize = sizeof(PIXELFORMATDESCRIPTOR); pfd.nVersion = 1; pfd.dwFlags=PFD_DOUBLEBUFFER | PFD_SUPPORT_OPENGL |PFD_DRAW_TO_WINDOW; pfd.iPixelType = PFD_TYPE_RGBA; pfd.cColorBits = 24; pfd.cDepthBits = 32; pfd.cStencilBits = 8; pfd.iLayerType = PFD_MAIN_PLANE; int32_t nPixelFormat = ChoosePixelFormat(hdc,&pfd); BOOL bRet = SetPixelFormat(hdc, nPixelFormat,&pfd); / Create OpenGL context and set to the current context */ HGLRC HGLRC = wglCreateContext(HDC); wglMakeCurrent(hdc, hglrc); /* Initialize glew/if (glewInit()) {printf(“glewInitfailed\n”); / / handleinit fail… } / release the current OpenGL context */ wglMakeCurrent(NULL, NULL); wglDeleteContext(hglrc); ReleaseDC(hwnd, hdc); DestroyWindow(hwnd); UnregisterClass(kszGlewInitClassName, NULL); /* Start selecting the real format and create the appropriate OpenGL context */ /* Create the window again /… / int32_t pixAttribs[] = {WGL_DRAW_TO_WINDOW_ARB, GL_TRUE, // Draw pixels in window format WGL_SUPPORT_OPENGL_ARB, GL_TRUE, // Support OpenGL rendering WGL_DOUBLE_BUFFER_ARB, GL_TRUE, WGL_PIXEL_TYPE_ARB, WGL_TYPE_RGBA_ARB, // The pixel format is RGL_acceleration_arb, WGL_FULL_ACCELERATION_ARB, WGL_DEPTH_BITS_ARB,24, WGL_STENCIL_BITS_ARB, 8, WGL_STENCIL_BITS_ARB MSAA WGL_SAMPLES_ARB, 4, // 4 times MSAA 0}; Uint32_t pixCount = 0; // uint32_t pixCount = 0; int32_t nPixelFormat = 0; wglChoosePixelFormatARB(hdc,&pixAttribs[0],NULL,1, &nPixelFormat, &pixCount); if (nPixelFormat ! SetPixelFormat(HDC, nPixelFormat, & PFD); / Create the corresponding OpenGL context */ int32_TATtribs [] = {WGL_CONTEXT_MAJOR_VERSION_ARB, 3, WGL_CONTEXT_MINOR_VERSION_ARB, 3, WGL_CONTEXT_FLAGS_ARB, 0, 0}; HGLRCwglrc = wglCreateContextAttribsARB(hdc, 0, attribs); if(wglrc) { wglMakeCurrent(hdc, wglrc); hglrc_ = wglrc; } else { printf(“wglCreateContextAttribsARBfailed\n”); / / handle failed… } } else { printf(“ChoosePixelFormatfailed\n”); / / handlefailed… } /* Query and print OpenGL */ const GLubyte *GLVersionString = glGetString(GL_VERSION); int32_t OpenGLVersion[2]; glGetIntegerv(GL_MAJOR_VERSION, &OpenGLVersion[0]); glGetIntegerv(GL_MINOR_VERSION,&OpenGLVersion[1]); printf(“OpenGLversion %d.%d\n”, OpenGLVersion[0], OpenGLVersion[1]);

Second, the macOS

MacOS provides glut, NSOpenGL, CGL and other interfaces to create and manage OpenGL environments. This article uses NSOpenGL as an example.

NSOpenGLView is a lightweight NSView subclass package that provides easy creation and management of OpenGL drawing environment. NSOpenGLPixelFormat and NSOpenGLContext objects are maintained internally, which can be easily accessed and managed by users.

Create an NSOpenGLView by setting the NSOpenGLPixel FormatAttribute to create an NSOpenGLPixelFormat object. Create an NSOpenGLView using NS OpenGLPixelFormat.

The static NSOpenGLPixelFormatAttribute kDefaultAttributes [] = {NSOpenGLPFADoubleBuffer, / / double buffer NSOpenGLPFADepthSize, 24, / / the depth buffer depth NSOpenGLPFAStencilSize, 8, / / stencil buffer depth NSOpenGLPFAMultisample, / / multiple sampling NSOpenGLPFASampleBuffers, (NSOpenGLPixelFormatAttribute) 1, / / multiple sampling buffer NSOpenGLPFASamples, (NSOpenGLPixelFormatAttribute) 4, / / multiple hits NSOpenGLPFAOpenGLProfile NSOpenGLProfileVersion3_2Core, / / OpenGL3.2 0}; NSOpenGLPixelFormat* pixelFormat = [[NSOpenGLPixelFormatalloc] initWithAttributes: kDefaultAttributes]; NSOpenGLView*renderView=[[NSOpenGLView alloc]initWithFrame:frameRect pixelFormat:pixelFormat]; Developers can subclass the NSOpenGLView implementation by implementing -(void)prepareOpenGL; And – (void) clearGLContext; Customize the behavior of OpenGL Context initialization and deletion. The specific drawing operation can be done at -(void)drawRect: (NSRect) bounds; In the implementation. It is important to note that in order to ensure the Retina screen display effect, can set the attribute of NSOpenGLView wantsBestResolutionOpenGLSurface to YES.

  • (void)prepareOpenGL { [super prepareOpenGL]; [self ensureGLContext]; / / setupOpenGL resources…

}

  • (void)clearGLContext { [self ensureGLContext]; // cleanup OpenGL resources…… [super clearGLContext];

} -(void) drawRect: (NSRect) bounds { [self ensureGLContext]; / / the draw OpenGL… [super drawRect:rect]; }

  • (void)ensureGLContext { NSOpenGLContext*context = [self openGLContext]; if([NSOpenGLContext currentContext] ! = context) { [context makeCurrentContext]; }

}

Third, the iOS

IOS support for OpenGL ES 3.0 began in September 2013 with iOS 7 and new devices released around the same time, and Apple has released 64-bit devices since then. With the exception of a few early iOS devices, OpenGL ES 3.0 is now available on most iOS devices.

Similar to macOS, iOS also provides a wrapped UIView — GLKView, which developers can easily use to draw OpenGL ES. OpenGL ES frameBuffer objects are also provided for off-screen rendering or CALayer layer drawing based on CAEAGLLayer. This article takes GLKView as an example. EAGLContext glContext = [[EAGLContext alloc]initWithAPI: kEAGLRenderingAPIOpenGLES3]; if (! GlContext) {/ / OpenGL ES 3 create failure, create OpenGL ES 2 glContext = [[EAGLContext alloc] initWithAPI: kEAGLRenderingAPIOpenGLES2]; } GLKViewglkView=[[GLKView alloc] initWithFrame:frameRect context:glContext]; / / configuration renderbuffers glkView. DrawableColorFormat = GLKViewDrawableColorFormatRGBA8888; glkView.drawableDepthFormat = GLKViewDrawableDepthFormat24; glkView.drawableStencilFormat = GLKViewDrawableStencilFormat8; glkView.drawableMultisample = GLKViewDrawableMultisample4X; glkView.delegate = self; GLKViewDelegate -(void)glkView:(glkView *)view drawInRect:(CGRect)rect; implementation

  • (void)glkView:(GLKView*)viewdrawInRect:(CGRect)rect { if ([EAGLContext currentContext] ! = glContext_) { [EAGLContext setCurrentContext:glContext_]; } // draw OpenGL…

}

Fourth, the Android

Android has also been supporting OpenGL ES 3 since Android 4.3 launched in 2013, but it’s not easy to tell which devices will support OpenGL ES 3 compared to the closed iOS ecosystem. This can be determined by the success of creating OpenGL ES 3 context.

Android also has a similar encapsulated OpenGL View with iOS GLKView — GLSurfaceView, developers can directly create and manage OpenGL ES context through the GLSurfaceView method. You can also create your own OpenGL context and render View based on SurfaceView and EGL interfaces. Here is a method based on EGL Native interface (using EGL’s Java interface, the call process is the same).

EGL is a layer of interface between graphics rendering apis (such as OpenGL ES) and local platform windowing systems, which ensures the platform independence of OpenGL ES. It provides management functions such as creating rendering surfaces, rendering contexts, synchronizing application and platform rendering apis, display device access, and rendering configuration. Egl-based context creation consists of four steps: initializing, selecting and setting the appropriate configuration, creating a surface, and creating a context.

1. EGL initialization

EGLBoolean success = EGL_FALSE; EGLint err = 0; EGLDisplay display = eglGetDisplay(EGL_DEFAULT_DISPLAY); if (EGL_NO_DISPLAY == display_) { printf(“eglGetDisplay error %d\n”,eglGetError()); return; } EGLint major=0, minor=0; if (EGL_FALSE == eglInitialize(display, &major, &minor);) { printf(“eglInitialize error %d\n”, eglGetError()); return; }

2. Select and set an appropriate configuration

const EGLint configAttribs[] = { EGL_RED_SIZE,8, EGL_GREEN_SIZE,8, EGL_BLUE_SIZE,8, EGL_ALPHA_SIZE, 8, EGL_STENCIL_SIZE, 8, EGL_SAMPLE_BUFFERS, 1, EGL_SAMPLES,4, EGL_RENDERABLE_TYPE, EGL_OPENGL_ES2_BIT, EGL_SURFACE_TYPE, EGL_WINDOW_BIT, // Window based surface EGL_NONE}; EGLConfig config; EGLint numConfigs; if (EGL_FALSE == eglChooseConfig(display_,configAttribs, &config, 1, &numconfigs)) {// handle andselect other configs…… }

3. Create surfaces

EGLint format; if (! eglGetConfigAttrib(display_, config,EGL_NATIVE_VISUAL_ID, &format)) { printf(“eglGetConfigAttrib error %d\n”, eglGetError()); return; } ANativeWindow *window; // ANativeWindow can be the Surface object ANativeWindow_setBuffersGeometry(window, 0, 0, format) obtained in the SurfaceView; EGLSurface surface = eglCreateWindowSurface(display, config, window, NULL); if (! surface_) { printf(“eglCreateWindowSurface error %d\n”, eglGetError()); return; }

4. Create a context

EGLint contextAttribs[] = { EGL_CONTEXT_CLIENT_VERSION, 3, EGL_NONE }; EGLContext context = eglCreateContext(display, config,EGL_NO_CONTEXT, contextAttribs); if (context == EGL_NO_CONTEXT) { printf(“eglCreateContext create OpenGL ES 3 contextfailed\n”); EGLintcontextAttribs2[] = { EGL_CONTEXT_CLIENT_VERSION, 2, EGL_NONE }; context = eglCreateContext(display_,config, NULL, contextAttribs2); if (context_== EGL_NO_CONTEXT) { printf(“eglCreateContextcreate OpenGL ES 2 context failed\n”); return; }} eglMakeCurrent(display, surface, surface, context) {} eglMakeCurrent(display, surface, surface, context) {} eglMakeCurrent(display, surface, surface, context) {} eglMakeCurrent(display, surface, surface, context) {} eglMakeCurrent(display, surface, surface, context)} In addition to Windows support for OpenGL is relatively weak, need to rely on third-party libraries to facilitate the use of. Other platforms can set up OpenGL contexts relatively quickly, and even have wrapped Views to help developers access them quickly. However, OpenGL developed earlier, has not adapted to the development of modern GPU graphics technology, encountered some problems: such as modern GPU rendering pipeline has changed, does not support multi-threaded operation, does not support asynchronous processing and so on.

A new generation of graphics API Vulkan may replace OpenGL in the future. Vulkan will significantly reduce the overhead of drawing commands, send multithreading performance, and render performance is faster. Google has also confirmed that Android will support Vulkan. The philosophy behind Microsoft’s DirectX12 is the same as Vulkan. Apple inc. launched its own Metal API in 2014, aiming to replace OpenGL and adapt to modern GPU technology. Its instruction overhead and rendering performance have also been greatly improved. In 2018, Apple announced that OpenGL and OpenGL ES apis are deprecated from macOS 10.14 and iOS 12 and will not be maintained in the future. Developers will migrate to new graphics apis in the future, but OpenGL has been the dominant graphics API for more than 20 years, and its eventual demise is surely a long way off.

Founded in 2019, Beileyun is the first real-time interactive communication cloud service provider with video conferencing background in China, bringing together a large number of senior technical experts focusing on audio, video, whiteboard, network, AI and other fields. Through the integration of Pano SDK, enterprise developers can quickly realize interactive small class, super small class, double normal university class, voice chat room, video social networking, live broadcast, game voice, video customer service, telemedicine, office collaboration and other scenes worldwide.

Pay attention to Patleyun Pano, we will share more technical exploration and practical experience with you, grow up with you and charge.