The problem

Recently I was debugging a problem with OpenGL ES and found that there were too many temporary textures.

When developing iOS apps, Xcode comes with tools to measure memory usage and leaks. But how do you measure how much memory OpenGL ES textures occupy, and whether they leak? GL lives in another world, not CPU and memory, but GPU and video memory.

I searched the web and also experimented with the measurement module with Instruments. Unfortunately, I can’t find a way to measure video memory occupancy directly, but there seems to be an indirect way, and I’m not quite sure, so I have to use the vague word “seems”.

record

The following is the record of my test process, using iPhone 7+ test, iOS system 11.3.1.

Xcode comes with Memory measurements, and this graph should be familiar.

Memory

The question naturally arises, does this Memory Use include texture graphics? Write code to experiment.

local tex = context:createTexture(1024, 1024)
print(tex)Copy the code

We’ve exported a bunch of Lua apis to make it easier to write. The above Lua code creates a texture that is not released and runs once per frame. The texture is 1024 * 1024, RGBA format, so it occupies 4M. If Memory Use counts texture Memory, you should see the value go up very quickly.

The actual result is that the value of Memory Use increases slowly.

Does that prove that Memory Use does not include texturing? It’s too early to say. If it leaks 4M each time, the App should crash quickly or appear some abnormality according to reason. But I noticed that the App always worked fine, whereas Windows 7 ran the code above and crashed the App quickly.

So guess 1: create a certain amount of textures on iOS, and when the limit is reached, the subsequent texture creation will fail, and no more video memory will be leaked, so the App will not crash.

Based on the above speculation, add detection to the corresponding C++ code:

glTexImage2D(d._target, 0, format, width, height, 0, format, GL_UNSIGNED_BYTE, OF_NULL);
GL_CHECK_ERROR();Copy the code

GL_CHECK_ERROR is defined as

#if GL_DEBUG #define GL_CHECK_ERROR() gl::checkGLError(__FILE__, __LINE__); #else #define GL_CHECK_ERROR() #endif void checkGLError(const char* file, int line) { GLenum error = glGetError(); if (error) { // xxxx printf("%d\n", (int)error); assert(false); }}Copy the code

GL_CHECK_ERROR fires assert if glTexImage2D fails to create the texture.

Continuing the test, it turns out that the App has been running steadily and without triggering assert, which means that glTexImage2D was successful. But notice that glTexImage2D passes OF_NULL at the end of the argument.

So guess 2: if the last parameter of glTexImage2D is OF_NULL, iOS will delay the allocation of video memory, which will consume less resources. Thus the texture was created successfully and assert was not triggered.

Based on guess 2, modify the lua code to:

local tex = context:createTexture(1024, 1024)
context:copyTexture(inTex, tex:toOFTexture())
context:copyTexture(tex:toOFTexture(), outTex)Copy the code

This code forces the actual use of Tex to make the system actually allocate video memory. And the real result is,

  1. After running for some time, it firesGL_CHECK_ERRORAssert, which confirms that glTexImage2D failed to create a texture.
  2. If you comment out assert, the App can continue to run without crashing, but with a splintered screen.
  3. Even after the App has run out of screens, Memory Use continues to grow slowly and the display number is 67.8 MB.
Flowers screen

Thus we can confirm that Memory Use does not include textured video Memory.

VM Tracker

I continued to experiment with Instruments one by one. Allocations, Leaks tool is for memory only. GPU Hardware can count the execution time of Framet Shader or Vertex Shader. Graphics Driver Activity can count Shader compilation time, some Driver events. But none of them support measuring video memory.

Finally, there is a tool VM Tracker, which can count Virtual Size.

VM Tracker

Notice the IOKit (IOKit is a driver module). If the texture is assigned and not released, the Virtual Size of IOKit will continue to grow. When the value is about 2.4g, iPhone 7+ will start to display the screen. However, if the texture is released correctly, IOKit’s Virtual Size will be stable. Thus IOKit’s Virtual Size can be used as an indirect measure of the texture.

conclusion

  1. Xcode comes with a Memory measurement tool, Memory Use numbers do not include texture Memory.
  2. In iOS 11.3.1, iPhone 7+ (not tested in other versions and models) glTexImage2D final parameter OF_NULL, iOS system will be optimized to delay the allocation of video memory.
  3. After creating a certain number of textures, the limit is reached, and the texture creation fails. The App might not crash, but it might have some anomalies, like a splintered screen.
  4. Video memory may be mapped to a virtual memory. The Virtual Size of IOKit for the VM Tracker tool can be used as an indirect measure of texture memory. If it keeps growing, it is likely to leak GPU resources such as textures.