The original article was first published on the wechat official account Byteflow

How to transfer a large array to a shader program?

In OpenGL ES graphic image processing, we often encounter a situation: pass a large array to the shader program.

There are three methods commonly used at present:

  • Load an array into a 2D texture and use texelFetch to fetch the data.
  • Use uniform buffer objects, known as UBO;
  • Use the texture buffer object, TBO.

Load the array into the texture

The easiest way to transfer a large array is by loading it into a texture.

To accurately exchange the value of each pixel, the sampling function texture should not be used at this time, because the sampling function involves complex operations such as normalization, filtering and interpolation, so it is almost impossible to get the value of an exact pixel.

TexlFetch is an API introduced in OpenGL ES 3.0. TexlFetch treats the texture as an image and can access the content of pixels precisely. We can get the value of an element in an array by analogy with an index.

vec4 texelFetch(sampler2D sampler, ivec2 P, int lod);

vec4 texelFetch(sampler3D sampler, ivec3 P, int lod);

vec4 texelFetch(samplerBuffer sampler, int P);
Copy the code

TexelFetch uses unnormalized coordinates to access the texture elements directly without any filtering or interpolation operations. The range of coordinates is the width and height of the actual loaded texture image.

TexelFetch is easy to use. In fragment shaders, the following 2 lines of code are interchangeable, but the final rendering results are slightly different. Why is this slightly different? You taste, you fine taste!

gl_FragColor = texture(s_Texture, v_texCoord);
gl_FragColor = texelFetch(s_Texture,  ivec2(int(v_texCoord.x * imgWidth), int(v_texCoord.y * imgHeight)), 0);

Copy the code

Use a Uniform buffer object

We often use uniform variables to pass some vectors to the shader program to participate in the rendering operation.

However, OpenGL ES has a limit on the number of UNIFORM variables that can be used. We can use the glGetIntegerv function to get the maximum number of UNIFORM variables that can be used.

int maxVertexUniform, maxFragmentUniform;
glGetIntegerv(GL_MAX_VERTEX_UNIFORM_COMPONENTS, &maxVertexUniform);
glGetIntegerv(GL_MAX_FRAGMENT_UNIFORM_COMPONENTS, &maxFragmentUniform);
Copy the code

At present, mainstream mobile phones generally support 1024 vectors of Uniform type. Using large arrays is easy to break this limit. Moreover, uniform variables are also difficult to manage, and you need to set uniform variables again and again.

So how do you break the uniform variable limit?

The answer is to use UBO (Uniform Buffer Object).

UBO, as the name suggests, is a buffer object that holds uniform variable data. It is essentially the same as other buffer objects in OpenGL ES, and it is created in much the same way.

When data is loaded into UBO, it is stored on UBO instead of being handed to the shader program, so it does not occupy the uniform storage space of the shader program itself. UBO is a new way of transferring data from memory to video memory, and UBO generally needs to be used with UNIFORM blocks.

This example sets the MVP transformation matrix to a Uniform block, meaning that the UBO we will create later will hold three matrices.

#version 310 es
layout(location = 0) in vec4 a_position;
layout(location = 1) in vec2 a_texCoord;

layout (std140) uniform MVPMatrix
{
    mat4 projection;
    mat4 view;
    mat4 model;
};

out vec2 v_texCoord;
void main(a)
{
    gl_Position = projection * view * model * a_position;
    v_texCoord = a_texCoord;
}
Copy the code

Set the uniform block binding point to 0 to generate a UBO.

GLuint uniformBlockIndex = glGetUniformBlockIndex(m_ProgramObj, "MVPMatrix");
glUniformBlockBinding(m_ProgramObj, uniformBlockIndex, 0);


glGenBuffers(1, &m_UboId);
glBindBuffer(GL_UNIFORM_BUFFER, m_UboId);
glBufferData(GL_UNIFORM_BUFFER, 3 * sizeof(glm::mat4), nullptr, GL_STATIC_DRAW);
glBindBuffer(GL_UNIFORM_BUFFER, 0);

// Define the range of buffer whose binding point is 0
glBindBufferRange(GL_UNIFORM_BUFFER, 0, m_UboId, 0.3 * sizeof(glm::mat4));
Copy the code

Update the Uniform Buffer data when drawing, update the data of the three matrices, pay attention to the offset.

glBindBuffer(GL_UNIFORM_BUFFER, m_UboId);
glBufferSubData(GL_UNIFORM_BUFFER, 0, sizeof(glm::mat4), &m_ProjectionMatrix[0][0]);
glBufferSubData(GL_UNIFORM_BUFFER, sizeof(glm::mat4), sizeof(glm::mat4), &m_ViewMatrix[0][0]);
glBufferSubData(GL_UNIFORM_BUFFER, 2 *sizeof(glm::mat4), sizeof(glm::mat4), &m_ModelMatrix[0][0]);
glBindBuffer(GL_UNIFORM_BUFFER, 0);
Copy the code

Use the texture buffer object

The Texture Buffer Object (TBO) was introduced in OpenGL ES 3.2, so check the OpenGL ES version first and ensure that the Android API >= 24.

TBO needs to be used in conjunction with Buffer Texture, which is a one-dimensional Texture that stores data from a Texture Buffer object (TBO) to allow shaders to access large memory tables managed by the Buffer object.

In GLSL, only the texelFetch function can be used to access the buffer texture, and the sampler type of the buffer texture is samplerBuffer.

A TBO is generated in the same way as a VBO, just by binding to GL_TEXTURE_BUFFER, and the buffer texture is generated in the same way as a normal 2D texture.

// Generate a Buffer Texture glGenTextures(1, &m_tbotexId); float *bigData = new float[BIG_DATA_SIZE]; for (int i = 0; i < BIG_DATA_SIZE; ++ I) {bigData[I] = I * 1.0f; } // Generate a TBO and upload a large array of buffers to TBO glGenBuffers(1, &m_tBOID); glBindBuffer(GL_TEXTURE_BUFFER, m_TboId); glBufferData(GL_TEXTURE_BUFFER, sizeof(float) * BIG_DATA_SIZE, bigData, GL_STATIC_DRAW); delete [] bigData;Copy the code

To use the fragment shader for the texture buffer, you need to introduce the extension texture Buffer, note that the version is declared as #version 320 ES.

#version 320 es #extension GL_EXT_texture_buffer : require in mediump vec2 v_texCoord; layout(location = 0) out mediump vec4 outColor; uniform mediump samplerBuffer u_buffer_tex; uniform mediump sampler2D u_2d_texture; uniform mediump int u_BufferSize; Void main() {mediump int index = int((v_texcoord.x + v_texcoord.y) /2.0 * float(u_BufferSize - 1)); mediump float value = texelFetch(u_buffer_tex, index).x; Mediump vec4 lightColor = vec4(vec3(value/float(u_buffersize-1)), 0.0), 1.0); outColor = texture(u_2d_texture, v_texCoord) * lightColor; }Copy the code

How to use buffer textures and TBO when drawing?

glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_BUFFER, m_TboTexId);
glTexBuffer(GL_TEXTURE_BUFFER, GL_R32F, m_TboId);
GLUtils::setInt(m_ProgramObj, "u_buffer_tex", 0);
Copy the code

The texture is used in much the same way as a normal texture, except that glTexBuffer is used to bind TBO to the buffer texture.

In this example, we normalize the value result by taking the value of buffer texture, which ranges from 0 to size-1, as the sampling result of illumination color superimposed on 2D texture.

The effect, as shown in the image above, is that the texture coordinates increase color intensity from the top left to the bottom right.

reference

www.khronos.org/opengl/wiki… www.khronos.org/registry/Op…