Text rendering is a frequently used feature in audio and video or OpenGL development, such as making cool captions, watermarking videos, setting special fonts, and so on.

OpenGL doesn’t actually define how to render text, so the best way we can do this is to upload the image with text to the texture and then texture map it.

This article introduces the common text rendering methods in the application layer and C++ layer respectively.

Generate Bitmap based on Canvas drawing

Text rendering in the application layer mainly uses Canvas to draw the text into a Bitmap, and then generates a small map, and then in the rendering time to map.

In a production environment, this small image would be converted to grayscale to reduce unnecessary data copy and memory usage, and then the grayscale could be colored as font color during rendering.

// Create a bitmap
Bitmap bitmap = Bitmap.createBitmap(width, hight, Bitmap.Config.ARGB_8888);
// Initialize the image drawn by the canvas to the bitmap
Canvas canvas = new Canvas(bitmap);
// Create the brush
Paint paint = new Paint(); 
// Obtain clearer image sampling, anti-jitter
paint.setDither(true); 
paint.setFilterBitmap(true);
// Draw text to bitmap
canvas.drawText text, x, y,paint);
Copy the code

Then generate the texture and upload the Bitmap to the texture.

int[] textureIds = new int[1];
// Create a texture
GLES20.glGenTextures(1, textureIds, 0);
mTexId = textureIds[0];
// Bind the texture
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, mTexId);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_REPEAT);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_REPEAT);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);

ByteBuffer bitmapBuffer = ByteBuffer.allocate(bitmap.getHeight() * bitmap.getWidth() * 4);//RGBA
bitmap.copyPixelsToBuffer(bitmapBuffer);
bitmapBuffer.flip();

// Set the memory size bound to memory address
GLES20.glTexImage2D(GLES20.GL_TEXTURE_2D, 0, GLES20.GL_RGBA, mWatermarkBitmap.getWidth(), mWatermarkBitmap.getHeight(),
        0, GLES20.GL_RGBA, GLES20.GL_UNSIGNED_BYTE, bitmapBuffer);

// Unbind the texture
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, 0);
Copy the code

Finally, map the texture with text to the corresponding location (texture map).

FreeType

FreeType is an open source library for text rendering based on C language. It is small, efficient, and highly customizable. It is mainly used for loading fonts and rendering them into place.

FreeType is also a popular cross-platform font library for Android, iOS, Linux, and other operating systems. TrueType fonts are not defined in pixels or other non-scalable ways, but rather by mathematical formulas (combinations of curves). These glyphs, similar to vector images, can be used to generate pixel images based on the font size you need.

FreeType website address:

https://www.freetype.org/
Copy the code

FreeType compilation

This section describes how to use the NDK to compile the FreeType library for the Android platform. First, download the latest version of the FreeType source code from the official website, then create a new jNI folder, put the source code into the JNI folder, the directory structure is as follows:

Create build files android. mk and application.mk.

Android.mk refers to Google’s build script:

LOCAL_PATH:= $(call my-dir)

include $(CLEAR_VARS)LOCAL_SRC_FILES := \ ./src/autofit/autofit.c \ ./src/base/ftbase.c \ ./src/base/ftbbox.c \ ./src/base/ftbdf.c \ ./src/base/ftbitmap.c \ ./src/base/ftcid.c \ ./src/base/ftdebug.c \ ./src/base/ftfstype.c \ ./src/base/ftgasp.c \ ./src/base/ftglyph.c \ ./src/base/ftgxval.c \ ./src/base/ftinit.c \ ./src/base/ftlcdfil.c \ ./src/base/ftmm.c \ ./src/base/ftotval.c \ ./src/base/ftpatent.c \ ./src/base/ftpfr.c \ ./src/base/ftstroke.c \ ./src/base/ftsynth.c \ ./src/base/ftsystem.c \ ./src/base/fttype1.c \ ./src/base/ftwinfnt.c \ ./src/bdf/bdf.c \ ./src/bzip2/ftbzip2.c \ ./src/cache/ftcache.c \ ./src/cff/cff.c \ ./src/cid/type1cid.c \ ./src/gzip/ftgzip.c \ ./src/lzw/ftlzw.c \ ./src/pcf/pcf.c \ ./src/pfr/pfr.c \ ./src/psaux/psaux.c \ ./src/pshinter/pshinter.c \ ./src/psnames/psmodule.c \ ./src/raster/raster.c \ ./src/sfnt/sfnt.c \ ./src/smooth/smooth.c \ ./src/tools/apinames.c \ ./src/truetype/truetype.c \  ./src/type1/type1.c \ ./src/type42/type42.c \ ./src/winfonts/winfnt.c LOCAL_C_INCLUDES +=$(LOCAL_PATH)/include

LOCAL_CFLAGS += -W -Wall
LOCAL_CFLAGS += -fPIC -DPIC
LOCAL_CFLAGS += "-DDARWIN_NO_CARBON"
LOCAL_CFLAGS += "-DFT2_BUILD_LIBRARY"

LOCAL_CFLAGS += -O2

LOCAL_MODULE:= freetype

include $(BUILD_STATIC_LIBRARY)
# https://android.googlesource.com/platform/external/freetype/+/android-6.0.1_r28/Android.mk
Copy the code

Application.mk:

APP_OPTIM := release APP_CPPFLAGS := -std=c++14 -frtti NDK_TOOLCHAIN_VERSION := clang APP_PLATFORM := android-28 APP_STL  := c++_static APP_ABI := arm64-v8a,armeabi-v7aCopy the code

Finally, run the ndk-build command in the jNI directory. If you do not want to compile, you can also directly go to the following project to fetch the ready-made static library:

https://github.com/githubhaohao/NDK_OpenGLES_3_0
Copy the code

OpenGL uses FreeType to render text

The use of the FreeType

Importing header files:

#include "ft2build.h"
#include <freetype/ftglyph.h>
Copy the code

Then to load a font, all we need to do is initialize FreeType and load the font into something FreeType calls Face. Here I found a font file antonio-regular. TTF in Windows and put it under sdCard for FreeType to load.


FT_Library ft;

if (FT_Init_FreeType(&ft))
	LOGCATE("TextRenderSample::LoadFacesByASCII FREETYPE: Could not init FreeType Library");


FT_Face face;
if (FT_New_Face(ft, "/sdcard/fonts/Antonio-Regular.ttf".0, &face))
	LOGCATE("TextRenderSample::LoadFacesByASCII FREETYPE: Failed to load font");


FT_Set_Pixel_Sizes(face, 0.96);
Copy the code

In the code fragment, FT_Set_Pixel_Sizes is used to set the size of the text. This function sets the width and height of the font face. Setting the width value to 0 means that we want to dynamically calculate the width of the font from the height given by the font face.

Face in a word Face contains a collection of all glyphs, and we can invoke the current glyphs to be represented by calling FT_Load_Char. Here we select in the load letter glyph ‘A’ :

if (FT_Load_Char(face, 'A', FT_LOAD_RENDER))
    std::cout << "ERROR::FREETYTPE: Failed to load Glyph" << std::endl;  
Copy the code

By setting FT_LOAD_RENDER to a load identifier, we tell FreeType to create an 8-bit grayscale bitmap, which we can obtain using face->glyph->bitmap.

Font bitmaps loaded with FreeType do not hold the same size as we do with bitmap fonts. The size of a glyph bitmap produced using FreeType is exactly the size that will contain the glyph. For example, production bitmaps for ‘.’ are much smaller in size than those for ‘A’.

As a result, FreeType also produces several metrics to describe the size and location of the generated glyph bitmap when it is loaded. The following figure shows what all the measurements of FreeType mean.

So you don’t have to take all of these properties and remember, it’s just conceptual. FreeType FreeType FreeType FreeType FreeType FreeType

FT_Done_Face(face);
FT_Done_FreeType(ft);
Copy the code

OpenGL text rendering

Use FreeType to load the glyph bitmap and generate the texture, then texture map.

However, it is not efficient to reload bitmaps every time we render. We should store the generated data in the application, retrieve it during rendering, and reuse it.

For convenience, we need to define a structure to store these attributes and create a character table to store these glyph attributes.

struct Character {
	GLuint textureID;   // ID handle of the glyph texture
	glm::ivec2 size;    // Size of glyph
	glm::ivec2 bearing;  // Offset from baseline to left/top of glyph
	GLuint advance;    // Horizontal offset to advance to next glyph
};

std::map<GLint, Character> m_Characters;
Copy the code

For simplicity, we just generate a character table representing 128 ASCII characters and store textures and some metrics for each character. In this way, all the required characters are saved for use.

void TextRenderSample::LoadFacesByASCII(a) {
	// FreeType
	FT_Library ft;
	// All functions return a value different than 0 whenever an error occurred
	if (FT_Init_FreeType(&ft))
		LOGCATE("TextRenderSample::LoadFacesByASCII FREETYPE: Could not init FreeType Library");

	// Load font as face
	FT_Face face;
	if (FT_New_Face(ft, "/sdcard/fonts/Antonio-Regular.ttf".0, &face))
		LOGCATE("TextRenderSample::LoadFacesByASCII FREETYPE: Failed to load font");

	// Set size to load glyphs as
	FT_Set_Pixel_Sizes(face, 0.96);

	// Disable byte-alignment restriction
	glPixelStorei(GL_UNPACK_ALIGNMENT, 1);

	// Load first 128 characters of ASCII set
	for (unsigned char c = 0; c < 128; c++)
	{
		// Load character glyph
		if (FT_Load_Char(face, c, FT_LOAD_RENDER))
		{
		LOGCATE("TextRenderSample::LoadFacesByASCII FREETYTPE: Failed to load Glyph");
			continue;
		}
		// Generate texture
		GLuint texture;
		glGenTextures(1, &texture);
		glBindTexture(GL_TEXTURE_2D, texture);
		glTexImage2D(
				GL_TEXTURE_2D,
				0,
				GL_LUMINANCE,
				face->glyph->bitmap.width,
				face->glyph->bitmap.rows,
				0,
				GL_LUMINANCE,
				GL_UNSIGNED_BYTE,
				face->glyph->bitmap.buffer
		);

		// Set texture options
		glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
		glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
		glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
		glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
		// Now store character for later use
		Character character = {
				texture,
				glm::ivec2(face->glyph->bitmap.width, face->glyph->bitmap.rows),
				glm::ivec2(face->glyph->bitmap_left, face->glyph->bitmap_top),
				static_cast<GLuint>(face->glyph->advance.x)
		};
		m_Characters.insert(std: :pair<GLint, Character>(c, character));
	}
	glBindTexture(GL_TEXTURE_2D, 0);
	// Destroy FreeType once we're finished
	FT_Done_Face(face);
	FT_Done_FreeType(ft);

}

Copy the code

The texture format to use for OpenGL ES grayscale images is GL_LUMINANCE rather than GL_RED.

OpenGL textures require 4 bytes of alignment by default, so set this to 1 to ensure that bitmaps (grayscale) that are not four multiples of width will render properly.

Render text using shader:

//vertex shader #version 300 es layout(location = 0) in vec4 a_position; // <vec2 pos, vec2 tex> uniform mat4 u_MVPMatrix; out vec2 v_texCoord; Void main() {gl_Position = u_MVPMatrix * vec4(a_position.xy, 0.0, 1.0); v_texCoord = a_position.zw; } //fragment shader #version 300 es precision mediump float; in vec2 v_texCoord; layout(location = 0) out vec4 outColor; uniform sampler2D s_textTexture; uniform vec3 u_textColor; Void main() {vec4 color = vec4(1.0, 1.0, 1.0, texture(s_textTexture, v_texCoord).r); OutColor = vec4(u_textColor, 1.0) * color; }Copy the code

The fragment shader has two uniform variables: one is the font bitmap texture with a single color channel, and the other is the text color, which we can adjust to change the final output font color.

Turn on blending and remove the text background.

glEnable(GL_BLEND);
glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);  
Copy the code

Generate a VAO and a VBO to manage the storage of vertex and texture coordinate data. GL_DYNAMIC_DRAW indicates that we will use glBufferSubData to refresh the VBO cache.


glGenVertexArrays(1, &m_VaoId);
glGenBuffers(1, &m_VboId);

glBindVertexArray(m_VaoId);
glBindBuffer(GL_ARRAY_BUFFER, m_VboId);
glBufferData(GL_ARRAY_BUFFER, sizeof(GLfloat) * 6 * 4.nullptr, GL_DYNAMIC_DRAW);
glEnableVertexAttribArray(0);
glVertexAttribPointer(0.4, GL_FLOAT, GL_FALSE, 4 * sizeof(GLfloat), 0);
glBindBuffer(GL_ARRAY_BUFFER, GL_NONE);
glBindVertexArray(GL_NONE);
Copy the code

Each 2D square requires 6 vertices, each of which in turn consists of a 4-dimensional vector (a texture coordinate and a vertex coordinate), so we allocate VBO memory to the size of 6*4 floats.

Finally, render the text, where the viewport is passed in to normalize the screen coordinates:

void TextRenderSample::RenderText(std: :string text, GLfloat x, GLfloat y, GLfloat scale,
                                  glm::vec3 color, glm::vec2 viewport) {
	// Activate the appropriate render state
	glUseProgram(m_ProgramObj);
	glUniform3f(glGetUniformLocation(m_ProgramObj, "u_textColor"), color.x, color.y, color.z);
	glBindVertexArray(m_VaoId);
	GO_CHECK_GL_ERROR();
	// Iterates over all characters in the text
	std: :string::const_iterator c;
	x *= viewport.x;
	y *= viewport.y;
	for(c = text.begin(); c ! = text.end(); c++) { Character ch = m_Characters[*c]; GLfloat xpos = x + ch.bearing.x * scale; GLfloat ypos = y - (ch.size.y - ch.bearing.y) * scale; xpos /= viewport.x; ypos /= viewport.y; GLfloat w = ch.size.x * scale; GLfloat h = ch.size.y * scale; w /= viewport.x; h /= viewport.y; LOGCATE("TextRenderSample::RenderText [xpos,ypos,w,h]=[%f, %f, %f, %f]", xpos, ypos, w, h);

		// VBO of the current character
		GLfloat vertices[6] [4] = {
				{ xpos,     ypos + h,   0.0.0.0 },
				{ xpos,     ypos,       0.0.1.0 },
				{ xpos + w, ypos,       1.0.1.0 },

				{ xpos,     ypos + h,   0.0.0.0 },
				{ xpos + w, ypos,       1.0.1.0 },
				{ xpos + w, ypos + h,   1.0.0.0}};// Draw a glyph texture on the square
		glActiveTexture(GL_TEXTURE0);
		glBindTexture(GL_TEXTURE_2D, ch.textureID);
		glUniform1i(m_SamplerLoc, 0);
		GO_CHECK_GL_ERROR();
		// Update the VBO of the current character
		glBindBuffer(GL_ARRAY_BUFFER, m_VboId);
		glBufferSubData(GL_ARRAY_BUFFER, 0.sizeof(vertices), vertices);
        GO_CHECK_GL_ERROR();
		glBindBuffer(GL_ARRAY_BUFFER, 0);
		// Draw the block
		glDrawArrays(GL_TRIANGLES, 0.6);
		GO_CHECK_GL_ERROR();
		// Update the position to the origin of the next glyph, noting that the unit is 1/64 pixel
		x += (ch.advance >> 6) * scale; / / (2 ^ 6 = 64)
	}
	glBindVertexArray(0);
	glBindTexture(GL_TEXTURE_2D, 0);
}

Copy the code

Render 2 text using RenderText:

	/ / (x, y) for the position of the screen coordinate system, namely the origin is located in the center of the screen, x (1.0, 1.0), y (1.0, 1.0)
RenderText("My WeChat ID is Byte-Flow.".0.9 f.0.2 f.1.0 f, glm::vec3(0.8.0.1 f.0.1 f), viewport);
RenderText("Welcome to add my WeChat.".0.9 f.0.0 f.2.0 f, glm::vec3(0.2.0.4 f.0.7 f), viewport);
Copy the code

For the full implementation code, see the project: github.com/githubhaoha…

Text rendering effect:

reference

Learnopengl.com/In-Practice… Android.googlesource.com/platform/ex…

Technical communication

Technical exchange/get video tutorials can be added to my wechat: bytes-flow

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

OpenGL ES text rendering

Generate Bitmap based on Canvas drawing

FreeType

FreeType compilation

OpenGL uses FreeType to render text

The use of the FreeType

OpenGL text rendering

reference

Technical communication

OpenGL ES text rendering

Generate Bitmap based on Canvas drawing

FreeType

FreeType compilation

OpenGL uses FreeType to render text

The use of the FreeType

OpenGL text rendering

reference

Technical communication

Related Posts

Use design patterns to remove unnecessary state variables — state patterns

ReactNative enables map navigation

Is it so hard to sell Google vests?