preface

To achieve various filter effects in three.js, you need to use a custom ShaderMaterial.

The basic knowledge of shaders is introduced in the article writing a 3D Earth with three.js. Let’s go straight to the custom shader.

Filter basics

The color representation in WEBGL is RGBA, where the color value of each pixel is determined by R, G, B, and A, where each component is A floating point number in the range of [0,1]. For example, black is (0.0, 0.0, 0.0, 1.0) and white is (1.0, 1.0, 1.0, 1.0).

Three. Js encapsulates many implementation details of WEBGL, making it easy to develop 3D applications. At the same time, it also opens ShaderMaterial, a custom coloring material, allowing developers to freely manipulate pixels and achieve various customized effects.

For example, with a custom shader, we can manually modify the four component values of each pixel’s color value. Let’s take the earth map as an example.

Create objects with ShaderMaterial

We create an earth with ShaderMaterial as follows.

Var VSHADER_SOURCE = 'VARYING VEC2 v_Uv; Void main () {v_Uv = uv; Gl_Position = projectionMatrix * viewMatrix * modelMatrix * VEC4 (position, 1.0); Var FSHADER_SOURCE = 'uniform sampler2D e_Texture; // The texture image varying VEC2 v_Uv; Void main () {gl_FragColor = texture2D(e_Texture, v_Uv); // Extract the color value of the texture image. Var earthTexture = loader.load('./images/earth.png') // Create a shader var material = new Should be imaginative.ShaderMaterial({imaginative: {e_Texture: {value: earthTexture}}, // VSHADER_SOURCE, // specify the fragmentShader fragmentShader: FSHADER_SOURCE, }) var sphere = new THREE. Geometry(20,30,30) Material) // Add object to scene scene.add(sphere)Copy the code

In the fragment shader code snippet above, we extract the color value of A point in the texture image using the texture2D function, which returns A four-dimensional vector representing R, G, B, and A components.

Once the stripper is obtained, it is assigned to gl_FragColor, which specifies the color of the chip.

RGBA component operation

In fact, after obtaining the stripper, we can first perform some operations on the R, G, B, and A components and then assign them to gl_FragColor.

For example, to set the green (G) component of each striene to 0, modify the fragment shader code as follows:

var FSHADER_SOURCE = `uniform sampler2D e_Texture; varying vec2 v_Uv; void main () { vec4 textureColor = texture2D( e_Texture, v_Uv ); Float green = texturecolor. g * 0.0; float green = texturecolor. g * 0.0; Gl_FragColor = vec4(texturecolor. r, green, texturecolor. b, 1.0); } `Copy the code

The effect is as follows:

Look, it’s starting to feel like a filter.

In fact, the essence of A filter is to modify the color value of each pixel according to some algorithm, and more specifically, to modify the values of R, G, B and A components.

Let’s implement a common grayscale filter.

Gray filter principle and matrix implementation

Fundamentals and implementations

The principle of gray scale is simply to turn a color picture into a gray and white picture.

There are three common grayscale algorithms:

  • The maximum value method

  • The average method

  • Weighted mean method

Now, what we’re going to do is we’re going to do a weighted average.

The specific realization idea is that we first weighted average the R, G and B component values of each pixel of the image, and then take this value as the new R, G and B component values of each pixel. The specific formula is as follows:

Where R, G and B are the color values of channels R, G and B in the original picture, V is the weighted average color value, a, B and C are the weighted coefficients, satisfying (a + B + C) = 1.

Based on the above principle, let’s implement it. Modify the fragment shader code:

var FSHADER_SOURCE = `uniform sampler2D e_Texture; varying vec2 v_Uv; void main () { vec4 textureColor = texture2D( e_Texture, v_Uv ); // Calculate the weighted average float w_a = Texturecolor. r * 0.3 + Texturecolor. g * 0.6 + Texturecolor. b * 0.1; Gl_FragColor = VEC4 (w_A, w_A, w_A, 1.0); } `Copy the code

The weighted coefficients here are a = 0.3, B = 0.6 and C = 0.1, respectively, because human vision has different sensitivity to R, G and B, so the weights of the three colors are different. Generally speaking, green is the highest, red is the second, and blue is the lowest. You can change it to something else and see what happens.

Matrix to achieve

If you want to adjust the weighting factor or implement other filters, you have to change the slice shader code again. It’s too much trouble.

Now we’ll use matrices to optimize the code.

Since the matrix in GLSL ES is at most four-dimensional, the matrix transformation of the three quantities R, G and B is carried out here, and the transparency alpha is set separately.

Js code modified as follows:

Public derdermaterial = new THREE.ShaderMaterial({imaginative: {e_Texture: {value: earthTexture}, t_Matrix: {value: // vertexShader: VSHADER_SOURCE, // fragmentShader: FSHADER_SOURCE})Copy the code

The fragment shader code has also been modified accordingly:

var FSHADER_SOURCE = `uniform sampler2D e_Texture; uniform mat4 t_Matrix; // The transform matrix varying VEC2 v_Uv; void main () { vec4 textureColor = texture2D( e_Texture, v_Uv ); Vec4 (R,G,B,1) -->vec4(R,G,B,1) Vec4 transColor = t_Matrix * vec4(texturecolor.r, texturecolor.g, texturecolor.g, texturecolor.g, TextureColor. B, 1.0); // Set transparency to transcolor. a = 1.0; gl_FragColor = transColor; } `Copy the code

According to the multiplication of matrix and vector, the transformation matrix of grayscale filter should be as follows:

OpenGL API accepts the matrix requirement is column main order, if an OpenGL application uses a row main order matrix, then before transferring the matrix to OpenGL API, it needs to be converted to column main order. We’re going to do the transpose by hand.

Reference: Row and column main order of matrices in OpenGL

With matrices, you can very well abstract and reuse. A matrix is a transformation, and you just multiply it by this matrix, which means you’ve done the same transformation, and when you want to modify a transformation, you just modify the transformation matrix.

Realization of several filter effects

To achieve the following filter effects, several different matrices are used. (It is worth noting that the following transformation matrices are also transposed column principal order matrices. To derive, transpose the following matrix back to row main order.

The color

The realization of inverse color, that is, R, G, B each component of 1 complement.

Transformation matrix:

var oppositeMatrix = new Float32Array([
   -1, 0, 0, 0,
   0, -1, 0, 0,
   0, 0, -1, 0,
   1, 1, 1, 1
])
Copy the code

brightness

Each component of R, G, and B is multiplied by a value p; P < 1 darken, P = 1 primary color, p > 1 lighten.

Transformation matrix :(pass in parameter p to dynamically generate transformation matrix)

function genBrightMatrix (p) {
    return new Float32Array([
      p, 0, 0, 0,
      0, p, 0, 0,
      0, 0, p, 0,
      0, 0, 0, 1
    ])
}
Copy the code

contrast

Multiply each component of R, G, and B by a value p, then add 0.5 * (1-p); P = 1 primary color, P < 1 reduce contrast, P > 1 enhance contrast.

Matrix-based implementation :(pass in parameter p, dynamically generate transformation matrix)

Function genContrastMatrix (p) {var d = 0.5 * (1 - p) return New Float32Array([p, 0, 0, 0, 0, 0, p, 0, 0, p, 0, d) d, d, 1 ]) }Copy the code

saturation

Pass in the parameter p and calculate it as follows. P = 0 for full grayscale, P = 1 for primary color, and P > 1 for enhanced saturation.

Function genSaturateMatrix (p) {var rWeight = 0.3 * (1-p) var gWeight = 0.6 * (1-p) var bWeight = 0.1 * (1-p) return new Float32Array([ rWeight + p, rWeight, rWeight, 0, gWeight, gWeight + p, gWeight, 0, bWeight, bWeight, bWeight + p, 0, 0, 0, 0, 1 ]) }Copy the code

That’s it for today, this article mainly discusses some basic filter principles and the elementary use of matrices. More cool filter algorithms will be unlocked in a later post, see you next time!

reference

Geektime: Visualizing The Shadow of the Moon