Graphics Rendering (3) Graphics pipeline with shaders

Welcome to follow the public account: Sumsmile/focus on image processing mobile development veterans ~~

This article mainly talk about shader implementation, by the way to mention the graphic pipeline code reference: github.com/Quanwei1992…

Implementation effect

Implement various maps based on a small cow model.

Model:

Texture map:

Feng illumination:

Normal map:

Bump map:

Unique texture:

The graphics pipeline

In OpenGL, the fixed process is encapsulated, transparent to developers, and only vertex shaders and chip shaders are exposed for customization. This is very good design. Take Feng lighting as an example. In the element shader, “ambient light”, “diffuse reflection”, and “specular reflection” are achieved to render the effect much like plastic products.

Graphic pipeline is not difficult, focus on the analysis of shader implementation

shader

For simple scenes, vertex shaders generally do not do complex calculations and are only used as entry points for parameters. The logic is mainly in the chip shader. More complex scenes may also use geometric shaders.

The vertex shader code is very simple and returns the position of the model.

Eigen::Vector3f vertex_shader(const vertex_shader_payload& payload)
{
    return payload.position;
}
Copy the code

Vertex shaders are generally much less computative than chip shaders. Because the number of vertices that make up a triangle is relatively limited, the slices need to interpolate based on the triangles that the vertices make up.

For example, a 100 * 100 rectangular plane with four vertices is computed four times, but the chip shader is computed 10,000 times.

Therefore, work that can be computed in the vertex shader should not be placed in the chip shader.

I did not do scientific tests here, because of the high concurrency characteristics of Gpus, the actual performance should not be compared by the number of times.

Texture map

Texture mapping is relatively simple, the model has generated the corresponding texture coordinates for each vertex, directly get the corresponding color

Texture models are usually designed with tools according to certain rules

Phong illumination

Feng lighting is relatively simple, there is a lot of information on the Internet. Phong illumination

Normal map

It’s also a little bit simpler. The model comes with normals. Note that the normal line ranges from [-1, 1] and needs to be mapped to [0,1] and then converted to [0, 255].

Eigen::Vector3f normal_fragment_shader(const fragment_shader_payload& payload)
{
    Eigen::Vector3f return_color = (payload.normal.head<3> ().normalized() + Eigen::Vector3f(1.0 f.1.0 f.1.0 f)) / 2.f;
    Eigen::Vector3f result;
    result << return_color.x(*)255, return_color.y(*)255, return_color.z(*)255;
    return result;
}
Copy the code

Bump map

It gets a little more complicated here. On the basis of Feng’s illumination, the bump texture map is added, and the bump texture is used to replace the normals on the surface of the cow model to simulate the uneven effect of the real object surface.

Normals are stored in RGB. Most normals are perpendicular to the surface, so the B component is larger, so the whole texture is blue.

Bump Mapping Principle

1. Based on the concave – convex texture, the normal vector is calculated. You have to use the partial derivative of calculus.

• dp/du = c1 * [h(u+1) – h(u)]
• dp/dv = c2 * [h(v+1) – h(v)]
• n = (-dp/du, -dp/dv, 1).normalized()

2. Calculate TBN

A tangent matrix can align normal maps to the model surface, and a normal map can be reused.

It is difficult to fully understand TBN, and I have referred to many materials in the process of learning

• Tangent space and normal vector transformation
• Tangent Space
• Derivation of the Tangent Space Matrix
• TBN problem
• Normal map, TBN
• Opengl-tutorial normal map

It will not be repeated here.

In real code, TBN can be calculated in a more ingenious way

• If we project the normal line n onto the xz plane and the Y-axis, we get two vectors A(xz) and B(y).
• Rotate A(xz) and B(y) at the same time, so that A(xz) is aligned with the Y-axis and B(y) is aligned with the Xz plane, and the two new vectors form the T vector, the tangent vector
• Cross T with the normal vector n, you get the complex tangent vector

I am also inspired by this post TBN fast algorithm explanation

Notice that the implementation in the code is wrong, the components of t are not positive or negative, and the resulting TBN is not perpendicular.

 Vector3f t(x*y/sqrt(x*x+z*z),sqrt(x*x+z*z),z*y/sqrt(x*x+z*z));
    Vector3f b = normal.cross(t);
Copy the code

You take TBN, you transform the normal, you get the normal of the view space

 Vector3f n = (TBN * ln).normalized(a);Copy the code

In addition, the normal vector, along with the model, needs to be calculated through the MV matrix, but direct calculation will result in the normal line not perpendicular to the surface of the model, for example, translation will change the direction of the normal line. The transformation matrix of the actual normal is the transpose of the inverse matrix

OpenGL Normal Vector Transformation

Displacement map

The displacement map is a further optimization of the bump map. In contrast to the figure above, the bump map simulates the bump by the direction of the normal vector, which actually affects the reflection of light. The displacement map actually shifts the vertices.

The contours of the bump map are still smooth, and the contours of the displacement map are also bumpy and more realistic.

    // Change the position of the original point and change the direction of the normal vector
    Vector3f ln(-dU,-dV,1);
    Vector3f n = (TBN * ln).normalized(a); Vector3f p = point + n * huv * kn;// Then render with feng lighting.Copy the code

The displacement mapping here is relatively simple, you can refer to learnopengl-cn.github. IO/on the displacement mapping, very detailed.

Supplementary, judge the point in the triangle

In the actual code, in the rasterization logic, the algorithm for judging points in triangles is optimized, and it’s hard to understand at first.

The common “point in the triangle” algorithm is easy to understand, based on the cross product or the dot product.

Determine whether the point is in the triangle

The implementation in this article’s code:

static bool insideTriangle(int x, int y, const Vector4f* _v){
    Vector3f v[3];
    for(int i=0; i<3; i++) v[i] = {_v[i].x(),_v[i].y(), 1.0};
    Vector3f f0,f1,f2;
    f0 = v[1].cross(v[0]);
    f1 = v[2].cross(v[1]);
    f2 = v[0].cross(v[2]);
    Vector3f p(x,y,1.);
    if((p.dot(f0)*f0.dot(v[2) >0) && (p.dot(f1)*f1.dot(v[0) >0) && (p.dot(f2)*f2.dot(v[1) >0))
        return true;
    return false;
}
Copy the code

Refer to the following figure and instructions:

Can move the figure of address: www.geogebra.org/3d/qf2fcj9t

To understand:

• Consider the triangle ABC as a vector whose starting point is at the origin
• The cross product calculates the normal vector of OA and OC, which is F0
• In this way, the Angle between OB and f0 is greater than 90°. If F and B are on the same side OF AC, then ob.dot (f0) has the same sign as of.dot (f0) and is either positive or negative

The principle is to use the cross product, and this algorithm does not subtract three times, which is to take the vectors AB, BC, and CA. It’s actually better than the insideTriangle implementation in the last article, but it’s less intuitive, and it took me a few hours to figure it out.

Welcome to follow the public account: Sumsmile/focus on image processing mobile development veterans ~~

reference

[1] Gouraud shader method: zh.wikipedia.org/wiki/Gourau…

[2] Learnopengl-cn.github. IO /02%20Lighti…

[3] The transformation of tangent space and normal vector: juejin.cn/post/696790…

[4] Tangent Space: www.opengl.org/archives/re…

[5] Tangent Space Matrix: www.blacksmith-studios.dk/projects/do…

[6] TBN problem: www.zhihu.com/question/43…

[7] TBN: Learnopengl-cn.github. IO /05%20Advanc…

[8] Opengl-Tutorial normal map: www.opengl-tutorial.org/cn/intermed…

[9] TBN fast algorithm: games-cn.org/forums/topi…

[10] OpenGL Normal Vector Transformation: www.songho.ca/opengl/gl_n…

[11] to determine whether a point within the triangle: www.cnblogs.com/graphics/ar…

[12] Cross product: en.wikipedia.org/wiki/Cross_…

[13] Adjoint matrix: zh.wikipedia.org/wiki/ Adjoint matrix

[14] the derivation process of the inverse matrix: mp.weixin.qq.com/s/NWmLZnIDA…