There are many ways to implement shadow in 3D world, among which ShadowMap is the most commonly used scheme. This article introduces the principle of shadow generation with ShadowMap, the related shader code, and the step-by-step optimization of shadow quality.
How are shadows created
In nature, a non-luminous object needs a light source to be seen. Since light travels in a straight line, when the light is blocked by something (the orange object in the picture), the areas (point C) that were originally colored turn black because they are not illuminated. These areas are called shadows.
How to use ShadowMap to generate shadows
1. The ShadowMap principle
In theory, when drawing the color of a point, you can know whether to draw a shadow as long as you judge whether the point is “blocked”. There are many schemes to judge “occlusion”, the most commonly used is ShadowMap. We just need to know if there is a point on the line between the point and the light source that is closer to the light source. The distance between the point and the light source is the depth in ShadowMap. The specific approach is:
- (1) Generate depth texture map: the so-called depth texture map is the minimum depth of each position. We stand at the position of the light source, observe the scene according to the perspective of light propagation, calculate the distance between the object in the scene and the light source (that is, the depth of the perspective), and record the minimum value at each position, so as to obtain a depth texture.
- (2) Use the depth texture map: For a certain point P in the world, we need to get its depth from the perspective of the light source, and then compare it with the corresponding depth in the depth texture map to determine whether it is in the shadow.
2. Shader code
(1) Generate depth texture map
Vertex shader code:
attribute vec4 a_Position;
uniform mat4 u_MvpMatrix; // The projection matrix with the light source as the observation point
void main() {
gl_Position = u_MvpMatrix * a_Position;
}
Copy the codeFragment shader code:
precision mediump float; // Specify the precision
void main() {
gl_FragColor = vec4(gl_FragCoord.z, 0.0.0.0.0.0); // Write the depth value of the chip to r
}
Copy the code(2) Use depth texture maps
Vertex shader code:
attribute vec4 a_Position;
attribute vec4 a_Color; // The color of the object when illuminated
uniform mat4 u_MvpMatrix; // The projection matrix of the human observation point
uniform mat4 u_MvpMatrixFromLight; // The projection matrix with the light source as the observation point
varying vec4 v_PositionFromLight;
varying vec4 v_Color;
void main() {
gl_Position = u_MvpMatrix * a_Position;
v_PositionFromLight = u_MvpMatrixFromLight * a_Position; // Use the light source as the coordinate of the observation point
v_Color = a_Color;
}
Copy the codeChip shader code
precision mediump float; // Specify the precision
uniform sampler2D u_ShadowMap; // Depth texture map
varying vec4 v_PositionFromLight; // Use the light source as the coordinate of the observation point
varying vec4 v_Color;
void main() {
vec3 shadowCoord = (v_PositionFromLight.xyz/v_PositionFromLight.w)/2.0 + 0.5; // the MVP matrix will be automatically converted to the clipped space coordinates, the range is [0,1], so there is also normalization here
vec4 rgbaDepth = texture2D(u_ShadowMap, shadowCoord.xy); // Get the data stored in the coordinates of the depth texture
float depth = rgbaDepth.r; // Get the depth of the coordinates stored in the depth texture
float visibility = (shadowCoord.z > depth) ? 0.7 : 1.0; // Check whether the slice is in shadow
gl_FragColor = vec4(v_Color.rgb * visibility, v_Color.a);
}
Copy the code3. Shadows
As you can see, although shadows have been generated, the quality of the shadows is very poor. Let’s optimize each one.
3. Defects and optimization of ShadowMap
1. Self-Shadowing && Shadow Bias
The case of the bar shadows in the figure is self-shadowing. Because we need to normalize and store the depth of the object from the perspective of the light source, precision loss will inevitably lead to depth errors.
For example, if there is a point P in the space, its actual depth under the perspective of the light source is 0.70001, which is also the minimum depth under the perspective of the light source, then it will not be blocked theoretically and should be displayed in white. However, we need to store the minimum depth from the perspective of the light source in advance. At this time, due to the loss of accuracy, 0.70001 -> 0.7000 is caused. Then when drawing point P, it is judged that the actual depth 0.70001 > the minimum depth of storage is 0.7000, which means it is blocked and painted black by mistake. The larger the tilt of the surface in the light view space, the greater the error.
Shadow Bias – When actually drawing, add a threshold to the memory depth taken from the deep texture.
float visibility = (shadowCoord.z > depth + 0.15)?0.7 : 1.0; // Determine whether the slice prefixes the shadow with a threshold
gl_FragColor = vec4(v_Color.rgb * visibility, v_Color.a);
}
Copy the code
The strip shadows have been removed at this point, but the shadows deviate too much. This situation is called Peter Panning
2. Peter Panning
Peter Panning was created because we added too much Shadow Bias, which made it too different from its actual depth. Solution: Control the threshold size.
float visibility = (shadowCoord.z > depth + 0.01)?0.7 : 1.0; // Control threshold :0.15 -> 0.01
gl_FragColor = vec4(v_Color.rgb * visibility, v_Color.a);
}
Copy the code
3. Shadow edge serrated
(1) Improve resolution
When the depth texture map is too small (the resolution is too low), multiple slices will correspond to the same pixel in the depth texture, resulting in serrations. Here is the effect of scaling the texture size from 128 * 128 to 1024 * 1024.
(2) Hard Shadow && PCF
After the depth texture resolution was improved, the shadow was still jagged. This is not because of the way the shadows are generated, but because the edges themselves are jagged.
We can either work with the jagged edges of various objects in the world, or we can take a more efficient approach and smooth out the shadow edges themselves.
PCF – Percentage Closer Filtering
The core idea of PCF is not to take the shadow of the current point directly, but to get it through the weighted average of the surrounding points.
The specific approach is to sample multiple adjacent values from the Shadow Map for each slice, and then compare each value in depth. If the element is in the shaded area, the comparison result is marked as 0; otherwise, it is marked as 1. Finally, by dividing all the comparison results by the number of sampling points, a percentage P can be obtained, indicating the probability that the element is in the shaded area. If p is 0, it means that the pixel is completely in the shadow area; if p is 1, it means that it is not in the shadow area at all. Finally, the mixing coefficient can be set according to the p value.
The shader code is as follows: The larger the number of samples, the better the smoothing effect
precision mediump float;
uniform sampler2D u_ShadowMap;
varying vec4 v_PositionFromLight;
varying vec4 v_Color;
void main() {
vec3 shadowCoord = (v_PositionFromLight.xyz/v_PositionFromLight.w)/2.0 + 0.5; // normalize to the [0,1] texture interval
float shadows = 0.0;
float opacity = 0.6; // Shadow alpha value, the smaller the darker
float texelSize = 1.0/1024.0; // The size of the shadow pixel, the smaller the value, the more lifelike the shadow
vec4 rgbaDepth;
// Remove the jagged edges of the shadow, simplifying the scheme here - compare the different recording depths of the current slice element with the surrounding points
for(float y=1.5; y <= 1.5; y += 1.0) {for(float x=1.5; x <=1.5; x += 1.0){
rgbaDepth = texture2D(u_ShadowMap, shadowCoord.xy + vec2(x,y) * texelSize);
shadows += (shadowCoord.z > rgbaDepth.r + 0.01)?1.0 : 0.0;
}
}
shadows /= 16.0; // 4*4 samples
float visibility = min(opacity + (1.0 - shadows), 1.0);
gl_FragColor = vec4(v_Color.rgb * visibility, v_Color.a);
}
Copy the codePCF effect comparison:
Overall optimization effect comparison of Shadow Map:
4. The appendix
- Related code: github.com/Zack921/vis…