Post-processing is the algorithm processing of the original game screen to improve the quality of the picture or enhance the picture effect technology, which can be achieved through the Shader program.

An overview of the

Deformation effect is a kind of post-processing technology to process and enhance the image effect, which is often applied in the effects of various camera short video apps, such as beauty and weight loss, and funhouse mirror effect.

Morphing effects for beauty camera

This paper mainly combs the following commonly used deformation effects from various beauty cameras:

  • Local distortion (Twirl effect)

  • Local swelling effect

  • Pinch effect in any direction

Among them, distortion can be used for local rotation of the eye, expansion can be used for large eyes, squeeze/stretch can be used for facial plasticity and lean face, etc. How to implement these transformations through Shader is the focus of this article. (PS: For those in a hurry to preview the code, see the end of this article)

Principle of deformation technique

Although the effects of deformation are varied, they are often dependent on three elements: the location of deformation, the scope of influence and the degree of deformation.

Shader Shader Shader Shader Shader

Therefore, its realization in Shader is to construct a deformation function, pass in the original UV coordinates, the deformation position, range and degree strength, and generate the deformation sampling coordinates after calculation. The code is as follows:

#iChannel0 "src/assets/texture/joker.png" vec2 deform(vec2 uv, vec2 center, float range, float strength) { // TODO: Deformation processing return UV; } void mainImage(out vec4 fragColor, vec2 coord) { vec2 uv = coord / iResolution.xy; vec2 mouse = iMouse.xy / iResolution.xy; uv = deform(uv, mouse, .5, .5); vec3 color = texture(iChannel0, uv).rgb; fragColor.rgb = color; }Copy the code

This article Shader code uses GLSL specification, follow Shader-Toy writing method, easy to preview.

Deformation tips: sampling distance field transformation

We set the fixed-point coordinate O, and the distance from any point to point O is dist. With different dist values as radii, countless equidistant sampling circles can be formed around point O, which are called the distance field of point O.

Sampling range field

We can change the size and position of the sampling circle, and then change the sampling position of the texture, to achieve expansion/contraction, extrusion deformation effect.

vec2 deform(vec2 uv, vec2 center, float range, float strength) { float dist = distance(uv, center); vec2 direction = normalize(uv - center); dist = transform(dist, range, strength); Center = transform(Center, dist, range, strength); Dist * direction (dist * direction); dist * dist * direction (dist * direction); }Copy the code

Before we get into the application of this technique, let’s start with a simple distortion.

distorted

The distortion effect is similar to the vortex shape, which is characterized by the more intense the rotation degree is closer to the center point. The relation between the distance D from the center point and the corresponding rotation Angle θ can be expressed by decreasing function.

In the figure below, A simple first-order function θ = -a /R *d + A is used, where A represents the rotation Angle of the distortion center, A positive number represents the rotation direction clockwise, A negative number represents the counterclockwise, and R represents the distortion boundary.

Principle of distortion

As shown in the figure above, the parameters A(central rotation Angle) and R(deformation Range) of the distortion function can be described as follows: 1) A represents the central rotation Angle. The greater the absolute value is, the higher the distortion degree is; 2) IF A > 0, the distortion direction is clockwise; otherwise, if A<0, it is counterclockwise; 3) R represents the distorted boundary, and the larger the value is, the larger the influence range is.

Twisting dynamic effect

We can introduce the time variable time to dynamically change the value of A to produce the twist effect, as shown in the figure above. The specific shader code is as follows:

#iChannel0 "src/assets/texture/joker.png" #define Range .3 #define Angle .5 #define SPEED 3. mat2 rotate(float a) // Rotation matrix {float s = sin(a); float c = cos(a); return mat2(c,-s,s,c); } vec2 twirl(vec2 uv, vec2 center, float range, float angle) { float d = distance(uv, center); uv -=center; // d = clamp(-angle/range * d + angle,0.,angle); // smoothstep(0., range, range-d) * Angle; uv *= rotate(d); uv+=center; return uv; } void mainImage(out vec4 fragColor, vec2 coord) { vec2 uv = coord / iResolution.xy; vec2 mouse = iMouse.xy / iResolution.xy; float cTime = sin(iTime * SPEED); uv = twirl(uv, mouse, Range, Angle * cTime); vec4 color = texture(iChannel0, uv); fragColor = color; }Copy the code

It is worth mentioning that, in addition to linear equations, you can also use the Smoothstep method, which is smoother at the distortion boundary than linear functions, as shown in the figure below.

Linear and Smoothstep distortion equation effect comparison

The following deformation methods also use smoothstep functions instead of linear equations for the sake of the smoothness of the boundary.

Expansion/contraction

The texture close to the expansion center is stretched, while the texture close to the expansion boundary is extruded. This means that within the expansion range, with the expansion center as the distance field, each sampling circle should be smaller than the original radius, and the circle spacing gradually expands from inside to outside.

On the right side of the figure below, we make the spacing between the new sampling circles monotonically increase from the inside out by mapping the isometric black sampling circles to the more cohesive red sampling circles.

Expansion sampling distance field transformation

We sample the smoothing increasing function Smoothstep to compute the scale value from sampling circle radius Dist:

The function in the figure above shows that the sampling circle has the most obvious scale and the smallest scale (1-s) near the expansion center. With the increase of dist, the scale value increases to 1 until it reaches the range of R boundary, and the scale stays at 1. The sampling circle is no longer scaled.

float scale = (1.- S) + S * smoothstep(0.,1., dist / R); // Calculate the expansion sampling radius scaling valueCopy the code

Thus, we obtained the sampling radius scaling formula above, where Strength(0 < S < 1) was set to represent the expansion degree. For the transformation process of the expansion distance field, it is easy to infer that, in order to achieve the reverse effect contraction of expansion, S can be directly located in the range [-1,0].

S value corresponds to Strength of expansion and contraction

As shown in the figure above, the parameters S(deformation degree Strength) and R(deformation Range) of the expansion function can be described as follows: 1) when S is in the Range [0,1], the expansion effect will be presented. The larger S value is, the higher the expansion degree will be. 2) When S is in the range [-10], the contraction effect is presented. The smaller S value is, the higher the contraction degree is; 3) R represents the boundary of deformation, and the larger the value is, the larger the influence area is;

Dynamic expansion effect

We can introduce the time variable time to dynamically change the value of Strength to simulate breathing animation, as shown in the picture above. The specific shader code is as follows:

# iChannel0 "SRC/assets/texture/joker. PNG" # define SPEED 2. / / # define SPEED RANGE. 2 / / # define Strength deformation RANGE. 5 * Sin (iTime * SPEED) // Degree of deformation vec2 inflate(VEC2 UV, VEC2 center, float range, float strength) {float dist = distance(uv, center); vec2 dir = normalize(uv - center); float scale = 1.-strength + strength * smoothstep(0., 1. ,dist / range); float newDist = dist * scale; return center + newDist * dir; } void mainImage(out vec4 fragColor, vec2 coord) { vec2 uv = coord / iResolution.xy; vec2 mouse = iMouse.xy / iResolution.xy; uv = inflate(uv, mouse, RANGE, Strength); vec3 color = texture(iChannel0, uv).rgb; fragColor.rgb = color; }Copy the code

Stretch vertically/horizontally

Original drawing – Longitudinal stretching – Lateral stretching – expansion

The front expansion is achieved by scaling the range field sampling circle, while the vertical/horizontal stretch is achieved by scaling only the X or y axis of the sampling circle, which is commonly used for beauty-looking “long leg effects”.

Transverse stretch distance field transformation

It can be found that the transverse stretching distance field is transformed into multiple elliptic sampling rings, and the code is as follows:

Vec2 inflateX(vec2 UV, vec2 Center, float radius, float strength) {vec2 inflateX(vec2 UV, vec2 Center, float radius, float strength) { return center + vec2(newDist, dist) * dir; // Scale only applies to X-axis}Copy the code

extrusion

Extruding usually indicates a point of application and a direction of extruding, which is characterized by pushing the texture near the point of application to the end of extrusion.

As shown in the figure below, the green point P is the starting point of extrusion, the arrow is the extrusion vector V, where the direction of the vector indicates the direction of extrusion, the length of the vector length(V) represents the distance of extrusion, and the end point of the vector is the position after extrusion. To achieve texture extrusion, the center of the sampling circle is shifted towards the extrusion vector V, and the sampling center point should be shifted to the position of point P.

Distance field transformation is used for extrusion

As the radius dist of the sampling circle gradually increases from inside to outside, the offset offset of the center of the circle is gradually shortened after transformation. We can handle the relationship between the sampling circle radius Dist and the offset offset by using the -SmoothStep smoothing decreasing function.

Offset = Length (V) -Length (V) * smoothstep(0, R, dist), where R represents the squeeze boundary range.

Extrusion dynamic effect

Similarly, we introduce the time variable time to dynamically change the length and direction of the extrusion vector, which can realize the jitter effect, such as the clown’s head and crotch effect in the figure above. The specific shader code is as follows:

# iChannel0 "SRC/assets/texture/joker. PNG" # define RANGE. The deformation RANGE 25 / / # define PINCH_VECTOR vec2 (sin (iTime * 10.), Cos (iTime * 20.)) *.03 // Pinch vec2 pinch(vec2 uv, vec2 targetPoint, vec2 vector, float range) { vec2 center = targetPoint + vector; float dist = distance(uv, targetPoint); vec2 point = targetPoint + smoothstep(0., 1., dist / range) * vector; return uv - center + point; } void mainImage(out vec4 fragColor, vec2 coord) { vec2 uv = coord / iResolution.xy; vec2 mouse = iMouse.xy / iResolution.xy; uv = pinch(uv, mouse, PINCH_VECTOR, RANGE); vec3 color = texture(iChannel0, uv).rgb; fragColor.rgb = color; }Copy the code

conclusion

This paper mainly introduces the realization principles of three types of local deformation shader, in which the expansion/contraction and extrusion effects are realized by sampling distance field transformation, the former is to transform the size of the sampling circle, the latter is to transform the position of the circle. In addition to the three kinds of local deformation described above, there are some more interesting global deformation effects, such as wave effect, dislocation effect and mirror image, etc., shader is easier to implement, so I won’t go into more details.

Wave – dislocation – mirror image

Preview code and effects

Distortion: www.shadertoy.com/view/slfGzN expansion/zoom: www.shadertoy.com/view/7lXGzN extrusion/drawing: www.shadertoy.com/view/7tX3zN

References:

GLSL basis transformation: thebookofshaders.com/08/?lan=ch Photoshop squeezing effects algorithms: blog.csdn.net/kezunhai/ar…

portal

WebGL advanced – into the pattern noise: zhuanlan.zhihu.com/p/68507311