Write a few thoughts on Shader transition

preface

Transitions are most common in video editing tools. Adding a “transition” effect between two videos or images makes the process smoother and more natural. Common transitions such as gradient transitions, rotations, erases, etc. (iMovie’s transitions are shown below) :

And now many video apps also have the photo album function, you can choose different transitions to create a dynamic photo album:

Transitions are quite different in WebGL compared to editors, and these differences lead to some thoughts:

First, material switching time

In the previous article, I mentioned the switching of two materials, but generally there are more than two pictures in the photo set. How to make the whole switching cycle and no perception? Here is a simple animation example:

To explain this briefly, suppose that our transition effect is to switch from right to left (as shown in the GIF) at the end of each animation round, reassign u_Sampler0 and u_Sampler1, and the first image of each animation round is the next image of the previous animation round. This instantaneous assignment makes the entire animation feel unchanged. This allows for looping of different materials without taking up too much texture space in WebGL (only two textures are needed), which comes from the experience of writing The Slider on the Web side.

The relevant codes are as follows:

// Change the material
function changeTexture(gl, imgList, count) {
    var texture0 = gl.createTexture();
    var texture1 = gl.createTexture();

    if(! texture0 && ! texture1) {console.log('Failed to create the texture object');
        return false;
    }

    var u_Sampler0 = gl.getUniformLocation(gl.program, 'u_Sampler0');
    if(! u_Sampler0) {console.log('Failed to get the storage location of u_Sampler0');
        return false;
    }
    var u_Sampler1 = gl.getUniformLocation(gl.program, 'u_Sampler1');
    if(! u_Sampler1) {console.log('Failed to get the storage location of u_Sampler1');
        return false;
    }

    loadTexture(gl, texture0, u_Sampler0, imgList[count%imgList.length], 0);
    loadTexture(gl, texture1, u_Sampler1, imgList[(count+1)%imgList.length], 1);
}

// Load the material
function loadTexture(gl, texture, u_Sampler, image, index) {
    gl.pixelStorei(gl.UNPACK_FLIP_Y_WEBGL, 1)
    gl.activeTexture(gl['TEXTURE'+index])
    gl.bindTexture(gl.TEXTURE_2D, texture)
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
    gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, image);
    gl.uniform1i(u_Sampler, index);
    return true;
}
Copy the code

Two, transition effect switch

A lot of times we use a combination of different transitions. Here are two ideas:

1. Perform transitions in the Shader

Pass in a variable that records the number of transitions, determine the number in the shader code, and switch transitions

precision mediump float;
varying vec2 uv;
uniform float time;     // Change time
uniform sampler2D u_Sampler0; 
uniform sampler2D u_Sampler1;

uniform float count;    // The number of cycles

void main() {

    if (count == 1.) {
        // The first transition
        // Play the first effect
    }
    else if (count == 2.) {
        // The second transition
        // Play the second effect}}Copy the code

This approach has obvious disadvantages: first, the file is not granular enough, and one file has multiple effects; Second logic and effect coupling together, not easy to do any combination of different transitions, such as I have 1, 2, 3 kinds of transitions, if it is a independent file storage, I can adjust the order at 123/132/231/213/312/321/1123 /… To control the playing time of each transition. Therefore, the second method is more recommended:

2. Each transition is independent of files and codes

// transition1.glsl
precision mediump float;
varying vec2 uv;
uniform float time;
uniform sampler2D u_Sampler0; 
uniform sampler2D u_Sampler1;

void main() {
    // ...
}
Copy the code

// transition2.gls
precision mediump float;
varying vec2 uv;
uniform float time;
uniform sampler2D u_Sampler0; 
uniform sampler2D u_Sampler1;

void main() {
    // ...
}
Copy the code

Then we control the transition in JavaScript:

// Add this code at the bottom of main()
void main() {
    function render() {
        var img1 = null;
        var img2 = null;
        
        // Remove the image one at a time
        if (imgList.length > 2) {   
            img1 = imgList.shift()
            img2 = imgList[0]}else {
            return;
        }
    
        // I added a random logic to switch the second transition when there were three images left.
        // File fetching is ignored here
        if (imgList.length == 3) {
            setShader(gl, VSHADER_SOURCE, FSHADER_SOURCE2);   
        } else {
            setShader(gl, VSHADER_SOURCE, FSHADER_SOURCE);
        }
        
         // Set the material
        setTexture(gl, img1, img2);   
        
        // Time and timeRange are used to determine the time of each round.
        GetAnimationTime () to get the progress time from 0 to 1
        var todayTime = (function() {
            var d = new Date(a); d.setHours(0.0.0.0);
            returnd.getTime(); }) ()var duration = 2000;
        var startTime = new Date().getTime() - todayTime;
    
        var timeRange = gl.getUniformLocation(gl.program, 'timeRange');
        gl.uniform2f(timeRange, startTime, duration);
        var time = gl.getUniformLocation(gl.program, 'time');
        gl.uniform1f(time, todayTime);
        
        // Since calling setShader resets program, all variables related to gl.program are reassigned
        var xxx = gl.getUniformLocation(gl.program, 'xxx');
        gl.uniform2f(xxx, 750..1334.);
    
        // Inside loop, each time this round of transitions to play the end
        var requestId = 0;
        (function loop(requestId) {
            var curTime = new Date().getTime() - todayTime;
            if (curTime <= startTime + duration) {
                gl.uniform1f(time, curTime)
                gl.clear(gl.COLOR_BUFFER_BIT);
                gl.drawArrays(gl.TRIANGLE_FAN, 0.4);
                requestId = requestAnimationFrame(loop.bind(this, requestId))
            } else {
                cancelAnimationFrame(requestId)
                render()
            }
        })(requestId)
    }
    render()
}


// Change the material
function setTexture(gl, img1, img2) {
    var texture0 = gl.createTexture();
    var texture1 = gl.createTexture();

    var inputImageTexture = gl.getUniformLocation(gl.program, 'inputImageTexture');
    var inputImageTexture2 = gl.getUniformLocation(gl.program, 'inputImageTexture2');

    loadTexture(gl, texture0, inputImageTexture, img1, 0);
    loadTexture(gl, texture1, inputImageTexture2, img2, 1);
}

// Switch different transitions (just change fshader)
function setShader(gl, vshader, fshader) {
    if(! initShaders(gl, vshader, fshader)) {console.log('Failed to intialize shaders.');
        return; }}Copy the code

3. Material transition mode

Transitions are generally accompanied by switching between two pictures. There are two common switching modes:

Linear interpolation of pixels in two graphsmix(), the switch is softer
According to the time to switch, switch more stiff

1. Linear interpolation

It is generally applicable to the transition field with smooth transition, and the process of alternation of two pictures can be clearly seen:

return mix(texture2D(u_Sampler0, uv), texture2D(u_Sampler1, uv), progress);
Copy the code

2. Switch based on the time

It is generally applicable to the case that the transition changes quickly, and this switch cannot be distinguished by the naked eye.

if (progress < 0.5) {
    gl_FragColor = texture2D(u_Sampler0, uv);
} else {
    gl_FragColor = texture2D(u_Sampler1, uv);
}
Copy the code

For example, the first transition in the image below switches the texture instantly according to time (but can’t be seen), the latter is a linear interpolation gradient:

Four, animation speed simulation

Basically no transition is as simple as a linear uniform motion, so you have to simulate different velocity curves here. In order to create a better transition effect, there are several steps:

1. Get a real time curve

Assuming the transition is designed by yourself, you can use some preset curves, such as the ones provided here:

Bessel formula of the curve can be obtained directly:

Assuming that the transition effect is provided by others, if the designer uses AE to produce the transition effect, the time change curve corresponding to the relevant movement can be found in AE:

2. Use speed curves

Once we get the curve, the next step of course is to get the mathematical formula and plug it into our variables (progress/time/uv.x, etc.).

The first thing to be clear about is that time in the real world does not get faster or slower, which means that time always moves at a constant speed. It’s just that when we apply the formula to unit time, the result changes in speed (if the X-axis motion is our independent variable, then y can be the dependent variable).

#ifdef GL_ES
precision mediump float;
#endif

# define PI (3.14159265359)
uniform vec2 u_resolution;
uniform vec2 u_mouse;
uniform float u_time;

float plot(vec2 st, float pct){
  return  smoothstep( pct0.01, pct, st.y) -
          smoothstep( pct, pct+0.01, st.y);
}

float box(vec2 _st, vec2 _size, float _smoothEdges){
    _size = vec2(0.5)-_size*0.5;
    vec2 aa = vec2(_smoothEdges*0.5);
    vec2 uv = smoothstep(_size,_size+aa,_st);
    uv *= smoothstep(_size,_size+aa,vec2(1.0)-_st);
    return uv.x*uv.y;
}

void main() {
    vec2 st = gl_FragCoord.xy / u_resolution;
    vec2 boxst = st + . 5;

    // The mathematical formula y = f(x)
    // The independent variable is st.x and the dependent variable is st.y
    float f_x = sin(st.x*PI);
    
    // Calculate the position of each movement of the small square
    // The formula is the same as f(x) above, only
    // Our dependent variable changed from st.x to FRact (u_time)
    // fract(u_time) keeps the time from 0 to 1 forever
    // The reason for *.6 is not to move so fast that you cannot see the speed change
    boxst.y -= sin(fract(u_time*6.)*PI);
    boxst.x -= fract(u_time*6.);

    // Draw time curves and squares
    float box = box(boxst, vec2(. 08.. 08), 0.001);
    float pct = plot(st, f_x);
    
    vec3 color = pct*vec3(0.0.1.0.0.0)+box;
    gl_FragColor = vec4(color,1.0);
}
Copy the code

We can use st.x or u_time/progress as the independent variable to get the corresponding motion curve and animation. Now we can try other animation curves:

// Show some code
float f_x = pow(st.x, 2.);

boxst.y -= pow(fract(u_time*6.), 2.);
boxst.x -= fract(u_time*6.);
Copy the code

float f_x = -(pow((st.x1.), 2.) 1.);

boxst.y -= -(pow((fract(u_time*6.)1.), 2.) 1.);
boxst.x -= fract(u_time*6.);
Copy the code

// easeInOutQuint
float f_x = st.x<. 5 ? 16.*pow(st.x, 5.) : 1.+16.*(--st.x)*pow(st.x, 4.);

boxst.y -= fract(u_time*6.) <. 5 ? 16.*pow(fract(u_time*6.), 5.) : 1.+16.* (fract(u_time*6.)1.) *pow(fract(u_time*6.)1..4.);
boxst.x -= fract(u_time*6.);
Copy the code

// easeInElastic
float f_x = ((. 04 -.04/st.x) * sin(25.*st.x) + 1.) *8.;

boxst.y -= ((. 04 -.04/fract(u_time*6.)) * sin(25.*fract(u_time*6.)) + 1.) *8.;
boxst.x -= fract(u_time*6.);
Copy the code

// easeOutElastic
float f_x = (. 04*st.x /(--st.x)*sin(25.*st.x))+2.;

boxst.y -= (. 04*fract(u_time*6.)/(fract(u_time*6.)1.) *sin(25.*fract(u_time*6.))) +2.;
boxst.x -= fract(u_time*6.);
Copy the code

More easing functions:

EasingFunctions = {
  // no easing, no acceleration
  linear: function (t) { return t },
  // accelerating from zero velocity
  easeInQuad: function (t) { return t*t },
  // decelerating to zero velocity
  easeOutQuad: function (t) { return t*(2-t) },
  // acceleration until halfway, then deceleration
  easeInOutQuad: function (t) { return t<. 5 ? 2*t*t : - 1+ (42 -*t)*t },
  // accelerating from zero velocity 
  easeInCubic: function (t) { return t*t*t },
  // decelerating to zero velocity 
  easeOutCubic: function (t) { return (--t)*t*t+1 },
  // acceleration until halfway, then deceleration 
  easeInOutCubic: function (t) { return t<. 5 ? 4*t*t*t : (t- 1) * (2*t2 -) * (2*t2 -) +1 },
  // accelerating from zero velocity 
  easeInQuart: function (t) { return t*t*t*t },
  // decelerating to zero velocity 
  easeOutQuart: function (t) { return 1-(--t)*t*t*t },
  // acceleration until halfway, then deceleration
  easeInOutQuart: function (t) { return t<. 5 ? 8*t*t*t*t : 1- 8 -*(--t)*t*t*t },
  // accelerating from zero velocity
  easeInQuint: function (t) { return t*t*t*t*t },
  // decelerating to zero velocity
  easeOutQuint: function (t) { return 1+(--t)*t*t*t*t },
  // acceleration until halfway, then deceleration 
  easeInOutQuint: function (t) { return t<. 5 ? 16*t*t*t*t*t : 1+16*(--t)*t*t*t*t },
  // elastic bounce effect at the beginning
  easeInElastic: function (t) { return (. 04 - . 04 / t) * sin(25 * t) + 1 },
  // elastic bounce effect at the end
  easeOutElastic: function (t) { return . 04 * t / (--t) * sin(25 * t) },
  // elastic bounce effect at the beginning and end
  easeInOutElastic: function (t) { return (t -= . 5) < 0 ? (. 02 + .01 / t) * sin(50 * t) : (. 02 - .01 / t) * sin(50 * t) + 1 },
  easeIn: function(t){return function(t){return pow(t, t)}},
  easeOut: function(t){return function(t){return 1 - abs(pow(t- 1, t))}},
  easeInSin: function (t) { return 1 + sin(PI / 2 * t - PI / 2)},
  easeOutSin : function (t) {return sin(PI / 2 * t)},
  easeInOutSin: function (t) {return (1 + sin(PI * t - PI / 2)) / 2}}Copy the code

3. Construct a custom speed curve

Custom speed curve we can draw by Bezier curve, how to convert our common bezier curve in CSS into a mathematical formula? This article gives us an idea. By modifying the JavaScript code provided by it, we get the following Shader functions:

float A(float aA1, float aA2) {
    return 1.0 - 3.0 * aA2 + 3.0 * aA1;
}

float B(float aA1, float aA2) {
    return 3.0 * aA2 - 6.0 * aA1;
}

float C(float aA1) {
    return 3.0 * aA1;
}

float GetSlope(float aT, float aA1, float aA2) {
    return 3.0 * A(aA1, aA2)*aT*aT + 2.0 * B(aA1, aA2) * aT + C(aA1);
}

float CalcBezier(float aT, float aA1, float aA2) {
    return ((A(aA1, aA2)*aT + B(aA1, aA2))*aT + C(aA1))*aT;
}

float GetTForX(float aX, float mX1, float mX2) {
    float aGuessT = aX;
    for (int i = 0; i < 4; ++i) {
        float currentSlope = GetSlope(aGuessT, mX1, mX2);
        if (currentSlope == 0.0) return aGuessT;
        float currentX = CalcBezier(aGuessT, mX1, mX2) - aX;
        aGuessT -= currentX / currentSlope;
    }
    return aGuessT;
}

float KeySpline(float aX, float mX1, float mY1, float mX2, float mY2) {
    if (mX1 == mY1 && mX2 == mY2) return aX; // linear
    return CalcBezier(GetTForX(aX, mX1, mX2), mY1, mY2);
}
Copy the code

First we get four parameters from the Bezier curve editor, such as bezier-easing-editor or cubic-bezier:

The corresponding curve can be obtained by substituting these four numbers and independent variables. For example, we constructed a curve by ourselves:

Then substitute in.1,.96,.89,.17 to get the desired motion curve:

But when we pass in some special values like 0.99,0.14,0,0.27 we get a strange curve:

The desired curve is actually:

This is because the author did not take into account multi-angle tilting when implementing the transformation, and after his update we get more robust code: github.com/gre/bezier-… Again, I convert them to Shader functions:

float sampleValues[11];
const float NEWTON_ITERATIONS = 10.;
const float NEWTON_MIN_SLOPE = 0.001;
const float SUBDIVISION_PRECISION = 0.0000001;
const float SUBDIVISION_MAX_ITERATIONS = 10.;

float A(float aA1, float aA2) {
    return 1.0 - 3.0 * aA2 + 3.0 * aA1;
}

float B(float aA1, float aA2) {
    return 3.0 * aA2 - 6.0 * aA1;
}

float C(float aA1) {
    return 3.0 * aA1;
}

float getSlope(float aT, float aA1, float aA2) {
    return 3.0 * A(aA1, aA2)*aT*aT + 2.0 * B(aA1, aA2) * aT + C(aA1);
}

float calcBezier(float aT, float aA1, float aA2) {
    return ((A(aA1, aA2)*aT + B(aA1, aA2))*aT + C(aA1))*aT;
}

float newtonRaphsonIterate(float aX, float aGuessT, float mX1, float mX2) {
    for (float i = 0.; i < NEWTON_ITERATIONS; ++i) {
    	float currentSlope = getSlope(aGuessT, mX1, mX2);
        if (currentSlope == 0.0) {
            return aGuessT;
        }
        float currentX = calcBezier(aGuessT, mX1, mX2) - aX;
        aGuessT -= currentX / currentSlope;
    }
 	return aGuessT;
}

float binarySubdivide(float aX, float aA, float aB, float mX1, float mX2) {
    float currentX, currentT;
    
    currentT = aA + (aB - aA) / 2.0;
    currentX = calcBezier(currentT, mX1, mX2) - aX;
    if (currentX > 0.0) {
        aB = currentT;
    } else {
        aA = currentT;
    }
    
    for(float i=0.; i<SUBDIVISION_MAX_ITERATIONS; ++i) {
    	if (abs(currentX)>SUBDIVISION_PRECISION) {
            currentT = aA + (aB - aA) / 2.0;
            currentX = calcBezier(currentT, mX1, mX2) - aX;
            if (currentX > 0.0) {
                aB = currentT;
            } else{ aA = currentT; }}else {
            break; }}return currentT;
}

float GetTForX(float aX, float mX1, float mX2, int kSplineTableSize, float kSampleStepSize) {
    float intervalStart = 0.0;
    const int lastSample = 10;
    int currentSample = 1;
    
    for (int i = 1; i ! = lastSample; ++i) {if (sampleValues[i] <= aX) {
            currentSample = i;
            intervalStart += kSampleStepSize;      
        }
    }
    --currentSample;

    // Interpolate to provide an initial guess for t
    float dist = (aX - sampleValues[9]) / (sampleValues[10] - sampleValues[9]);
    
    float guessForT = intervalStart + dist * kSampleStepSize;

    float initialSlope = getSlope(guessForT, mX1, mX2);
    
    if (initialSlope >= NEWTON_MIN_SLOPE) {
      return newtonRaphsonIterate(aX, guessForT, mX1, mX2);
    } else if (initialSlope == 0.0) {
      return guessForT;
    } else {
      returnbinarySubdivide(aX, intervalStart, intervalStart + kSampleStepSize, mX1, mX2); }}float KeySpline(float aX, float mX1, float mY1, float mX2, float mY2) {
    const int kSplineTableSize = 11;
    float kSampleStepSize = 1. / (float(kSplineTableSize) - 1.);
    
    if(! (0. <= mX1 && mX1 <= 1. && 0. <= mX2 && mX2 <= 1.)) {
        // bezier x values must be in [0, 1] range
        return 0.;
    }
    if (mX1 == mY1 && mX2 == mY2) return aX; // linear
    
    for (int i = 0; i < kSplineTableSize; ++i) {
    	sampleValues[i] = calcBezier(float(i)*kSampleStepSize, mX1, mX2);
    }
    
    if (aX == 0.) return 0.;
    if (aX == 1.) return 1.;
    
    return calcBezier(GetTForX(aX, mX1, mX2, kSplineTableSize, kSampleStepSize), mY1, mY2);
}
Copy the code

Finally, we have the motion curve we want:

With bezier curves as a powerful tool, we can basically meet the needs of any animation rate. For us to achieve elegant natural transition effect to provide a strong guarantee.

Here are two giFs to feel the subtle sensory differences between the transitions caused by uniform and non-uniform motion (the first is uniform, and the second is Bessel curve, GIF will affect the final effect, but you can get a general feeling) :

Related links:

gist.github.com/gre/1650294
Greweb. Me / 2012/02 / bez…

Write a few thoughts on Shader transition

preface

First, material switching time

Two, transition effect switch

3. Material transition mode

Four, animation speed simulation

Related Posts

NGINX proxy static resources to solve the problem of refreshing page 404 in vue- Router mode=history caused content-Type to always be text/ HTML

What are the options for optimizing the first screen loading?

The URL is analysed