When it was leaked four years ago that SONY wanted to make a new Spider-Man movie, one obvious word came up repeatedly: “Rejuvenate,” which translates as “rejuvenate.” At the time, SONY had produced five live-action Spider-Man films (the trilogy starring Tobey Maguire and the two Amazing Spider-Man movies starring Andrew Garfield), in addition to Homecoming. SONY executives want to reinvigorate the franchise with an animated film.
For the animated film, SONY turned to directors Phil Lord and Christopher Miller, who were then known in the industry for films like The Meatballs franchise and the Lego Movie. The two directors, with their wild ideas and wacky ideas, wanted the animation to feel “like stepping into a comic book.” They were also excited to be able to use storytelling in a way that live-action films couldn’t.
When I first saw spider-man: Into the Spider-Verse, I was amazed by its innovative visual dynamics. In contrast to the natural and smooth 3D animation of the past, spiderman: Into the Spider-Verse is moving closer to the comic book experience, making 3D animation 2D, not just 2D.
As I saw in the comment of an article: the different comic styles of various parallel universes are mixed, the dialog box, cartoon division and subtitles are retained, and the hand-drawn graffiti is especially strengthened to deliberately create the “naive and naive feeling” of this part. When these are combined with the magic of the split-point mirror, it is enough to make a person’s eyes dizzying. Let’s briefly analyze the highlights of the movie:
First, the innovation of visual expression
1. 2D animation simulation in one beat two
Not long after entering the main story, I believe that everyone who is used to watching 3D animation will have a slightly uncomfortable feeling… “Is there a hitch in this movie?”
Yes, it is. This is the first “abnormal” part of the film, but also the most special part of the film. Look closely and you can see that only the characters are stuck, while their surroundings are fluid, creating a subtle contrast. About this, a netizen in Zhihu gave us an answer:
Movies generally use the 24-frame per second format, that is to say, if it is a hand-drawn 2D animation, it needs to draw 24 consecutive pictures per second to make the animation in the screen smooth, which is also commonly known as “1 shoot 1” in animation.
But someone found me drawing the same action only 1,3,5,7…. The total number of frames is only 12, and then each frame stays twice as long, the audience will not be aware of it, but the animator can save half of the drawing time, this is commonly known in animation 1 shoot 2, this behavior was invented just to save cost. This is generally not done in 3D animation because the intermediate frames are automatically generated after the 3D character animator sets the key frames, so there is no need to save the intermediate frames.
1 beat 2 is the traditional practice of 2D animation, 3D animation forced to do so can give people the illusion of 2D animation texture, but at the same time, Sony chooses background animation, lens animation and other displacement animation to keep 1 beat 1, so as to maintain the unique advantage of 3D animation lens smooth.
Such processing technique on the one hand, the picture has a 2 d animation unique frame by frame by frame texture (see any lift like the childhood animation), on the other hand to produce a natural but a bit acosmia feeling, like characters and scenes is not in the same space and time, which makes the film’s visual expression and produce a kind of echo and unified story theme.
2. Cartoon techniques are restored
In the movie, we can see many techniques and effects commonly used in American comics. Such as BOOM, AHHHHH when jumping off a building, THWIP when spinning silk these hand-painted onomatopoeia; A dialog box describing the mental activity of the main character while running and before engaging in combat; Speed lines, warning signals of perceived danger, etc. All of this keeps reminding us of what it was like to read comics. Here’s how these representations are used in movies:
These techniques of expression make the film pictures more tense than those in graphic comics, which is related to their properties changed by years of history (the history of comics follows different paths of different cultures and has a lot of integration in the development, mainly from the perspective of American comics techniques). Let’s take the common dialog box in comics as an example.
Image is known to all become the medium of expression for human early time in the text, however, when people learn to use the continuous image to express a story that begins before, met to be sure, an image is unable to meet alone, then, has been to BC two thousand years ago in Egypt, they think of a way to a kind of can explain.
This is a mural of two men slaughtering hyenas in the tomb of Mereruka, ancient Egypt. Remarkably, in the space between the two characters float hieroglyphics that, translated, say of one to the other: “Hold it tight!”
The appearance of words helps the painter to enrich the information outside the image — to express the thought activities of the protagonist, and at the same time increase the tension of the picture.
Although these words are part of the whole, they are also independent of the image and have little relevance to the image. In Mesoamerican civilization around the 7th century BC, the use of abstract symbols to summarize language developed into a scroll or flag-like figure that appeared in Medieval and Renaissance Art in Europe. Perhaps due to the limitation of composition, the language involved in this way of dialogue is relatively simple, but at this time, the words can be better integrated into the picture.
Political caricatures have been popular since the 18th century, and there is a lot of dialogue in them. Cartoonists of this period were already familiar with the use of dialogs, some of which looked like simplified scrolls, some of which looked like balloon bubbles in front and some of which looked like balloons in front and some of which looked like modern dialogs. And these dialog boxes have been able to carry more text, to the picture played a certain role in the rendering atmosphere.
At the beginning of the 20th century, dialog box has played a key role in comics, and has also derived onomatopoeia, mood symbols and other comic techniques. They are not only conveying content and ideas, but also have the role of emotional expression, plot promotion, atmosphere rendering and so on.
Back to our movie. It is these techniques and elements that are common in graphic comics that make static comics “sound”, and because of these techniques, Spider-Man: Into the Parallel Universe enrichis the audience’s other senses at the visual level compared to ordinary 3D films. Looking back at the previous GIF, even if there is no sound, we can more specifically “hear” what is happening through our eyes.
3. No motion blur
In order to restore the manga reading experience, some of the techniques commonly used in 3D animation naturally have to be abandoned, such as motion blur. But remove the motion blur, and the smooth motion and speed of 3D animation becomes very inadequate, a decision that often has a direct impact on the style of the film.
This is what Dimian, the film’s special effects supervisor, said about the problem, which we knew we had to solve (without motion blur). We just didn’t want to solve it with anything that looked too traditional. We want to look for solutions and ideas from 2D animation. We were really looking for something that looked like it was inspired by animation, or a way of showing motion blur through smears, stretching geometry, but didn’t do it that way. Finally, some solutions are adopted. Imageworks’ line tools allow artists to draw moving lines that also connect shapes, as well as techniques inspired by camera shutters. For example, backgrounds on flat-bottom mirrors of fast cameras, either composited or aided by special effects, are heavily smeared and underlined, but appear in a very blocky illustration, Mr. Dimian said. We basically try to avoid anything that looks like smooth gradients or blurs because of motion blur. Instead, it’s a very graphical process.
As we can see later, the motion of the picture is supplemented by overlapping (camera shutter-inspired method), scene dislocation, speed lines, etc., and the stylistic language of the film itself is guaranteed.
4. Dispersion/specular overflow color
In fact, we can see the texture (specular color) that usually results from older or earlier shots, as well as some shot dispersion effects, which are also intentional by the studio.
Director Peter Ramsey explained in an interview: In a sense, you don’t have to pay attention to anything, because what we do is we restore the comics, and when we look at the comics, sometimes there are mistakes in the comics, sometimes the printing is not good, the colors are out of the line, so it looks blurry. And this is similar to live action movies, for example, sometimes things are blurred out of focus, so that the audience can focus on the important and clear images.
The effect of dispersion/specular color overflow here is to blur and blur. For example, the bad printing situation mentioned by the director often appeared in the printing industry in the early and mid-20th century. At that time, the CMYK 4-color printing would lead to the wrong printing of spray color due to the error of the printing plate. Such wrong printing also became a kind of stylized processing in the later development. This not only restores the texture of the cartoon to a better position and resonates with the audience, but also generates a sense of visual scene through such dislocation, which makes the film more layered and solves the problem that the film appears flat when it is close to 2D.
The film made many breakthroughs in visual expression. The creative team was bold enough to mix various characters with different styles, rationalized the concept of parallel universe, and achieved a high degree of unity through the combination of visual language and the theme of the whole film through the way of graphic comics. The whole film reveals a sense of unstable contradictions everywhere. The creative team did not weaken these contradictions, but showed them all. Such a sense of collision makes people admire their control and confidence for the whole.
Two, visual effect and filter implementation
The film is eager to demonstrate its stylistic innovations from the opening credits, but a closer look reveals that the effects are not zero-to-one innovations, but that SONY’s animation team has applied these visual elements to the fullest.
Since my work involved the compilation of OpenGL Shader code for short video filters and transitions, I paid special attention to the special effects in the movie when I was watching the movie for the second time and thought about what could be realized by Shader. Of course, the effects in movies are tweaked and reworked by the animators, not perfectly simulated by a piece of code. This is just a technical discussion of how some of the effects in the movie are possible at the Shader code level.
1. HalfTone filter
The film highly restores the viewing experience of comics, and the use of Ben-day dot in the rendering of the picture makes us feel the texture of reading a paper comic book. Ben-day Dot differs from HalfTone in that the ben-day dot is always the same size and distribution in a given area, while HalfTone can render the size and gradient of the dot according to the color detail of the image. We can use the HalfTone filter to create a similar texture. Here are some screenshots:
Ben-day dot, named after the illustrator and printer Benjamin Henry Day, Jr., son of the 19th-century publisher Benjamin Henry Day, A printing technique invented in 1879. According to the principle of color and optical illusion, it generates the desired effect through the spacing, size or overlap of small colored dots. For example, magenta dots that are widely spaced produce pink. Color comic books were popular in the 1950s and 1960s, but full-color comics were expensive, so comic book publishers used ben-Day Dot in four-color printing (cyan, magenta, yellow, black) to print out shadows and secondary colors like green, purple, orange, flesh, etc., Because of the characteristics of this method can save a lot of ink, comic book manufacturers through this cheap printing method printing on cheap paper to produce comic books, so as to obtain higher profits.
Later, this method was fully used in painting by the American pop art master RoyLichtenstein (1923 ~ 1997) and promoted it. Performance in the middle of the 20th century, abstract painting reached its zenith, he and the young generation of painters in New York, puts forward a new form of figurative painting – pop art, in the ’60 s, who rose to fame as the new art movement of the main moving, even many have affected by American business outside culture, and also feel the power of his work. Lichtenstein’s paintings or sculptures, quoting from comics, daily necessities, advertisements and the works of some famous artists in the history of art, combined with the characteristics of the American society at that time of the strong commercial culture and characteristics of The Times.
Let’s take a look at the implementation. In The Shader language of OpenGL, by creating an arranged circular pattern, combining the RGB channel color of the image with the half-tone pattern of the image, we can get the half-tone texture we want by performing the “lighten” image mixing mode with the original image:
precision highp float;
uniform sampler2D inputImageTexture;
varying vec2 textureCoordinate;
vec2 center = vec2(. 5.. 5);
float angle = 1.57;
float scale = 1.;
vec2 tSize = vec2(1000..563.);
float blendLighten(float base, float blend) {
return max(blend,base);
}
vec3 blendLighten(vec3 base, vec3 blend) {
return vec3(blendLighten(base.r,blend.r),blendLighten(base.g,blend.g),blendLighten(base.b,blend.b));
}
vec3 blendLighten(vec3 base, vec3 blend, float opacity) {
return (blendLighten(base, blend) * opacity + base * (1.0 - opacity));
}
float pattern(a) {
float s = sin( angle ), c = cos( angle );
vec2 tex = textureCoordinate * tSize - center;
vec2 point = vec2( c * tex.x - s * tex.y, s * tex.x + c * tex.y ) * scale;
return ( sin( point.x ) * sin( point.y ) ) * 4.0;
}
void main(a) {
vec4 color = texture2D(inputImageTexture, textureCoordinate);
float average = ( color.r + color.g + color.b ) / 3.0;
vec4 halftone = vec4( vec3( average * 10.0 - 5.0 + pattern() ), color.a );
gl_FragColor = color + vec4(. 02)*halftone;
gl_FragColor = vec4(blendLighten(color.rgb, halftone.rgb, .05), 1.);
}
Copy the code
Then use Layer Blend mode “Lighten” to make pixel fusion of the two images:
Here is a web Demo. Click the link and you can see the before and after effect: Demo1 through the filter switch in the upper right corner
1. 2. Smart water bottle
Glitch can also be found in many Internet products, most notably douyin’s LOGO animation and the filter effect in short videos. In the movie, atomic instability due to the fact that several other Spider-Man characters are from different parallel worlds has a similar glitchy effect.
SONY studios used a combination of hand-drawn graphics and multi-layer effects to create the fault effect. Of course, it’s not easy for the code to simulate this effect, but if you can find a suitable mask, you can probably approximate it. But here’s just a normal image-based failure effect (the code simulates the failure effect by doing pixel and color channel offsets based on time) :
precision highp float;
uniform sampler2D inputImageTexture;
uniform sampler2D inputImageTexture2;
varying vec2 textureCoordinate;
uniform float time;
float amount = 0.1;
float speed = 0.5;
float random1d(float n){
return fract(sin(n) * 43758.5453);
}
float random2d(vec2 n) {
return fract(sin(dot(n, vec2(12.9898.4.1414))) * 43758.5453);
}
float randomRange (in vec2 seed, in float min, in float max) {
return min + random2d(seed) * (max - min);
}
float insideRange(float v, float bottom, float top) {
return step(bottom, v) - step(top, v);
}
float rand(vec2 co){
return fract(sin(dot(co.xy ,vec2(12.9898.78.233))) * 43758.5453);
}
float blendDarken(float base, float blend) {
return min(blend,base);
}
vec3 blendDarken(vec3 base, vec3 blend) {
return vec3(blendDarken(base.r,blend.r),blendDarken(base.g,blend.g),blendDarken(base.b,blend.b));
}
vec3 blendDarken(vec3 base, vec3 blend, float opacity) {
return (blendDarken(base, blend) * opacity + base * (1.0 - opacity));
}
void main(a) {
vec2 uv = textureCoordinate;
float sTime = floor(sin(time*0.0005) * speed * 6.0 * 24.0);
vec3 inCol = texture2D(inputImageTexture, uv).rgb;
vec3 outCol = inCol;
float maxOffset = amount/4.0;
vec2 uvOff;
for (float i = 0.0; i < 10.0; i += 1.0) {
if (i > 10.0 * amount) break;
float sliceY = random2d(vec2(sTime + amount, 2345.0 + float(i)));
float sliceH = random2d(vec2(sTime + amount, 9035.0 + float(i))) * 0.25;
float hOffset = randomRange(vec2(sTime + amount, 9625.0 + float(i)), -maxOffset, maxOffset);
uvOff = uv;
uvOff.x += hOffset;
vec2 uvOff = fract(uvOff);
if (insideRange(uv.y, sliceY, fract(sliceY+sliceH)) == 1.0){ outCol = texture2D(inputImageTexture, uvOff).rgb; }}float maxColOffset = amount/6.0;
vec2 colOffset = vec2(randomRange(vec2(sTime + amount, 3545.0),-maxColOffset,maxColOffset), randomRange(vec2(sTime , 7205.0),-maxColOffset,maxColOffset));
uvOff = fract(uv + colOffset);
float rnd = random2d(vec2(sTime + amount, 9545.0));
if (rnd < 0.33) {
outCol.r = texture2D(inputImageTexture, uvOff).r;
} else if (rnd < 0.66) {
outCol.g = texture2D(inputImageTexture, uvOff).g;
} else {
outCol.b = texture2D(inputImageTexture, uvOff).b;
}
vec3 inCol2 = texture2D(inputImageTexture2, uv).rgb;
vec3 finalColor = blendDarken(outCol, inCol2);
gl_FragColor = vec4(finalColor, 1.0);
}
Copy the code
Again, you can see the before and after effect of Demo2 by using the filter switch in the upper right corner of the Demo
3. RGB Shift/Split RGB
It can be found carefully that the second Glitch effect above is also used for RGB color separation. In the film, a large number of such effects appear, and the color change is applied to the extreme, some magic and some unreal.
Shader is easier to implement than the above two filters, which can be achieved by splitting the RGB three colors of the image and coordinate offset:
precision highp float;
uniform sampler2D inputImageTexture;
varying vec2 textureCoordinate;
uniform float time;
float amount = 0.01;
float angle = 0.;
void main(a) {
vec2 offset = amount * vec2(cos(time*001.), sin(time*001.));
vec4 cr = texture2D(inputImageTexture, textureCoordinate + offset);
vec4 cga = texture2D(inputImageTexture, textureCoordinate);
vec4 cb = texture2D(inputImageTexture, textureCoordinate - offset);
gl_FragColor = vec4(cr.r, cga.g, cb.b, cga.a);
}
Copy the codeThe following two pictures compare the before and after effects:
Again, you can see the before and after effect of Demo3 by using the filter switch in the upper right corner of the Demo
In the end, there is too much to say about this movie, and there are too many amazing points to mention all of them. We can only briefly talk about the visual aspects and the possibility of realization. Welcome to add.
As I was finishing my article, Sony said “Rejuvenate” at the beginning. We can feel the film from the story and the style is filled with young people will like elements, now can say this is a pretty cool, but when you see a lot of information we can not help but have some regrets, the very cool effect and the elements of style in the last century is actually already exists and has deep historical trace, in a sense, Rather than being younger, it feels like a throwback. From this we also extend some questions:
Some people say that style is a reincarnation, so a look, the so-called “younger style” or “outdated style” are like a false proposition. In other words, are there no styles that are young or outdated, only styles that get tired over time?
Or is it because of technological innovation that the product (film or otherwise) is made younger, and has nothing to do with style? That sounds interesting. Think about it.
Related links:
- www.gcores.com/articles/10…
- Deadline.com/2018/12/son…
- www.zhihu.com/question/29…
- www.douban.com/review/9831…
- En.wikipedia.org/wiki/Ben-Da…
- www.bilibili.com/video/av768…
- Wapbaike.baidu.com/tashuo/brow…
- www.docin.com/p-92574055….