Review images

Canvas is a new “Canvas” element of HTML5, which allows us to use JavaScript to draw graphics. Currently, all major browsers support Canvas.

Review images

The most common use of Canvas is to render animation. The basic principle of rendering animation is simply to erase and redraw over and over again. For the sake of smooth animation, I was left with only 16ms to render a frame. In those 16ms, not only did I have to deal with some game logic and calculate the position and state of each object, but I also had to draw it all. If it takes a little more time, users will experience “lag.” So when I was writing animation (and games), I was constantly worrying about the performance of the animation, lest I should call an API too often and make rendering take too long.

To this end, I did some experiments, looked up some materials, sorted out some experiences of using Canvas in daily life, and summarized the so-called “best practices” in this area. I hope this article will be of some value to you if you and I have similar problems.

This article deals only with Canvas 2D.

Calculation and Rendering

To render a frame of the animation, go through the following steps:

  1. Computation: Handles the game logic, calculating the state of each object, without DOM manipulation (including of course operations on the Canvas context).
  2. Render: Actually draw the object. 2.1. JavaScript calls the DOM API (including the Canvas API) for rendering. 2.2. The process by which the browser (usually another rendering thread) renders the rendered result to the screen.

Review images

As mentioned earlier, we were left with only 16ms to render each frame. However, all we are really doing is steps 1 and 2.1 above, and step 2.2 is done by the browser in a different thread (at least that is the case with almost all modern browsers). The real premise for smooth animation is that all of the above work is done in 16ms, so the time consumed at the JavaScript level is best kept under 10ms.

Although we know that, in general, rendering is much more expensive than computation (3~4 orders of magnitude). Unless we use some time-intensive algorithm (which is discussed in the last section of this article), there is no need to delve into the optimization of the computation.

What we need to explore is how to optimize rendering performance. The general idea of optimizing rendering performance is very simple, which can be summarized as follows:

  1. Minimize the number of calls to render related apis per frame (usually at the expense of computational complexity).
  2. In each frame, the API with the lowest rendering overhead is called whenever possible.
  3. Within each frame, the rendering API is called in a way that “results in low rendering overhead” whenever possible.

The Canvas context is a state machine

The Canvas API is called on its context object.

var context = canvasElement.getContext('2d');
Copy the code

The first thing we need to know is that context is a state machine. You can change several states of the context, and almost all render operations, the final effect depends on the state of the context itself. For example, the width of the rectangle drawn by strokeRect depends on the context’s state lineWidth, which was set earlier.

context.lineWidth = 5; StrokeColor = 'rgba(1, 0.5, 0.5, 1)'; context.strokeRect(100, 100, 80, 80);Copy the code

Review images

At this point, it doesn’t seem to have anything to do with performance. I’m going to tell you now how you feel about the overhead of assigning to context.lineWidth being much higher than the overhead of assigning to a normal object.

Of course, it’s easy to understand. The Canvas context is not a normal object. When you call context.lineWidth = 5, the browser needs to do something immediately so that the next time you call an API like Stroke or strokeRect, The lines drawn are exactly 5 pixels wide (which is also an optimization, as you can imagine, because otherwise these things would have to wait until the next stroke and affect performance even more).

I tried the following assignment 106 times, and the result was that it only took 3ms to assign an attribute to a normal object and 40ms to assign an attribute to the context. It is worth noting that if you assign an invalid value, the browser takes some extra time to process the invalid input, as shown in the third/fourth case, which takes 140ms or more.

somePlainObject.lineWidth = 5; // 3ms (10^6 times) context.lineWidth = 5; // 40ms context.lineWidth = 'Hello World! '; // 140ms context.lineWidth = {}; // 600msCopy the code

For context, the assignment overhead for different properties is also different. LineWidth is just one of the less expensive types. The overhead of assigning to some of the other attributes of the context is summarized below.

attribute overhead Overhead (illegal assignment)
line[Width/Join/Cap] 40 + 100 +
[fill/stroke]Style 100 + 200 +
font 1000 + 1000 +
text[Align/Baseline] 60 + 100 +
shadow[Blur/OffsetX] 40 + 100 +
shadowColor 280 + 400 +

The overhead of changing the context state is relatively small compared to the actual drawing operation, since we haven’t actually started drawing yet. We need to understand that changing the properties of the context is not completely free. We can reduce the frequency of context state changes by properly arranging the order in which drawing apis are called.

Layered Canvas

Layered Canvas is very necessary in almost any situation where the animation area is large and the animation is complex. Layered Canvas can greatly reduce completely unnecessary rendering performance overhead. The idea of layered rendering is widely used in graphics-related fields: from the ancient shadow puppetry and color printing to the modern film/game industry, virtual reality, and so on. However, layered Canvas is only the most basic application of layered rendering idea in Canvas animation.

Review images

The starting point of layered Canvas is that each element (layer) in the animation has different requirements for rendering and animation. For many games, the frequency and magnitude of the main characters’ changes is large (they’re usually walking around and killing each other), while the frequency or magnitude of the background changes is relatively small (basically constant, slowly changing, or only occasionally changing). Obviously, we need to update and redraw people quite frequently, but for backgrounds we might only need to paint once, maybe every 200ms, definitely not every 16ms.

For Canvas, the ability to keep different redraw frequencies on each Canvas is the biggest benefit. However, hierarchical thinking solves much more than that.

The layered Canvas is also easy to use. All we need to do is generate multiple Canvas instances and place them on top of each other. Each Canvas uses a different Z-index to define the stacking order. Then redraw the layer only when it needs to be drawn (perhaps “never”).

var contextBackground = canvasBackground.getContext('2d');
var contextForeground = canvasForeground.getContext('2d');

function render(){
  drawForeground(contextForeground);
  if(needUpdateBackground){
    drawBackground(contextBackground);
  }
  requestAnimationFrame(render);
}Copy the code

Remember, the content on the top Canvas overwrites the content on the bottom Canvas.

The plot

By far the most used API on Canvas is drawImage. (Except, of course, if you’re writing a chart on Canvas, you won’t use a single sentence.)

The drawImage method has the following format:

context.drawImage(image, sx, sy, sWidth, sHeight, dx, dy, dWidth, dHeight);
Copy the code

Review images

Data source and rendering performance

Due to our ability to “draw a part of the image to the Canvas”, many times we will put multiple game objects in a single image to reduce the number of requests. This is often referred to as a “Sprite diagram”. However, there are actually some potential performance issues. I found that using drawImage to draw an area of the same size where the data source is an image of the same size is less expensive than if the data source is a larger image (we just saved the data). It can be argued that the cost difference between the two is the cost of a single operation, clipping.

I tried 104 times to draw a 320×180 rectangular area, which took 40ms if the data source was a 320×180 image, and 70ms if the data source was a 320×180 area cropped out of an 800×800 image.

Although it may seem like the overhead difference is not huge, drawImage is one of the most commonly used apis, and I think there is a need to optimize it. The idea of optimization is to do the “cropping” step in advance, save, each frame only draw not cropping. More on this in the “Off-screen Drawing” section.

Out of sight drawing

Sometimes, Canvas is just a “window” of the game world. If we draw the whole world in every frame, there will be a lot of things drawn outside the Canvas. Drawing API is also called, but it has no effect. We know that determining whether an object is in the Canvas involves additional computational overhead (such as inverting the global model matrix of a game character to factor out the object’s world coordinates, which is not a particularly cheap overhead) and increases the complexity of the code, so the question is, is it worth it?

I did an experiment and drew a 320×180 image 104 times. When I drew inside the Canvas each time, it took 40ms, while when I drew outside the Canvas each time, it only took 8ms. Think about it, and considering that the overhead of computation is two or three orders of magnitude different from the overhead of drawing, I think it’s still necessary to filter out what objects are outside the canvas through computation.

Off-screen drawing

As mentioned in the previous section, when drawing the same area, performance is better if the data source is a similar-sized image, and worse if the data source is part of a larger image, because each drawing also involves clipping. Maybe we can crop out the area to draw first and save it so that each time we draw it, it will be much easier.

The first argument to the drawImage method can receive not only an Image object but also another Canvas object. Also, the overhead of drawing with a Canvas object is almost identical to the overhead of drawing with an Image object. We just need to implement drawing objects on a Canvas that is not inserted into the page, and then drawing each frame using that Canvas.

Var canvasOffscreen = document.createElement('canvas'); canvasOffscreen.width = dw; canvasOffscreen.height = dh; canvasOffscreen.getContext('2d').drawImage(image, sx, sy, sw, sh, dx, dy, dw, dh); Context. DrawImage (canvasOffscreen, x, y); // drawImage(canvasOffscreen, x, y);Copy the code

The benefits of off-screen painting go beyond that. Sometimes the game object is drawn with multiple drawImage calls, or it’s not an image at all, but a vector shape drawn using a path. Off-screen drawing also helps you simplify these operations into a single drawImage call.

When I first saw the getImageData and putImageData apis, I had the illusion that they were designed for this scenario. The former can save a certain area on a Canvas as ImageData object, and the latter can redraw the ImageData object on the Canvas. But in reality, putImageData is an extremely expensive operation that should not be invoked in every frame at all.

Avoid “blocking”

“Blocking” can be understood as JavaScript code that runs continuously for more than 16ms, and JavaScript code that “causes the browser to take more than 16ms to process.” Even on a non-animated page, blocking is immediately apparent to the user: blocking makes objects on the page unresponsive — buttons can’t be pressed, links can’t be opened, tabs can’t even be closed. On pages with a lot of JavaScript animation, blocking will stop the animation for a while, and not resume execution until the blocking is restored. If there are frequent “minor” blocks (such as these optimizations mentioned above that take more than 16ms to render a frame), then “frame loss” will occur.

CSS3 Transition and Animate are not affected by JavaScript blocking, but are not the focus of this article.

Review images

Occasional and small blocks are acceptable, frequent or large blocks are not. In other words, we need to solve two types of blocking:

  • Frequent (usually minor) blocking. The main reason for this is the high rendering performance overhead, with too many things to do in each frame.
  • Large (though occasionally) blockages. The main reasons are running complex algorithms, large-scale DOM operations, and so on.

For the former, we should carefully optimize the code, sometimes making the animation less complex (and cool), as the optimization solution in the previous sections of this article addresses.

For the latter, there are mainly the following two optimization strategies.

  • Using the Web Worker, perform the calculation in another thread.
  • The task is divided into several smaller tasks and inserted into multiple frames.

Web Workers are great things, with good performance and good compatibility. The browser uses another thread to run the JavaScript code in the Worker without blocking the main thread at all. Animations (especially games) inevitably have some algorithms with high time complexity, and Web workers are perfect for running them.

Review images

However, the Web Worker cannot manipulate the DOM. So, sometimes we use another strategy to optimize performance, which is to break tasks into smaller tasks and insert them into each frame. While doing so would almost certainly make the total time to execute the task longer, at least the animation wouldn’t get stuck.

Review images

Looking at the Demo below, our animation moves a red div to the right. This is done in the Demo by changing its transform property every frame (the same goes for Canvas drawing).

I then created a task that would block the browser: get the average of 4×106 math.random (). Click the button and the task is executed, with the results printed on the screen.

Review images

As you can see, if you perform this task directly, the animation will “freeze” significantly. Using a Web Worker or splitting tasks does not stall.

The above two optimization strategies have the same premise, that is, the task is asynchronous. That is, when you decide to start a task, you don’t need to know the outcome immediately (in the next frame). For example, even if a user action in a strategy game triggers a pathfinding algorithm, you can wait a few frames without the user knowing it before moving your game character. In addition, splitting tasks to optimize performance introduces a significant increase in code complexity, as well as additional overhead. Sometimes I think it might be a good idea to prioritise slashing requirements.

summary

That’s where the body ends, and we’ll conclude with a little summary of the “best practices” to follow in most cases.

  1. Shift the overhead of the rendering phase onto the calculation phase.
  2. Use multiple layered Canvas to draw complex scenes.
  3. Do not set the font property of the drawing context too often.
  4. Do not use the putImageData method in an animation.
  5. Through calculation and judgment, avoid unnecessary drawing operations.
  6. Predraw fixed content on the off-screen Canvas to improve performance.
  7. Use Worker and split tasks to avoid complex algorithms blocking the animation.