preface

I am taking the girl on the points, and the team battle is stuck, how should I explain to the girl? Online, etc.

It means “card”

In both mobile and mobile games, we all have a “card” from time to time, which can be defined in two ways:

  • Frame drop

  • Torn picture

So the question is, what causes all this? And how to solve it?

Frame drop

First, know what frame is and what frame rate is.

Frame is the smallest unit in the image animation of a single image picture, equivalent to every frame on the film. A frame is a still picture, and successive frames form an animation, such as a television image.

Frame rate (Frames Per Second) is simply the number of Frames of a picture transmitted in 1 Second. It can also be understood as the number of times a graphics processor refreshes Per Second, usually expressed as Frames Per Second

A frame is a still frame, and many frames together make up a video, movie, or game frame.

Frame rate refers to the number of frames drawn per second in Hertz (Hz).

The human eye considers continuity only when the number of frames is at least 10 to 12 frames per second. This phenomenon is called “visual transient phenomenon”, which is determined by the biological structure of the human eye. Through this phenomenon, the early silent films were able to play the picture quickly, by hand drive, as if they were playing a full continuous video.

According to our understanding, this frame rate is generally larger and more consistent, the less stuck. But at the same time, there are more costs, like more film for movies, better hardware for computers. Therefore, the general frame rate of movies is 24Hz, while the frame rate of computers and mobile phones is 60Hz, so that people can be comfortable to watch and use under normal conditions.

The game’s FPS was 60, but suddenly it dropped to 20, so there were only 20 frames per second.

So, what is the cause of frame drop?

In fact, we all know the reason, do not believe you continue to see…

Hardware reasons

“My phone is stuck playing games.”

“You that what broken mobile phone, hurry up to change a ~”

This conversation should happen from time to time, so we all know that the quality of the hardware is to some extent the difference between playing a game and not playing it properly.

Software for

“You this what App ah, do what game ah, so card, my mobile phone configuration is so high, play your card”

“Well, maybe the game wasn’t optimized well,”

The second reason is that the game/software itself is not optimized well, the image is made too big, the layout is nested too deep, then the frame calculation and rendering is more time-consuming, which may lead to the loss of frames.

Network reasons

“Not line, too card, I this ping are nearly 1000, how to play ah”

“Quickly change the flow ah, the group war is going to lose, how to fight less people”

By the way, the third reason is the network, which is the most common reason, the network fluctuation will affect the transmission of the picture, so the frame will drop.

Screen refresh mechanism

All three of these reasons are related to the basic mechanism of the screen refresh.

In a typical display system, whether mobile phone or computer, there are generally three parts:

  • CPUCentral processing unit. Used for computing data, information processing.
  • GPUGraphics processor. Used for processing image graphics, also known as the graphics card.
  • display, display screen. Used to display images, that is, our cell phone screens, our computer monitors.

The entire display process is:

  • CPUCalculate the data the screen needs and hand it to the GPU.
  • GPUThe image is processed and drawn, and then stored in the cache.
  • displayRead the data from this cache and display it.

Each frame repeats this work, which is 60 cycles per second, and the time required for each cycle is approximately 16.6ms. In Android, the screen refreshes every 16.6ms.

Here’s a classic picture of the screen refresh mechanism:

As can be seen, as soon as 16.6ms arrives, the system sends VSync signal, and then the screen obtains a new frame image from the cache area and displays it. At the same time, the CPU also starts to calculate the data of the next frame, and then delivers the data to the GPU, and finally puts it into the cache area and waits for the next VSync signal.

What is the VSync signal? We’re going to hit the no table, and we’ll talk about it later, but you can think of it as a synchronous refresh signal, synchronizing the CPU and the screen. When the signal comes, the screen starts to switch and the CPU starts the next frame.

To make it easier to understand, I made a little animation:

From the above explanation, we know that the display time of a frame is 16.6ms, during this time, the CPU and GPU must process the data and put it into the buffer.

If the CPU and GPU fail to draw the next frame in 16.6ms, then display cannot update the next frame, which is called frame drop.

So there are three reasons to drop frames. Let’s review:

  • 1. Hardware reasons. The hardware is poor, that is, the CPU and GPU are not effective in computing, leading to the data of a frame is not ready, so the frame is dropped.
  • 2. Software reasons. If the hardware is sufficient, one frame of data of App or game is too complicated to calculate, too much nesting or too large graph, which will also lead to the data of the next frame cannot be prepared in 16.6ms, resulting in frame loss.
  • 3. Network reasons. Under the normal condition of both hardware and software, CPU calculation data cannot be obtained from the network due to network fluctuation, which will definitely lead to CPU data preparation delay and eventually lead to frame drop.

So after dropping frames, what does the screen refresh mechanism do with subsequent frames?

  • If it isThe gameIn this case, because timeliness is important, the lost frames are not cared about any more, but are prepared to be displayed at the current time, and finally displayed on the screen. So in this case the lost frames are actually lost.
  • If it isapplicationTherefore, if frame 2 is dropped, the screen continues to display frame 1, while the CPU and GPU continue to prepare data for frame 2. If frame 2 is completed within the next 16.6ms, the screen continues to display frame 2. For example, sometimes when the mobile phone is stuck, when we try to operate the App, the operation will be delayed, that is, the frame will be dropped. In this case the frame is not actually dropped, but delayed.

Torn picture

Let’s take a look at the rip. Why are there two frames in one frame?

First understand the concept of “progressive scanning”

“Progressive scanning” is that the display screen is not “rubbing” a picture, but from top to bottom line displayed line by line, but is displayed faster so the naked eye can not see it.

As mentioned earlier, the screen data is taken from the Buffer. If the data in the Buffer changes during the screen data fetching and line-by-line scanning of the display, the screen may tear.

The most obvious example is this: the graphics card has an FPS of 180, while the monitor has an FPS of 60. That means the graphics card can produce 180 pictures a second, while the monitor can read only 60 pictures a second.

The display takes 1/60 of a second to read from the Buffer, but 1/180 of a second to scan, the graphics card saves the data from the next screen into Buffer, and the bottom half of the display displays the content of the second screen. And that’s what caused the tear.

Here’s another animation to illustrate:

To prevent this, display systems typically add a double cache + vSYNC concept:

  • First, turn it onVertical synchronization, will limit the GPU FPS to the same FPS as the display.

The system will send a vSYNC signal after the display has drawn a frame, and then the CPU and GPU will prepare the contents of the next frame and send another Vsync signal after the display has drawn the next frame. This repetition limits the FPS of the graphics card and allows the graphics to be drawn to monitor standards.

This VSync signal is called a VSync signal.

Those of you who play games know that many in-game Settings have an option to enable vSYNC to match the FPS of the graphics card to the FPS of the monitor to prevent the picture from tearing.

  • Secondly, throughDouble cacheEnsure the consistency of a frame of data.

1. Buffer backBuffer is used for CPU/GPU graphics processing. 2

When this is done, the screen reads only the contents of the framebuffer, which is a complete frame. A new frame calculated by the CPU/GPU will be placed into the backbuffer without affecting the contents of the framebuffer.

Only after the screen has drawn a frame will a new frame (backbuffer and framebuffer) calculated by the CPU/GPU be swapped.

This ensures the complete coherence of a frame of data.

In this case, we swap the memory addresses of two buffers, A and B, with A acting as the first framebuffer and B acting as the backbuffer. After the screen finishes drawing a frame, swap the AB memory address. When A new round of VSync signal comes, A acts as the backbuffer and B acts as the framebuffer.

Summary: When the screen finishes scanning frame 1, the system sends the VSync signal and three things happen:

  • 1. Exchange the contents of two buffer areas (framebuffer and backbuffer).
  • 2. The display begins to display the contents of frame 2, which is the swapped framebuffer.
  • 3. CPU/GPU starts to compute and process the content of the third frame, and puts the content in backbuffer after it is processed.

Here’s another animation:

Android Project Butter

Have you solved all the problems? Don’t.

With the addition of the VSync signal, the frame drop problem becomes worse:

It can be found that after the addition of VSync signal, although the CPU processing time is unified, the frame drop problem may be magnified again, from one frame drop to two frames drop directly. Because the 16.6ms in the middle is wasted.

What to do? Is it possible to maximize CPU/GPU utilization while retaining VSync signals?

Three caches are coming:

TripleBuffer is used for CPU/GPU graphics processing. 3. FrameBuffer is used for display

The reason for the situation just mentioned is that when the second VSync signal comes, the CPU cannot start the calculation of a new frame because the backBuffer is occupied by the GPU. A third buffer is added so that when the second VSync signal arrives, the CPU can start a new frame using the TripleBuffer, ignoring the backBuffer being occupied by the GPU.

CPU, GPU, and Display each have a cache, so all three can do their own work at the same time without affecting each other, maximizing the use of each module.

The three caches and the Vsync signal mentioned above were introduced as part of The Project Butter release for Android 4.1 to make Android Butter/cream smooth.

Finally, post a picture of the refresh mechanism under the three-cache mechanism:

summary

Today I learned about the refresh mechanism of The Android system. Although there is no code, it is often asked in interviews. Here is a summary again:

1. In order to solve the problem of picture tearing, VSync signal and double cache are introduced.

  • Every time the VSync signal arrives, the screen switches,CPU/GPUStart preparing for the next frame.
  • CPU/GPUEach time the data is prepared, put it into a separate cache, backBuffer, and when the screen is ready, put it intobackBufferData andframeBufferData exchange, screen only readframeBufferCache data, to ensure the integrity of data continuity.

2. In order to solve the problem that CPU/GPU cannot maximize utilization under VSync signal, three caches are introduced.

When the VSync signal comes, even if the GPU has not processed the last frame of data and the backBuffer is not idle, the CPU can use the third cache to normally start processing the next frame of data, maximizing the UTILIZATION of CPU/GPU and ensuring that the VSync mechanism does not waste resources.

3. The root cause of frame loss is that the CPU/GPU cannot prepare the data of the next frame within one frame (generally 16.6ms).

Even with the reference to triple cache and vertical synchronization, frame dropping still happens. As App developers, what we can do is to optimize the layout as much as possible, reduce nesting, reduce the time for CPU/GPU to calculate picture data, and make the next frame of image data properly prepared within each frame.

How does the refresh mechanism work in Android source code? This comes down to the Choreographer class parsing.

reference

Why is vSYNC not so popular among gamers