Rendering principle

  • The App interface we see is jointly computed and processed by CPU and GPU.
  • CPU internal pipeline structure has parallel computing capability, generally used to display the content of the calculation. The GPU, on the other hand, has stronger parallel computing capability and can display graphics results in screen pixels through calculation. The process of converting graphic data in memory to display on screen is called rendering. And it’s the GPU that’s responsible for doing that.
  • In the process of rendering, GPU needs to process every pixel on the screen and ensure that the update of these pixels is smooth, which requires very high parallel computing capability of GPU.
  • In the early days, Graphics rendering was accomplished by VGA(Video Graphics Array). With the need of 3D acceleration, technologies such as triangle generation, rasterization and texture mapping were introduced. The processors that handle this set of technologies are collectively known as gpus.
  • The GPU’s main job is to convert 3D coordinates into 2D coordinates, and then convert 2D coordinates into actual pixels, Concrete implementation can be divided into the vertex shader (points) to determine the shape, shape, assembly (determine the shape of the line), geometry shader (determine the number of triangles), rasterizer (determine screen pixels), the fragment shader (to the pixel shader), test and hybrid (check depth and transparency to mix) six stages.
  • In order to control the computation of GPU more conveniently, the programming ability of GPU has been continuously strengthened, and it began to support C and C++ languages. It is easier to manipulate gpus through libraries defined by the OpenGL standard.
  • In the rendering process, CPU is specially used to deal with the calculation of rendering content, such as view creation, layout, image decoding, etc. After the content calculation is completed, it is transmitted to GPU for rendering.
  • In this process, the combination of CPU and GPU can make full use of mobile phone hardware to improve users’ experience of using App. Of course, during this process, if the CPU takes longer to compute than the screen refresh rate requires, the interface will not operate smoothly

Native rendering

  • The process of updating and rendering the native interface can be divided into the following four steps.
  • The first step is to update the view tree and update the layer tree simultaneously.
  • In the second step, the CPU calculates what to display, including view creation (setting Layer properties), layout calculation, view rendering (creating Layer Backing Image), and Image decoding conversion. When Runloop is in BeforeWaiting and Exit, the registered listeners are notified, the layers are packaged, and the packaging data is sent to Render Server, a separate process responsible for rendering.
  • In the third step, the data will be deserialized after reaching the Render Server to obtain the layer tree, filter the shielded part of the layer according to the layer order, RGBA value and layer frame in the layer tree, then transform the layer tree into a rendering tree, and the information of the rendering tree will be transferred to OpenGLES/Metal. These are collectively called Commit Transactions.

Why does it feel like WebViews and React Native render slower than Native?

From the first load of content

  • Even when loaded locally, the big front end has more scripting to parse than native.
  • Webviews require additional parsing of HTML+CSS+JavaScript code, while React Native solutions require parsing of JSON+JavaScript. HTML+CSS is more complex than JSON, so parsing is slower than JSON. That is, the WebView will be slower than the React Native class when the first content loads.

From the interpretation performance of the language itself

  • Interface updates after large front-end loads are performed through JavaScript interpretations, which perform worse than native implementations, especially when interpreting complex logic or large amounts of computation. So,
  • The big front end is much slower than the native.
  • In addition to the time it takes to load and parse the first time, and the performance problems caused by the slow interpretation of the JavaScript language itself, the WebView’s rendering process is separate, and each frame update calls the GPU process through IPC. Frequent IPC process communication also has a performance cost.
  • The WebView’s individual renderer cannot access the GPU context yet, so there is no way for the two processes to share texture resources. Texture resources cannot be rasterized using THE GPU Context directly, so they can only be transmitted to the GPU process through IPC, which results in GPU unable to give full play to its performance advantages. As WebView rasterization cannot be synchronized to GPU in time, it is easy to appear a white screen when sliding, which is difficult to avoid.
  • After talking about big front-end rendering, you will find that there is a performance gap between Webview and React Native due to the performance problems of scripting language itself, compared with Native rendering. How does Flutter render differently, and how does it perform, with a framework that doesn’t use a scripting language and has a new rendering engine?

Flutter rendering

  • The Flutter interface is made up of widgets. All widgets make up the Widget Tree. When the page is updated, the Widget Tree is updated, then the Element Tree is updated, and finally the RenderObject Tree is updated.
  • The subsequent rendering process of Flutter includes Build, Wiget Tree, Element Tree, RenderObject Tree, Layout, Paint, Composited Layer, etc. Combine layers, generate textures, and submit rendering content to GPU for rasterization and composition using OpenGL interface. This is the C++ Layer of Flutter, using Skia library. After submitting to the GPU process, the process of compositing and displaying the screen is basically similar to iOS native, so the performance is similar.
  • Flutter’s main advantage is that it can run on both Android and iOS platforms. However, with the introduction of SwiftUI and Preview at WWDC 2019, the advantages of Flutter in interface writing and Hot Reload will gradually diminish.

Refer to the article

  • IOS tips for keeping the interface smooth