preface

This is an article about the view rendering process and performance optimization for iOS, from WWDC video.

The body of the

First, view rendering

The view rendering hierarchy is shown below:



UIKit is a commonly used framework that displays and animates through CoreAnimation.

CoreAnimation is the CoreAnimation, which relies on OpenGL ES for GPU rendering. (at present, the latest iPhone already uses Metal.In order to be consistent with the picture and text, OpenGL ES is continued to be used for description), CoreGraphics does CPU rendering;

The lowest level of GraphicsHardWare is GraphicsHardWare.

The overall process of view rendering is as follows:

Rendering a view to the screen requires the CPU and GPU to work together. App will call CPU to preprocess part of the data through CoreGraphics and CoreImage, and finally transfer the data to GPU through OpenGL ES, and finally display to the screen.

Second, rendering process



The rendering process can be described in the image above:

  • 1. CoreAnimation submits the session, including the layout state of itself and view Hierarchy;
  • RenderServer parses the submitted subtree state and generates the rendering instructions;
  • 3. GPU executes drawing instruction;
  • 4. Display rendered data;

The App Commit process is divided into Layout, Display, Prepare, and Commit steps.

1. Layout

Call the layoutSubviews method; Call the addSubview: method;

Causes CPU and I/O bottlenecks;

2. Display

Draw a view with drawRect; Draw string (string);

Causes CPU and memory bottlenecks;

Each UIView has a CALayer, and the layer has a pixel storage space for the view; When -setNeedsdisplay is called, only the layer is set to dirty. When the rendering system is ready, the -display method of the view is called, pixel storage space is assembled, a CoreGraphics context (CGContextRef) is established, the context is pushed into the context stack, and the drawing program enters the corresponding memory storage space.

UIBezierPath *path = [UIBezierPath bezierPath];
[path moveToPoint:CGPointMake(10, 10)];
[path addLineToPoint:CGPointMake(20, 20)];
[path closePath];
path.lineWidth = 1;
[[UIColor redColor] setStroke];
[path stroke];
Copy the code

By implementing the above code in the -drawRect method, UIKit places the automatically generated CGContextRef on the context stack. When drawn, the pixels of the view are rendered onto the screen; The next time the view’s -setNeedsdisplay is called, the -drawRect method is called again.

3. They are prepared to submit the report.

Decode the picture; Picture format conversion;

When we use UIImage, CGImage, the image is not really decoded. IOS creates objects with some basic image information, then creates bitmaps and decodes them when they are actually used. Avoid using image formats that do not support hard resolution, such as WebP.

4. Commit

Package layers and send them to the render server. Recursively submit subtree layers; If the subtree is too complex, it will consume a lot and affect performance;

Simplify viewTree as much as possible;

When displaying a UIImageView, Core Animation creates an OpenGL ES texture and ensures that the bitmap in this layer is uploaded to the corresponding texture. When you override the -drawinContext method, Core Animation will ask you to allocate a texture and make sure that Core Graphics puts whatever you draw in the -drawinContext into the bitmap data of the texture.

Tile-Based rendering

Tile-based rendering is the mainstream of mobile devices. The entire screen is decomposed into N*Npixels Tiles, which are stored in the SoC cache (SoC= System on chip). Tiles are designed to realize a complex system function on the entire chip, such as Intel CPU, integrated display, memory controller, CPU operation core, cache, Queue, non-core, and I/O controllers). The geometry is broken up into tiles. For each tile, the necessary geometry is submitted to OpenGL ES and rendered (rasterized). When finished, the tile data is sent back to the CPU.

Transferring data is very performance – intensive. Multiple computations are relatively more economical and efficient than sending data multiple times, but the additional computations also incur some performance costs. PS: Controlling frame rate at an appropriate level on mobile platform can save power, effectively prolong battery life, and relatively improve user experience.

1. Rendering process

The normal tile-based rendering process is as follows:

CommandBuffer, accept OpenGL ES processing finished render instructions;

2. Tiler, call vertex shader, divide vertex data into blocks (Tiling);

ParameterBuffer: accepts tiles that are partitioned and corresponding render parameters.

4. Renderer: invokes the chip shader for pixel rendering;

5. RenderBuffer, which stores rendered pixels;

2. Off-screen Rendering — Mask



1. Mask texture of rendering layer, same as tile-based basic rendering logic;

2. Content texture of rendering layer, same as basic tile-based rendering logic;

Compositing operation, merge textures 1 and 2;

3. Off-screen Render — UIVisiualEffectView

With UIBlurEffect, it should be as small a view as possible because of the performance drain.

On a 60FPS device, it only takes 16.67ms per frame to process.

4. Render wait

Because the vertex and pixel processing in each frame are relatively independent, iOS arranges CPU processing, vertex processing, and pixel processing in three adjacent frames. As shown, when a render command is submitted, the render result is not displayed until 3 frames after the render.

5. Rasterization

To render and cache the contents of a view, we can turn on rasterization via CALayer’s shouldRasterize attribute. Note that rasterized elements are limited to 2.5 times the total size of the screen. When updating content, off-screen rendering will be enabled, so the update cost is high and can only be used for static content; And if the rasterized element 100ms is not used it will be removed, so rasterization of uncommon elements will not optimize the display.

6. Group transparency

The allowsGroupOpacity property of CALayer, and the alpha property of UIView is the same as CALayer opacity property.

当GroupOpacity=YESWhen, transparency is not considered at first, and transparency is calculated uniformly after all layers (itself + sublayers) are drawn.

Suppose A view A has A word view B, and they both have alpha of 0.5 (root view is black, A and B are white), when we draw the view:

If group transparency is not enabled, first draw view A (0.5 white), then draw view B. When drawing view B, overlay 0.5 white of view B on the basis of parent view 0.5 white and root view 0.5 black, and finally get 0.75 white.

If group transparency is enabled, first draw view A (white), then draw view B (white) directly on the basis of VIEW A, and finally calculate transparency 0.5 uniformly, so A and B keep the same color. (The boundary is deliberately added to distinguish view B)

The default value is read from The Boolean UIViewGroupOpacity property in The main bundle’s info.plist file. If no value is found, the default value is YES for apps linked against the iOS 7 SDK or later and NO for apps linked against an earlier SDK. In order to maintain the same transparency and optimize performance of the child view as the parent view, this feature has been enabled globally by default since iOS 7. For today’s developers, it’s almost a no-no.

Fourth, performance optimization

Here’s what THE WWDC recommends:

1. What is the frame rate?

60 frames per second; (The TimeProfiler tool can view the elapsed time)

2. Are there CPU and GPU bottlenecks? (View share)

Less use of CPU and GPU can effectively save battery;

3. Do you use extra CPU for rendering?

Overwriting drawRect causes CPU rendering; In most cases, the GPU is in a waiting state when the CPU is rendering.

4. Is there too much off-screen rendering?

Less is better; Off-screen rendering will cause context switch and GPU will generate IDLE.

5. Are too many views rendered?

Fewer views is better; Views with transparency of 1 are preferred;

6. Using weird image formats and sizes?

Avoid format conversion and resizing images; An image that is not supported by the GPU needs to be converted by the CPU. (Xcode has special algorithm optimization for PNG images)

7. Are expensive special effects used?

View effects consume, adjust the appropriate size; For example UIBlurEffect mentioned earlier;

8. Are there unnecessary elements in the view tree?

Understand the necessity of all points on the view tree and remove unnecessary elements; It is common to forget remove views, especially when the View analogy is large.



Above, are the tools corresponding to the 8 questions. Encounter performance problems, firstAnalyze and locate problemsInstead of diving into a sea of code.

5. Performance optimization examples

1, the shadow



The above method will result in off-screen rendering; The following is the right thing to do.

2, rounded corners,



Do not use unnecessary mask, can preprocess the image to be round; Or add a white background with a transparent circle in the middle. Even adding additional views results in additional computation; But it’s still a little faster, because gpus are better at rendering than context switching.

Off-screen rendering results in less than 100% GPU utilization with low frame rates. (Idle time is generated when context is switched)

3, tools,

Using instruments’ CoreAnimation tool to check the off-screen rendering, yellow is an undesirable color.

Use a real machine for debugging, because the simulator uses an OSX CALayer, not an iOS CALayer. If you debug with the simulator, you’ll see that all the views are yellow.

conclusion

So this is a good example of using View rendering for iOS, using CALayer in CA is two Triangles.

IOS has made a good encapsulation of the underlying rendering. The excellent development environment allows us to directly use UIKit for view operations and CoreAnimation to complete complex animations without knowing the complex process of view rendering, let alone the tile-based architecture of GPU. However, we should not be satisfied with the use of API, but should understand the complex operation rules and design principles behind it.

Because WWDC video content is older, it still uses OpenGL ES (this part has been replaced by Metal now), but this does not affect our learning of the whole view rendering process on mobile.

The appendix

TileBasedArchitectures PDF Introduction to TileBasedArchitectures