Off-screen rendering

What is off-screen rendering

  • Ordinary rendering

In general, the system places bitmaps that need to be displayed directly into the frame buffer and then displays them layer by layer according to the oil painting algorithm

Oil painting algorithm

All rendering operations are performed by CoreAnimation’s Render Server module by calling the OpenGL/Metal interface provided by the graphics card driver. Generally, for each layer, the Render Server will follow the “painter algorithm” and output to the frame buffer in sequence, which will overlay the previous layer (discard the previous layer) to get the final display result

  • Off-screen rendering

Although GPU can output layer by layer to the canvas, as the frame buffer is deleted immediately after it is used up, it cannot go back to change a part of it, so it can complete some more complex and multiple modification/clipping operations with the help of the temporary offScreen buffer.

Causes (For example, rounded corners)

For iOS developers, rounded corners cause off-screen rendering, but it’s not clear why, and does rounded corners always cause off-screen rendering?

case

Once the simulator turns on color offscreen-Rendered, those layers that need to be rendered off-screen will be highlighted yellow, which means the yellow layers may have a performance problem.

` / / 1. The button is the background image UIButton * btn1 = [UIButton buttonWithType: UIButtonTypeCustom]; btn1.frame = CGRectMake(100, 30, 100, 100); btn1.layer.cornerRadius = 50; [self.view addSubview:btn1]; [btn1 setImage:[UIImage imageNamed:@"btn.png"] forState:UIControlStateNormal]; btn1.clipsToBounds = YES; / / 2. The button does not exist the background image UIButton * btn2 = [UIButton buttonWithType: UIButtonTypeCustom]; btn2.frame = CGRectMake(100, 180, 100, 100); btn2.layer.cornerRadius = 50; btn2.backgroundColor = [UIColor blueColor]; [self.view addSubview:btn2]; btn2.clipsToBounds = YES; //3.UIImageView sets image + background color; UIImageView *img1 = [[UIImageView alloc]init]; img1.frame = CGRectMake(100, 320, 100, 100); img1.backgroundColor = [UIColor blueColor]; [self.view addSubview:img1]; img1.layer.cornerRadius = 50; img1.layer.masksToBounds= YES; img1.image = [UIImage imageNamed:@"btn.png"]; //4.UIImageView sets only the image, no background color; UIImageView *img2 = [[UIImageView alloc]init]; img2.frame = CGRectMake(100, 480, 100, 100); [self.view addSubview:img2]; img2.layer.cornerRadius = 50; img2.layer.masksToBounds = YES; img2.image = [UIImage imageNamed:@"btn.png"];Copy the code

Btn1 and IMg1 trigger off-screen rendering. The reason is that BTN1 is a blend of its layer and UIImageView layer (UIButton has imageView), so rounded corners trigger off-screen rendering. Img2 setting a cornerRadius and masksToBounds will not trigger off-screen rendering. Setting imG2 with a corner color and masksToBounds will trigger off-screen rendering.

According to the example, it can be concluded that the off-screen rendering will not be triggered only when the control is set with rounded corners or (rounded corners + clipping), and the off-screen rendering will be triggered only when the parent layer needs to be clipped and the child layer also needs to be clipped because the parent layer has rounded corners (i.e. the view contents have contents and multiple image layers are clipped)

Analysis of the

CALayerstructure

CALayer consists of backgroundColor backgroundColor, content contents, borderwide & bordercolor

cornerRadius The role of

The cornerRadius documentation clearly states that the cornerRadius setting only applies to CALayer backgroundColor and borderWide & BorderColor, if the contents have content or if the background of the content is not transparent, This works only if masksToBounds is set to true

Setting the radius to a value greater than 0.0 causes the layer to begin drawing rounded corners on its background.by Default, the corner radius does not apply to the image in the layer’s contents property; it applies only to the background color and border of the layer. However, setting the masksToBounds property to true causes the content to be clipped to the rounded corners.

Rounded corners cause off-screen rendering

When a background or layer set on the borders and masksToBounds = YES, set the rounded, backgroundColor/borderWidth&borderColor and content ` s ` need to perform the rounded operation, after superposition merger finally displayed on the screen. There are multiple processing operations in this process, and the rendering line cannot find an algorithm that can complete the rendering in a single traversal, so the data cannot be directly passed to the ‘frame buffer’. The ‘backgroundColor’/’ borderWidth ‘&’ borderColor ‘and’ contents’ are rounded separately and stored in the ‘off’ Screen Buffer, which is then synthesized and passed to the frame Buffer. Triggers an off-screen rendering. As you can see from the figure, only the four vertex regions of the UIImageView are rendered off-screen.

Note: Content refers not only to images, but also to images, drawn content, subviews with image information, etc

Summary of off-screen rendering reasons

If the rendering picture is more complex, according to the normal process of rendering

  1. backgroundColorThe rendered bitmap first goes into the frame buffer -> screen, the frame bufferbackgroundColorIt’s going to be emptied
  2. contentThe bitmap then enters the frame buffer -> screen, frame buffercontentBe cleared.
  3. When you cutcontentYou can’t find the content

Therefore, we need to create an extra buffer, wait for compositing and clipping to be completed before submitting to the frame buffer and finally displaying on the screen, resulting in an off-screen rendering

Off-screen rendering process

RenderServer goes through the process from GPU to display

  1. First render the Texture. The Texture can be interpreted ascontentThe content, in imageView, is the image.
  2. Render background color
  3. The texture and background are combined in the open off-screen render buffer
  4. submitframe buffer-> Screen display

The pros and cons of off-screen rendering

  • advantages

    • Data that appears multiple times on the screen can be rendered in advance for reuse, thus reducing CPU/GPU duplication
    • Special motion effects require multiple layers and an off-screen cache to hold intermediate states, forcing the use of off-screen rendering

  • disadvantage

    • Off-screen rendering requires context switching and opening up new memory, all time-consuming CPU operations
    • If the off-screen rendering results in the final frame storage exceeding 16.67ms, frames will drop, resulting in stuttering

When an off-screen rendering will occur

cornerRadius +masksToBounds

CornerRadius +masksToBounds will not always trigger an off-screen rendering, the key is to look at several layers of render data: backgroundColor, content(image, text, etc.). Off-screen rendering occurs only when content is set and the background is not transparent.

Frosted glass blur effect

Bitmaps can’t be placed directly in the frame buffer to wait for display. Instead, render data after blurring -> frame buffer -> display.

shadow

Shadow is a rectangle, which is the background of layer. According to the principle of canvas, it must be rendered to the screen first. Shadow is derived from layer, so the size and position of shadow can be determined only after the layer is known. Layer does not, it is impossible to render shadow first, so we can only use the off-screen rendering buffer, wait for shadow, layer and other rendering and merging to complete, and then send it into the frame cache for display.

ShouldRasterize rasterizer

ShouldRasterize is a CALayer attribute that transforms the final render of layer, including shadows, cuts, etc. into a bitmap and puts it in the off-screen buffer for reuse

  • ShouldRasterize precautions
    • If the layer is not static, it will be frequently modified. For example, if it is in animation, opening it will affect efficiency.
    • If layer cannot be reused, there is no need to turn on rasterization.
    • The off-screen buffer is time-limited and will be released if it is not in use for more than 100ms.
    • The size of the off-screen buffer cannot exceed 2.5 pixels of the screen, otherwise it will be released

Group Opacity group

Since the opacity is not 1, the blending operation of pixel color needs to wait for these several layers with different opacity to be rendered, and then submit to Screen Buffer to calculate the new color according to the opacity group, and finally submit the frame Buffer for display.

Use the masks

Masklayer, as a mask, may have transparency, shape (for example, to show the contents of a specified area), etc. The final Masked Image -> frame buffer can only be displayed when the Image and Mask are cropped and processed in the off-screen rendering buffer.