I hope that this article will review the content related to the principles of rendering in iOS from beginning to end. I will start with the principles of computer rendering, then slowly talk about the principles and frameworks of iOS rendering, and finally discuss the off-screen rendering in depth.

Hope to help you a little ~

In addition: my article Github collection address, welcome to pay attention to the public number: a melon technology

1. Principle of computer rendering

CPU and GPU architecture

For a modern computer system, it can be viewed as a three-tier architecture: hardware, operating system, and process. For mobile terminal, process is app, while CPU and GPU are important components of hardware. The CPU and GPU provide computing power and are invoked by the APP through the operating system.

  • Central Processing Unit (CPU) : the computing and control core of the entire modern computer system.
  • Graphics Processing Unit (GPU) : a dedicated microprocessor that can perform Graphics operations. It is the link between a computer and a display terminal.

CPU and GPU are designed for different purposes. They are designed for two different application scenarios. CPU is the computing core and the control core. It needs to have strong computing versatility and be compatible with various data types. At the same time, it also needs to be able to handle a large number of different instructions such as jump and interrupt, so the internal structure of CPU is more complex. On the other hand, GPU is faced with more pure operation of uniform type and does not need to deal with complex instructions, but it also shoulders a larger operation task.

Therefore, the ARCHITECTURE of CPU and GPU is different. As you can see from the figure above, the CPU has more Cache space and more complex control units. Computing power is not the main requirement of the CPU. The CPU is designed for low latency, and more caching means faster data access; At the same time, the complex control unit can process the logic branch more quickly, which is more suitable for serial computation.

However, GPU has more Arithmetic Logic units, stronger computing power, and more control units. GPU is designed for high throughput, and each part of the cache is connected to a stream processor, which is more suitable for large-scale parallel computing.

Image rendering pipeline

The image rendering process is roughly divided into the following steps in coarse granularity:

In the above image rendering pipeline, except for the first part of the Application stage, GPU is mainly responsible for the rest of the process. For the convenience of the subsequent explanation, the rendering flow chart of GPU is firstly shown:

The picture above shows the rendering pipeline that the GPU is responsible for when a triangle is rendered. As you can see, a simple triangle drawing requires a lot of computation. If you have more and more complex vertex, color, texture information (including 3D textures), the amount of computation is unimaginable. This is why gpus are better suited for rendering processes.

Next, the specific tasks of each part in the rendering pipeline are explained in detail:

Application processing phase: get the primibles

This phase refers specifically to the phase in which the image is processed in the application, which is still in the CPU’s charge. At this stage the application may perform a series of operations or changes to the image, eventually passing the new image information to the next stage. This part of information is called primitives, which are usually triangles, line segments, vertices, etc.

Geometry processing stage: processing primitives

Once you get to this stage, and beyond, the GPU is in charge. At this time, the GPU can get the information of the primiprimies passed down from the previous stage. The GPU will process these primiprimies and then output new primiprimies. This series of stages includes:

  • Vertex Shader: This stage converts Vertex information into perspective, adds light, adds texture, and so on.
  • Shape Assembly: Triangles, line segments, and points in primitives correspond to three, two, and one Vertex, respectively. This stage joins the Vertex into the corresponding shapes.
  • Geometry Shader: Adds additional Vertex to transform the original primitives into new primitives to build a different model. In simple terms, this is based on building more complex geometries through triangles, line segments and points.

Rasterization: Conversion of pixels from pixels

The main purpose of rasterization is to transform the primitized information after geometric rendering into a series of pixels for subsequent display on the screen. In this stage, the pixel information covered by each pixel is calculated according to the pixel information, so as to divide the pixel into different parts.

A simple partition is based on the center point. If the center point of a pixel is inside the pixel, then the pixel belongs to the pixel. As shown in the figure above, the dark blue line is the triangle created by the meta-information; By whether the center point is covered, all pixels belonging to the pixel can be traversed, that is, the light blue part.

Pixel Processing stage: Processing pixels to get a bitmap

After the rasterization phase above, we have the pixels corresponding to the primifiers. At this point, we need to fill these pixels with colors and effects. So the final stage is to fill the pixels with the right content and finally display them on the screen. These processed, information-rich collections of pixels are called bitmaps. In other words, the final output of the Pixel stage is the bitmap, and the process includes:

The dots can be arranged and dyed in different ways to form a pattern. When you zoom in on the bitmap, you can see the countless individual squares that make up the entire image. As long as there are enough pixels of different colors, a colorful image can be produced that can realistically represent the scene of nature. Scaling and rotation are easy to distort, and file capacity is large.

  • Fragment Shader: Also known as the Pixel Shader, the purpose of this stage is to give each Pixel the correct color. The source of the color is previously obtained information such as vertices, textures, lighting, etc. This is often a performance bottleneck due to the complexity of processing textures, lighting, and so on.
  • Tests and Blending: Also known as the Merging stage, this stage mainly deals with the front and rear positions of fragments and the transparency. In this stage, the depth z-coordinate of each colored segment is detected to determine the front and rear positions of the segment and whether it should be discarded. At the same time, the corresponding transparency alpha value is calculated, and the fragments are mixed to get the final color.

2. Screen imaging and lag

After the image rendering process is complete, the next step is to display the resulting pixel information on the physical screen. After the final rendering of the GPU, the pixel information is stored in the Framebuffer. After that, the Video Controller will read the information in the Framebuffer and pass it to the Monitor after digital-to-analog conversion for display. The complete process is shown in the figure below:

The set of pixels processed by the GPU, known as the bitmap, is cached by the frame buffer for later display. The display’s electron beam scans line by line from the top left corner of the screen, and the image information for each point on the screen is read from the bitmap in the frame buffer and displayed accordingly on the screen. The scanning process is shown in the figure below:

As the electron beam scans, the screen displays its results, rendering a complete frame of the image each time the entire screen is scanned. The screen is constantly refreshed, constantly presenting new frames, and a continuous image is displayed. The rate at which the screen is refreshed is called the Frame per Second (FPS). Due to the visual retention effect of the human eye, when the screen refresh rate is high enough (FPS is usually around 50 to 60), the picture will look continuous and smooth. For iOS, apps should be as good as 60 FPS.

Screen Tearing

In this single-cache mode, the ideal situation is a smooth pipeline: each time the electron beam scans a new frame from scratch, the CPU+GPU rendering process for that frame has finished, and the rendered bitmap has been placed in the frame buffer. But this perfect scenario is very fragile and can easily cause screen tearing:

CPU+GPU rendering process is a very time-consuming process. If the bitmap is not rendered when the electron beam starts to scan a new frame, but is rendered when it is in the middle of the screen, and is placed in the frame buffer —- then the scanned part will be the previous frame, and the unscanned part will show a new frame, which will cause screen tearing.

Vsync + Double Buffering

One strategy to solve screen tearing and improve display efficiency is to use Vsync and Double Buffering. According to apple’s official documentation, iOS devices will always use the Vsync + Double Buffering policy.

Vertical synchronisation (Vsync) is the equivalent of locking the frame buffer: when the beam has completed a frame and is about to start from scratch, it sends out a Vsync signal. Only when the video controller receives Vsync does it update the bitmap in the frame buffer to the next frame, ensuring that the same frame is displayed each time, thus avoiding screen tearing.

However, in this case, the video controller has to pass in the next frame bitmap after receiving Vsync, which means that the entire CPU+GPU rendering process has to be completed in a flash, which is obviously unrealistic. So the double-buffering mechanism adds a new standby buffer. The render results are stored in the back buffer, and upon receiving the Vsync signal, the video controller replaces the contents of the back buffer into the frame buffer, ensuring that the displacement is done almost instantaneously (in effect, swapping memory addresses).

Frame drop Jank

Enabling Vsync signals and double buffering solves the screen tearing problem, but introduces a new problem: frame dropping. If the CPU and GPU have not rendered the new bitmap by the time the Vsync is received, the video controller will not replace the bitmap in the frame buffer. The screen will then rescan and display the exact same image as the last frame. The same image is displayed for two cycles, which is called frame drop.

As shown in the figure, A and B represent two frame buffers. When B does not finish rendering, it receives the Vsync signal, so the screen can only show the same frame A again, which causes the first frame drop.

Triple Buffering

In fact, there is room for improvement. We notice that at the time of the frame drop, the CPU and GPU are idle for A period of time: when A’s content is being scanned and displayed on the screen, while B’s content has been rendered, the CPU and GPU are idle. So if we add a frame buffer, we can use this time for the next render and temporarily store the render results in the new frame buffer.

As shown in the figure, due to the addition of a new frame buffer, the frame loss period can be taken advantage of to a certain extent, and the CPU and GPU performance can be reasonably utilized to reduce the number of frames lost.

The nature of screen lag

The direct cause of cell phone lag is frame drop. As mentioned earlier, the screen refresh rate must be high enough to be smooth. On an iPhone, the maximum screen refresh rate is 60 FPS, and 50 FPS is generally a good experience. However, if there are too many frames dropped, resulting in a low refresh rate, it will make the use of the experience is not smooth.

From this point of view, it can be summarized roughly

  • The root cause of screen lag is that the CPU and GPU render pipeline takes too long, resulting in frame loss.
  • Meaning of Vsync and double buffering: Force synchronous screen refresh to solve screen tearing problem at the cost of frame dropping.
  • Three buffer meaning: reasonable use of CPU, GPU rendering performance, reduce the number of frames dropped.

3. Render framework in iOS

The rendering framework for iOS still conforms to the basic architecture of the rendering pipeline, and the specific technology stack is shown in the figure above. On the basis of hardware, there are Core Graphics, Core Animation, Core Image, OpenGL and other software frameworks in iOS to draw content and carry out higher-level encapsulation between CPU and GPU.

GPU Driver: These software frameworks are also dependent on each other, but all frameworks eventually connect to the GPU Driver through OpenGL. The GPU Driver is a block of code that communicates directly with the GPU and directly connects to the GPU.

OpenGL is an API that provides 2D and 3D graphics rendering. It works closely with the GPU to maximize the power of the GPU for hardware accelerated rendering. Efficient implementations of OpenGL (which take advantage of graphics-accelerated hardware) are typically provided by the display device manufacturer and are heavily dependent on the hardware provided by that manufacturer. A lot of things that extend on Top of OpenGL, like Core Graphics, end up relying on OpenGL, and in some cases, for efficiency, like game programs, they even call the OpenGL interface directly.

Core Graphics: Core Graphics is a powerful 2d Graphics rendering engine. It is the Core Graphics library of iOS. Commonly used Graphics such as CGRect are defined under this framework.

Core Animation: On iOS, almost everything is drawn using Core Animation, which gives you more freedom and can be used more widely.

Core Image: Core Image is a high-performance Image processing and analysis framework. It has a series of off-the-shelf Image filters, which can effectively process existing images.

Metal: Metal is a set of third-party standards similar to OpenGL ES, implemented by Apple. Core Animation, Core Image, SceneKit, SpriteKit and other rendering frameworks are all built on top of Metal.

What is Core Animation

Render, compose, and animate visual elements. —- Apple

Core Animation, which is essentially a composite engine, is responsible for rendering, building, and implementing animations.

Usually we use Core Animation to make animations efficient and easy, but its predecessor is actually called Layer Kit, and the implementation of animations is only part of its functionality. For iOS apps, whether or not Core Animation is directly used, it is deeply involved in the building of the app at the bottom level. For OS X apps, Core Animation can also be used to implement some functions conveniently.

Core Animation is the perfect underlying support for AppKit and UIKit. It is also integrated into Cocoa and Cocoa Touch workflows. It is the most basic infrastructure for app interface rendering and building. Core Animation’s job is to combine the different visual content on the screen as quickly as possible. This content is broken down into separate layers (CALayer in iOS) and stored in a tree hierarchy. This tree also forms the basis for UIKit and everything you can see on the screen in an iOS application.

Basically, the content on the screen that the user can see is managed by the CALayer. So how exactly does CALayer manage? In addition, in iOS development process, the most used view control is actually UIView, not CALayer, so what is the relationship between the two?

The CALayer is the foundation of display: it stores bitmaps

At its simplest, the CALayer is the basis of screen display. So how did CALayer pull it off? Let’s explore the source code further. In calayer. h, the CALayer has this property contents:

/** Layer content properties and methods. **/

/* An object providing the contents of the layer, typically a CGImageRef,
 * but may be something else. (For example, NSImage objects are
 * supported on Mac OS X 10.6 and later.) Default value is nil.
 * Animatable. */

@property(nullable, strong) id contents;
Copy the code

An object providing the contents of the layer, typically a CGImageRef.

Contents provides the contents of the layer, which is a pointer type, which in iOS is CGImageRef (or NSImage in OS X). We further found that Apple’s definition of CGImageRef is:

A bitmap image or image mask.

Looking at bitmap, we can now relate to the rendering pipeline we talked about earlier: The contents property in the CALayer actually holds the bitmap rendered by the device rendering pipeline (also known as the backing Store), which will be read from the CALayer and rendered on the screen when the device screen is refreshed.

So, if we set the contents property of the CALayer in our code, like this:

Struct CGImage CGImageRef; // struct CGImage CGImageRef; layer.contents = (__bridge id)image.CGImage;Copy the code

At runtime, the operating system will call the underlying interface to render the image through the CPU+GPU rendering pipeline to get the corresponding bitmap, which will be stored in calayer. contents. When the device screen is refreshed, the bitmap will be read and rendered on the screen.

Because the content to be rendered is statically stored each time, Core Animation triggers a call to the drawRect: method each time it is rendered to perform a new display using the stored bitmap.

The relationship between a CALayer and a UIView

As the most commonly used view control, UIView is also closely related to CALayer, so what is the relationship between the two and what is the difference between them?

Of course, there are a number of explicit differences, such as the ability to respond to click events. But in order to get to the bottom of these questions, we must first understand the roles of both.

UIView – Apple

Views are the fundamental building blocks of your app’s user interface, and the UIView class defines the behaviors that are common to all views. A view object renders content within its bounds rectangle and handles any interactions with that content.

According to Apple’s official documentation, A UIView is the basic structure in an app that defines a number of unified specifications. It’s responsible for rendering the content and handling the interaction events. Specifically, its responsibilities fall into the following three categories

  • Drawing and animation
  • Layout and subview management
  • Event handling: Click Event handling

CALayer – Apple

Layers are often used to provide the backing store for views but can also be used without a view to display content. A layer’s main job is to manage the visual content that you provide…

If the layer object was created by a view, the view typically assigns itself as the layer’s delegate automatically, and you should not change that relationship.

From the official documents of CALayer, we can see that the main responsibility of CALayer is to manage the internal visual content, which is also consistent with what we have mentioned above. When you create a UIView, the UIView will automatically create a CALayer, provide itself with a place to store bitmaps (the “backing Store”), and set itself as the CALayer’s delegate for the backing view.

From here we can roughly summarize the following two core relationships:

  1. A CALayer is a property of a UIView that renders and animates, providing visual content.
  2. UIView provides the encapsulation of some of the functions of the CALayer, and is also responsible for handling interactive events.

With these two key underlying relationships in mind, it’s easy to explain the obvious similarities and differences that often appear in interview answers. A few examples:

  • Same hierarchy: We are familiar with the hierarchy of UIViews. Since each UIView has a CALayer responsible for drawing the page, the CALayer also has a hierarchy.

  • Partial effects: UIView encapsulates only some of the CALayer’s functions, while other effects such as rounded corners, shadows, and borders need to be set by calling the Layer properties.

  • Whether to respond to click events: The CALayer is not responsible for click events, so it does not respond to click events, whereas the UIView does.

  • Different inheritance relationships: CALayer inherits from NSObject, UIView inherits from UIResponder because it is responsible for interactive events.

The last question, of course, is why do you want to separate the CALayer and use UIView to manage it? Why not just use one unified object to handle everything?

The main reason for this design is to separate responsibilities, split functions, and facilitate code reuse. The Core Animation framework is responsible for rendering visual content, which can be rendered using Core Animation on both iOS and OS X. At the same time, the two systems can further encapsulate unified controls according to different interaction rules. For example, iOS has UIKit and UIView, while OS X has AppKit and NSView.

4. Core Animation renders the entire content

Core Animation Pipeline

Now that we know the basics of Core Animation and CALayer, let’s take a look at Core Animation’s rendering pipeline.

The entire assembly line consists of the following steps:

Handle Events: This process processes click Events first, which may require changing the layout and interface hierarchy of the page.

** At this point, the APP will use the CPU to handle the pre-calculation of the display content, such as layout calculation, image decoding, etc., which will be explained in detail in the following section. After that, the calculated layers are packaged and sent to the Render Server.

**Decode: ** After the packed layer is transferred to the Render Server, it is first decoded. Note that you need to wait for the next RunLoop to execute the next Draw Calls after decoding.

**Draw Calls: ** After decoding, Core Animation will call the methods of the underlying rendering framework (such as OpenGL or Metal) to Draw, and then call the GPU.

**Render: ** This stage is mainly rendered by the GPU.

**Display: **Display phase, which needs to wait for the next RunLoop to trigger the Display.

What happens to the Commit Transaction

The Handle Events and Commit Transaction phases that affect development are where developers are most exposed. Handle Events is to Handle touch Events, while the part of Commit Transaction mainly carries out four specific operations: Layout, Display, Prepare, Commit and so on.

Layout: Builds the view

This stage mainly deals with the construction and layout of the view, the specific steps include:

  1. Call overloadedlayoutSubviewsmethods
  2. Create view and passaddSubviewMethod to add a subview
  3. Calculate the view Layout, which is all the Layout constraints

Since this phase takes place in the CPU, usually due to CPU or IO limitations, we should try to minimize this part of the time by making it as efficient and lightweight as possible, such as reducing unnecessary view creation, simplifying layout calculations, reducing view hierarchy, etc.

Display: Draws a view

In this stage, I mainly give the primitives data to Core Graphics to draw the view. Note that the primitives data are not displayed, but the primitives data are obtained.

  1. According to the results of the previous phase of the Layout to create the meta information.
  2. If I rewritedrawRect:Method, then overloaded calls will be calleddrawRect:Methods,drawRect:Method to manually draw bitmap data, so as to customize the drawing of the view.

Note Normally, only primitives information is displayed during the Display phase, while bitmaps are drawn on the GPU based on primitives information. However, if you override the drawRect: method, the method will call the Core Graphics draw method directly to obtain the bitmap data, and the system will request an extra chunk of memory to store the drawn bitmap.

Because the drawRect: method was overwritten, the drawing process was moved from the GPU to the CPU, which resulted in some loss of efficiency. At the same time, this process uses additional CPU and memory, so it needs to be drawn efficiently, otherwise it can cause CPU congestion or memory explosion.

Prepare: Additional work for Core Animation

This step is mainly: image decoding and conversion

Commit: Package and send

This step is to package the layers and send them to the Render Server.

Note that the commit operation is performed recursively depending on the layer tree, so if the layer tree is too complex, the commit can be expensive. This is why we want to reduce the view hierarchy and thus reduce the complexity of the layer tree.

Rendering Pass: Specific operations of Render Server

Render Server is usually OpenGL or Metal. Taking OpenGL as an example, the above figure mainly shows the operations performed in the GPU, including:

  1. Description The GPU received the Command Buffer containing primitives information
  2. Tiler starts by processing the vertices through the Vertex Shader and updating the meta information
  3. Tiling: Tiling generates the geometry of the tile bucket. This step converts the primitive information into pixels and then writes the result to the Parameter Buffer
  4. When Tiler has updated all the primitives, or the Parameter Buffer is full, the next step begins
  5. Renderer: Processes the pixel information to get a bitmap, which is then stored in the Render Buffer
  6. The Render Buffer stores the rendered bitmap for later Display operations

Using Instrument’s OpenGL ES, you can monitor the process. The OpenGL ES Tiler Utilization and OpenGL ES Renderer Utilization can monitor the performance of the Tiler and renderer, respectively

5. Off-screen Rendering

Off-screen rendering is one of the most frequently asked interview questions, so let’s talk about off-screen rendering from start to finish.

Details of off-screen rendering process

From the above, to simplify, the usual rendering process looks like this:

Through the cooperation between THE CPU and GPU, the App continuously renders the content into the Framebuffer, and the display screen continuously obtains the content from the Framebuffer and displays the real-time content.

The off-screen rendering process looks like this:

Instead of the GPU directly putting the rendered content into the Framebuffer, you need to create an extra Offscreen Buffer and put the pre-rendered content into it. Wait until the appropriate time to further overlay and render the contents of the Offscreen Buffer, and then switch the results to the Framebuffer.

Efficiency of off-screen rendering

From the above process, the App needs to render some content in advance and save it to the Offscreen Buffer, and the content of the Offscreen Buffer and Framebuffer needs to be changed when necessary. So it takes longer to process (in fact both of these steps are very expensive to switch buffers).

And since the Offscreen Buffer itself requires extra space, a large number of off-screen renderings may strain memory too much. At the same time, the total size of the Offscreen Buffer is limited and cannot exceed 2.5 times the total screen pixels.

It can be seen that the overhead of off-screen rendering is very large. Once there is too much content to render off-screen, it is easy to cause the problem of frame dropping. So for the most part, we should try to avoid off-screen rendering.

Why use off-screen rendering

So why use off-screen rendering? This is mainly because of two reasons:

  1. Some special effects require an extra Offscreen Buffer to hold the render’s intermediate state, so off-screen rendering has to be used.
  2. For the purpose of efficiency, the content can be rendered in advance and saved in the Offscreen Buffer to achieve the purpose of reuse.

For the first case, where you have to use off-screen rendering, it’s usually automatically triggered by the system, such as shadows, rounded corners, etc.

One of the most common situations is when a mask is used.

As shown in the figure, since the final content is overlaid by two layers of render results, extra memory space must be used to save the intermediate render results, so the system will trigger off-screen rendering by default.

Or here’s an example of a UIBlurEffectView that iOS 8 started offering:

The whole fuzzy process is divided into several steps: Pass 1 first renders the content to be blurred; Pass 2 zooms the content; Pass 3 and 4 respectively blur the content of the previous step in horizontal and vertical directions; in the last step, the results of the blur are superimposed and synthesized to achieve the complete fuzzy effect.

In the second case, the use of off-screen rendering for reuse and efficiency is generally an active action, achieved through the CALayer’s shouldRasterize rasterize operation.

ShouldRasterize rasterizer

When the value of this property is YES, the layer is rendered as a bitmap in its local coordinate space and then composited to the destination with any other content.

After rasterization is enabled, off-screen rendering will be triggered, and the Render Server will force to save the result bitmap of CALayer’s rendering bitmap, so that it can be reused directly in the next rendering to improve efficiency.

The saved Bitmap contains layer subLayer, rounded corners, shadows, group opacity, etc. Therefore, if the layer structure contains the above elements, the structure is complex and needs to be reused repeatedly, you can consider turning rasterization on.

Rounded corners, shadows, group transparency, etc., are automatically triggered by the system for off-screen rendering, so turning on rasterization can save second and subsequent rendering time. The multi-layer subLayer does not automatically trigger off-screen rendering, so it takes more time for the first off-screen rendering, but saves the cost of repeated rendering later.

However, there are a few things to keep in mind when using rasterization:

  1. If the layer cannot be reused, there is no need to turn rasterization on
  2. If the layer is not static and needs to be changed frequently, such as while it is in animation, then turning on off-screen rendering can be counterproductive
  3. There is a time limit for off-screen rendering of cached content. If cached content is not used within 100ms, it will be discarded and cannot be reused
  4. There is limited cache space for off-screen rendering, and any rendering larger than 2.5 times the screen pixel size will be invalid and cannot be reused

Off-screen rendering of rounded corners

In general, setting the rounded corners of layer automatically triggers off-screen rendering. But when does setting rounded corners trigger off-screen rendering?

As shown in the figure above, layer is made up of three layers. When we set the rounded corners, we usually start with this line of code:

view.layer.cornerRadius = 2
Copy the code

According to the cornerradius-apple description, only the backgroundColor and border corners are set by default, not the content corners. Unless you also set layer.masksToBounds to true (corresponding to the UIView’s clipsToBounds property) :

Setting the radius to a value greater than 0.0 causes the layer to begin drawing rounded corners on its background. By default, the corner radius does not apply to the image in the layer’s contents property; it applies only to the background color and border of the layer. However, setting the masksToBounds property to true causes the content to be clipped to the rounded corners.

With a cornerRadius setting and no masksToBounds set, the off-screen render will not be triggered because there is no need for overlay cropping. When the clipping property is set, masksToBounds will crop the layer and all the contents of the subLayer, so you have to trigger the off-screen render.

view.layer.masksToBounds = true// The reason for triggering off-screen renderingCopy the code

So Texture also suggests not triggering off-screen renderings when rounded corners are not necessary:

The specific logic of off-screen rendering

MasksToBounds will crop everything on the layer and induce an off-screen render. So let’s go into the details of this process.

The layer overlay follows roughly the “painter’s algorithm,” in which the layers are drawn, first drawing the distant scene and then covering the distant part with the closer scene.

In ordinary layer drawing, the upper sublayer overwrites the lower sublayer. After the lower sublayer is drawn, it can be abandoned to save space and improve efficiency. After all sublayer are drawn in turn, the whole drawing process is completed and the subsequent rendering can be carried out. Suppose we need to draw a three-layer sublayer without clipping and rounded corners. The drawing process would look like this:

When we set the cornerRadius and masksToBounds for rounded corners + clipping, the masksToBounds clipping property is applied to all sublayer as described earlier. This means that all sublayer must be filleted and clipped again. This means that all sublayer must be stored in the Offscreen buffer for the next round of filleted and clipped, instead of being discarded immediately after being drawn. This induces off-screen rendering, and the process is as follows:

In fact, it’s not just rounded corner + clipping. If you set opacity + group transparency (layer.allowsGroupopacity +layer.opacity), the shadow attribute (shadowOffset, etc.) will have similar effects, because group opacity and shadow are similar to clipping. Will apply to the layer and all of its sublayer, which will inevitably cause off-screen rendering.

Avoid off-screen rendering with rounded corners

Besides minimizing the use of rounded clipping, is there any other way to avoid off-screen rendering caused by rounded corners + cropped?

As we mentioned earlier, the essence of off-screen rendering caused by rounded corners is clipping overlay, causing masksToBounds to double process the layer and all sublayer. By avoiding masksToBounds and instead preprocessing all sublayer, we can just do the “artist algorithm” and draw with a single overlay.

Then there are several possible implementation methods:

  1. Use the image with rounded corners directly, or replace the background color with a solid color background with rounded corners to avoid using rounded corners. However, this method depends on the situation and is not universal.
  2. Mask: Add a mask with the same color as the background. Mask is placed on the top layer, covering the four corners to create the rounded shape. However, this method is difficult to solve the background color of the picture or gradient.
  3. [UIBezierPath] Draw a closed rectangle with rounded corners using bezier curves, set only the inside to be visible in the context, render the layer without rounded corners as an image and add it to the Bezier rectangle. This method is more efficient, but once the layout of the layer changes, the Bezier curve needs to be manually redrawn, so you need to manually monitor and redraw the frame, color, etc.
  4. 【CoreGraphics】 to rewritedrawRect:, and use CoreGraphics related methods to manually draw when the rounded corners need to be applied. However, CoreGraphics efficiency is also very limited, if the need for multiple calls will also be efficient problems.

Summary of causes for triggering off-screen rendering

To summarize, the following situations trigger off-screen rendering:

  1. Layer with mask (layer.mask)
  2. The layer to be clipped (layer.masksToBounds / view.clipsToBounds)
  3. Layer with group opacity set to YES and opacity not to 1 (layer.allowsGroupOpacity/layer.opacity)
  4. Added the drop shadow layer (layer.shadow*)
  5. The rasterized layer (layer.shouldRasterize)
  6. Draw the text layer (UILabel.CATextLayer.Core TextEtc.)

However, it is important to note that overriding the drawRect: method does not trigger off-screen rendering. As mentioned earlier, overwriting drawRect: it moves rendering from the GPU to the CPU and requires extra memory space. According to the Apple engineers, this is not the same as a standard off-screen render, and when you start debugging Color offscreen Yellow in Instrument, it will not be rendered off-screen.

6. Quiz yourself

Generally speaking, do some questions to deepen understanding and consolidation, so here is a simple extract from the article, hoping to help you:

  1. What are the CPU and GPU designed for?
  2. CPU or GPU which Cache/ALU/Control unit ratio is higher?
  3. What is the general flow of computer image rendering pipeline?
  4. What does the Framebuffer do?
  5. B: How do you cause Screen Tearing?
  6. How to solve the problem of screen tearing?
  7. How does frame drop happen?
  8. What are the responsibilities of CoreAnimation?
  9. What is the relationship between UIView and CALayer? What’s the difference?
  10. Why do you have a UIView and a CALayer, and can you synthesize one?
  11. What tasks will the CPU be responsible for in the render pipeline?
  12. Why is off-screen rendering efficient?
  13. When should I use off-screen rendering?
  14. ShouldRasterize what is rasterize?
  15. What are some common situations that trigger off-screen rendering?
  16. Does the cornerRadius setting trigger off-screen rendering?
  17. What are the solutions for off-screen rendering triggered by rounded corners?
  18. Does overwriting the drawRect method trigger off-screen rendering?

References:

  • Inside look at modern web browser – Google
  • 1.2 Differences between CPU and GPU design – Magnum Programm Life
  • CUDA Programming (3): GPU architecture understand! – SeanDepp
  • Graphics pipeline – Wiki
  • GPU Rendering Pipeline – GPU Rendering Pipeline introduction – Pioneer dog article – Zhihu
  • Computer Things (8) – Graphic image rendering principles – Chu Quan’s world
  • Tips for keeping the iOS interface flowing – ibireme
  • Framebuffer – Wiki
  • Frame Rate (iOS and tvOS) – Apple
  • Understand Vsync-Chen Meng
  • Getting Pixels onto the Screen – objc.io
  • In-depth understanding of iOS Rendering process-lision
  • Core Animation Programming Guide – Apple
  • IOS Core Animation: Advanced Techniques
  • Advanced Graphics and Animations for iOS Apps – Joakim
  • Principles of iOS graphics rendering – Chuquan
  • Texture – Corner Rounding
  • Mastering Offscreen Render – seedante
  • An in-depth look at iOS off-screen rendering
  • Offscreen rendering / Rendering on the CPU – Stack Overflow