From code to image, the drawing principle of Flutter

In the previous article, We explored the three trees created by Flutter during the rendering process: the Widget tree, the Element tree and the RenderObject tree. But there’s more to the rendering process than just these three trees, and there’s a lot more that we haven’t covered, which we’ll continue to discuss in this article.

Ask questions

Let’s start by listing some of the problems we didn’t explain in the previous article and some of the new problems that have arisen since the extension of the previous article:

If the RenderObject is a real RenderObject, why do you need an Element? So why more widgets? Wouldn’t it be nice to create renderings directly from a RenderObject?
The RenderObject tree is the rendering tree of Flutter. How to render the tree after the RenderObject tree is created?
As mentioned in the previous article, only classes that inherit from RenderObjectWidget are classes that need to be rendered. That’s similar to classes like Container that inherit from StatelessWidgets or StatefulWidgets. How are their properties, such as the background color, rendered on the screen?

With these questions in mind, we begin today’s journey of discovery. We’ll start with rendering the RenderObject tree.

The view to render

The frame callback

Before discussing the rendering mechanism, let’s recall our Flutter project. The first line of code usually looks like this:

void main() {
  runApp(MyApp());
}
Copy the code

This line of code was generated from a template, but it was the beginning of our application, and we thought, what does this method actually do?

MyApp() is the root Widget of our custom Widget, so let’s start with this method to see how Flutter is rendered and drawn.

voidrunApp(Widget app) { WidgetsFlutterBinding.ensureInitialized() .. scheduleAttachRootWidget(app) .. scheduleWarmUpFrame(); }Copy the code

First get the singleton of WidgetsFlutterBinding through ensureInitialized(), and then bind the Element to the Widget through the following call chain, which we covered in the previous article but not in this one.

We move on to the scheduleWarmUpFrame() method below.

void scheduleWarmUpFrame() {
    // ...
    
    // We use timers here to ensure that microtasks flush in between.
    Timer.run(() {
        assert(_warmUpFrame);
        handleBeginFrame(null);
    });
    Timer.run(() {
        assert(_warmUpFrame);
        handleDrawFrame();
        // ...
    });

    // ...
}
Copy the code

The handleBeginFrame() and handleDrawFrame() methods are called by the Flutter engine respectively to prepare for and generate a new frame, which are timing differences. Before we do that, let’s take a look at FrameCallback, There are three queues of Framecallbacks in Flutter (specifically in the SchedulerBinding class) — transientCallbacks, persistentCallbacks, and postFrameCallbacks. They are executed at different times. It is in the handleBeginFrame() method that transientCallbacks are executed, PersistentCallbacks and postFrameCallbacks are executed in the handleDrawFrame() method. Their functions are as follows:

transientCallbacks: Used to store temporary callbacks, usually animation callbacks. Can be achieved bySchedulerBinding.instance.scheduleFrameCallbackAdd a callback.
persistentCallbacks: used to store persistent callbacks in which new frames cannot be drawn. Persistent callbacks cannot be removed once registered.SchedulerBinding.instance.addPersitentFrameCallback()This callback handles layout and drawing.
postFrameCallbacks: is called only once at the end of the Frame, and is removed by the system after the callSchedulerBinding.instance.addPostFrameCallback()Register, and be careful not to trigger a new Frame in such a callback, as this can cause a loop to refresh.

Having said that, let’s recall what we would do if we had a need to execute some command (such as a refresh view operation) after the view is rendered. Is often use WidgetsBinding. Instance. AddPostFrameCallback () this method to the equivalent implementation, WidgetsBinding implementation of SchedulerBinding, So this method actually uses frame completion to fulfill the requirements. Since the callback is removed after the first call, we don’t have to worry about it.

draw

Before I became a student who knocked on Flutter, I was a student who knocked on Android. Therefore, when it comes to interface drawing, the first thing that comes to mind is the well-known three processes — measure, layout and draw. Then, how is drawing done in Flutter? When we talk about rendering the Flutter view, what we are really talking about is rendering the RenderObject. When I look at the structure of the RenderObject in Android Studio (the default shortcut is Alt + 7), Seeing that RenderObject also has a Layout method, I locate the PipelineOwner class through the following call chain.

PipelineOwner manages the render channel, provides the interface that drives the draw channel and stores the state that the RenderObject needs to access at all times. To refresh the draw channel, perform the following methods in sequence:

flushLayout()Update any RenderObject that needs to update its layout. At this stage, the dimensions and layout of all RenderObjects have been calculated, but their status may have been marked as “dirty” and need to be redrawn.
flushCompositingBits()Update anyneedsCompositingThe variable is marked true for the RenderObject. At this stage, each RenderObject knows if its children need to be reassembled, and this information is used to choose how to implement visual effects (such as cropping) during the drawing stage. If a RenderObject has composite children, then Layer is used to create clippings that can be applied to the composite children (the clippings will be drawn into the child’s own Layer).
flushPaint()Access any RenderObject that needs to be updated. At this stage, the RenderObject has the opportunity to record the draw commands into the PictureLayer and construct other layers.
flushSemantics()If semantic is enabled, this method will integrate semantics with RenderObject, which is typically used for semantic feedback (such as screen reading for visually impaired people, please do your own research for more information).

FlushLayout () method flushLayout()

/// PipelineOwner
void flushLayout() {
    // ...
    try {
        // TODO(ianh): assert that we're not allowing previously dirty nodes to redirty themselves
        while (_nodesNeedingLayout.isNotEmpty) {
            final List<RenderObject> dirtyNodes = _nodesNeedingLayout;
            _nodesNeedingLayout = <RenderObject>[];
            for (final RenderObject node indirtyNodes.. sort((RenderObject a, RenderObject b) => a.depth - b.depth)) {if (node._needsLayout && node.owner == this) node._layoutWithoutResize(); }}}finally {
        // ...}}Copy the code

In this method, you iterate through the _nodesNeedingLayout list and then call its _layoutWithoutResize() method. Leaving this method aside for a moment, _nodesNeedingLayout queues the RenderObject for relayout by calling RenderObject’s markNeedsLayout() method.

As we saw in the previous article, RenderObject is inserted into the tree by calling the Insert () method, and in insert() the adoptChild() method is executed. Let’s look at the code:

/// RenderObject
@override
void adoptChild(RenderObject child) {
    assert(_debugCanPerformMutations);
    assert(child ! =null);
    setupParentData(child);
    markNeedsLayout();
    markNeedsCompositingBitsUpdate();
    markNeedsSemanticsUpdate();
    super.adoptChild(child);
}
Copy the code

In this method, the markNeedsLayout() method is executed, which marks the need to rearrange the RenderObject as soon as it is inserted into the tree. The markNeedLayout() method is also called at appropriate times in each implementation of the RenderObject (such as changes in size, position, etc.). MarkNeedsCompositingBitsUpdate () and markNeesPaint () is also in the same way. So ** when the “dirty nodes” are marked and the next frame synchronization signal arrives, The drawFrame() method of RendererBinding is called so that flushLayout(), flushCompositingBits(), flushPaint(), and flushSemantics() are executed in turn. The execution of these methods will eventually allow layouts and paint to be implemented for renderObjects. ** This is the main flow of RenderObject’s rendering mechanism. Let’s move on to the implementation details.

layout

In flushLayout(), the dirty nodes are iterated over and their _layoutWithoutResize() method is executed in turn, which then executes the performLayout() method, which is not implemented in the RenderObject class, Implemented by a RenderObject subclass, which basically calls its child’s Layout () method, which passes down the Constraints object parameters, Size the child node (the layout() method calls the performResize() method to set the size). The Layout() method calls the performLayout() method of the same object… RenderView is the root node of the RenderObject tree, and there is no layout() method.

composite

Check whether RenderObject needs to redraw, and update the RenderObject needsCompositing attribute, if the attribute value is marked as true need to be redrawn.

paint

The flushPaint() method, like the flushLayout() method, iterates over the objects that were previously marked dirty by markNeedsPaint(), Pass the node RenderObject in turn into the repaintCompositedChild() method of the PaintingContext as an argument.

/// PipelineOwner
void flushPaint() {
    // ...
    try {
        final List<RenderObject> dirtyNodes = _nodesNeedingPaint;
        _nodesNeedingPaint = <RenderObject>[];
        // Sort the dirty nodes in reverse order (deepest first).
        for (final RenderObject node indirtyNodes.. sort((RenderObject a, RenderObject b) => b.depth - a.depth)) {assert(node._layer ! =null);
            if (node._needsPaint && node.owner == this) {
                if(node._layer! .attached) { PaintingContext.repaintCompositedChild(node); }else{ node._skippedPaintingOnLayer(); }}}assert(_nodesNeedingPaint.isEmpty);
    } finally {
        // ...}}Copy the code

The PaintingContext class holds the Canvas instance, and the repaintCompositedChild() method passes in its own instance as an argument when calling the _paintWithContext() method of the RenderObject class, so that, When the paint() method is then executed, the example can be drawn by a RenderObject subclass in the paint() method implementation. Let’s look at an example of the paint() method implemented in class _RenderRadio:

/// _RenderRadio
@override
void paint(PaintingContext context, Offset offset) {
    final Canvas canvas = context.canvas;

    paintRadialReaction(canvas, offset, size.center(Offset.zero));

    final Offset center = (offset & size).center;
    finalColor radioColor = onChanged ! =null ? activeColor : inactiveColor;

    // Outer circle
    finalPaint paint = Paint() .. color = Color.lerp(inactiveColor, radioColor, position.value) .. style = PaintingStyle.stroke .. strokeWidth =2.0;
    canvas.drawCircle(center, _kOuterRadius, paint);

    // Inner circle
    if (!position.isDismissed) {
        paint.style = PaintingStyle.fill;
        canvas.drawCircle(center, _kInnerRadius * position.value, paint);
    }
}
Copy the code

What? And a fourth tree? !

RepaintBoundary

The markNeedsPaint() method is used to mark the need for repainting, but without looking at its implementation, we can now look at its source code:

/// RenderObject
void markNeedsPaint() {
    assert(owner == null| |! owner! .debugDoingPaint);if (_needsPaint)
        return;
    _needsPaint = true;
    if (isRepaintBoundary) {
        // ...
        if(owner ! =null) { owner! ._nodesNeedingPaint.add(this);
            owner!.requestVisualUpdate();
        }
    } else if (parent is RenderObject) {
        final RenderObject parent = this.parent as RenderObject;
        parent.markNeedsPaint();
        assert(parent == this.parent);
    } else {
        // ...
        // If we're the root of the render tree (probably a RenderView),
        // then we have to paint ourselves, since nobody else can paint
        // us. We don't add ourselves to _nodesNeedingPaint in this
        // case, because the root is always told to paint regardless.
        if(owner ! =null)
            owner!.requestVisualUpdate();
    }
}
Copy the code

We have an isRepaintBoundary() variable here, and the code shows that when this variable is false and the parent is RenderObject, the parent will also be marked as being redrawn, so if we have a large view tree and we want to redraw only a small part of it at a time, So what to do?

The official explanation for this variable is whether the RenderObject needs to be redrawn separately from its parent. If the variable is true, then the markNeedsPaint() method does not perform the mark redraw of the parent RenderObject, so progress can be blocked within a small range during redraw.

How does this variable set its value to true? When using the CustomPaint control, we may have seen in some blogs or articles that it is recommended to wrap a RepaintBoundary control around it to avoid unnecessary repainting. RepaintBoundary inherited from SingleChildRenderObjectWidget class, so it will be realized createRenderObject () method, the following is the implementation:

@override
RenderRepaintBoundary createRenderObject(BuildContext context) => RenderRepaintBoundary();
Copy the code

RenderRepaintBoundary is a subclass of RenderObject and its isRepaintBoundary variable is true, so if we need to redraw the above part of the boundary, we can use this control to help us achieve that.

Layer

If you’ve worked with some game frameworks before, you’re probably familiar with Layer, which is a drawing paper on which all the controls are drawn. In order to have an arrangement of all the controls, they should be drawn on different “drawing paper”, easy to do some unified operations, such as updating all the controls on a “drawing paper” at one time. Yes, this Layer also has a tree, which is called the “fourth tree”.

The Layer can be divided into ContainerLayer and other layers. ContainerLayer has children, and it has the following subclasses:

Displacement (OffsetLayer/TransformLayer);
OpacityLayer
Cut class (ClipRectLayer ClipRRectLayer/ClipPathLayer);
Shadow class (PhysicalModelLayer)

The PaintingContext paintChild() method is called by a RenderObject subclass to draw the child node:

/// PaintingContext
void paintChild(RenderObject child, Offset offset) {
    // ...

    if (child.isRepaintBoundary) {
        stopRecordingIfNeeded();
        _compositeChild(child, offset);
    } else {
        child._paintWithContext(this, offset);
    }

    // ...
}
Copy the code

When the child node’s isRepaintBoundary property is true, the child node’s _paintWithContext() method is no longer executed to draw, Instead, the _compositeChild() method is called to create a new Layer and append it to the original Layer tree.

/// PaintingContext
void _compositeChild(RenderObject child, Offset offset) {
    // ...
    final OffsetLayer childOffsetLayer = child._layer asOffsetLayer; childOffsetLayer.offset = offset; appendLayer(child._layer!) ; }/// PaintingContext
@protected
void appendLayer(Layer layer) {
    assert(! _isRecording); layer.remove(); _containerLayer.append(layer); }/// ContainerLayer
void append(Layer child) {
    // ...
    adoptChild(child);
    child._previousSibling = lastChild;
    if(lastChild ! =null) lastChild! ._nextSibling = child; _lastChild = child; _firstChild ?? = child;assert(child.attached == attached);
}
Copy the code

Does the code in the Container’s append() method look familiar? For, in the previous article introduces MultiChildRenderObjectElement child node insert, its _insertIntoChildList chain () method is similar to the double bind operations. So instead of a Layer tree, it’s a Layer chain.

Once the Layer is chained, if you need to make the most of it, you should find someone who can handle it.

Each RenderObject has a layer object bound to it. In the RendererBinding drawFrame() method, the RenderView object’s compositeFrame() method is executed:

/// RenderView
void compositeFrame() {
    Timeline.startSync('Compositing', arguments: timelineArgumentsIndicatingLandmarkEvent);
    try {
        final ui.SceneBuilder builder = ui.SceneBuilder();
        finalui.Scene scene = layer! .buildScene(builder);if (automaticSystemUiAdjustment)
            _updateSystemChrome();
        _window.render(scene);
        scene.dispose();
        // ...
    } finally{ Timeline.finishSync(); }}Copy the code

A new class Scene is introduced here. It is a native class corresponding to scene.cc in the Flutter engine. In the Flutter framework, Scenes can only be created from the SceneBuilder class.

We know that RenderView is the root node of the RenderObject tree, so the layer in the above method is the first node in the layer chain. With that in mind, let’s go to buildScene() :

/// ContainerLayer
ui.Scene buildScene(ui.SceneBuilder builder) {
    // ...
    updateSubtreeNeedsAddToScene();
    addToScene(builder);
    // Clearing the flag _after_ calling `addToScene`, not _before_. This is
    // because `addToScene` calls children's `addToScene` methods, which may
    // mark this layer as dirty.
    _needsAddToScene = false;
    final ui.Scene scene = builder.build();
    // ...
    return scene;
}

/// ContainerLayer
@override
void addToScene(ui.SceneBuilder builder, [ Offset layerOffset = Offset.zero ]) {
    addChildrenToScene(builder, layerOffset);
}

/// ContainerLayer
void addChildrenToScene(ui.SceneBuilder builder, [ Offset childOffset = Offset.zero ]) {
    Layer? child = firstChild;
    while(child ! =null) {
        if (childOffset == Offset.zero) {
            child._addToSceneWithRetainedRendering(builder);
        } else{ child.addToScene(builder, childOffset); } child = child.nextSibling; }}Copy the code

The above call chains simply iterate through the Layer chain and add Layer to Scene by executing their addToScene() method, declared in the Layer base class and implemented by its subclasses.

After all layers have performed addToScene(), The _windower.render (scene) method in the RenderView’s compositeFrame() method pushes the scene containing all the layers onto the GPU for rendering.

Are all three trees needed

Now that the process of Flutter rendering has been described above, let’s consider another of the three questions raised at the beginning of this article — if the RenderObject tree is the tree responsible for rendering, why does Flutter build two other trees? Wouldn’t it be easier just to maintain the RenderObject? Since Widget trees correspond to Element trees, are they redundant?

For these problems, I also searched some articles on the Internet, but did not find a very detailed explanation of this issue. Presumably, it’s all for performance reasons, because rendering is expensive and every change in global rendering will drag down the system. I’ve thought about this a lot in the days I’ve been writing this article, but I’ve never thought of a scenario where a tree can’t be optimized for performance, because it’s perfectly possible to render locally and only when necessary. When I think of the encapsulation idea often used in Flutter design mentioned in my previous articles, that is, to minimize the exposure of the system to users and only expose the interfaces needed by users, the implementation part is completely completed by the system itself, I begin to wonder whether this is also due to such design considerations. Therefore, as a primary school student who is coding, I have no way to answer this question for the time being, and I am still wondering. If I have a chance to learn the secret of it later, I will make modifications and supplements. Please excuse me for my lack of capacity.

The last

There are many issues that we have not covered, such as BuildContext, some states and life cycles of Element and State, etc. First of all, these are relatively simple, I believe they will not be a problem with your wisdom. The second reason is that I don’t think they are as important as the process of rendering and drawing. It took a lot of time to complete these two articles, but I am very happy to finish them. Please advise me if there is any mistake.

Today (2020/12/28) is the third day after Christmas, the first day after postgraduate exams, and four days before New Year’s Day. First of all, I wish you a happy “double Dan”, and then I wish you a successful battle!

I’m a coding pupil, see you next!

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

From code to image, the drawing principle of Flutter

Ask questions

The view to render

The frame callback

draw

layout

composite

paint

What? And a fourth tree? !

RepaintBoundary

Layer

Are all three trees needed

Other problems

The last

From code to image, the drawing principle of Flutter

Ask questions

The view to render

The frame callback

draw

layout

composite

paint

What? And a fourth tree? !

RepaintBoundary

Layer

Are all three trees needed

Other problems

The last

Related Posts

The Appium automation framework has been upgraded to the latest version 2.0 with more capabilities

Rounding out the React Native

Android Studio Hacks/Shortcuts collection