Paint on screen with Flutter rendering

Portal:

Initiation of Flutter rendering and construction of three trees
Draw startup and Layout of Flutter rendering
Paint on screen with Flutter rendering

The last article covered the calculation of the start and layout of the rendering, and here we continue with the CompositingBits, flushPaint, and compositeFrame processes.

Draw the principle

Before we start exploring the drawing process, let’s take a look at how to render a graph without using the Widgets of the Flutter Framework

import 'dart:ui';
import 'package:flutter/material.dart';
void main() {
  // runApp(MyApp());
  // Create the first square
  // Create an artboard with PictureRecorder
  PictureRecorder recorder = PictureRecorder();
  Canvas canvas = Canvas(recorder);

  / / drawing canvas
  // draw from 100,100 coordinates
  Offset offset = Offset(300.300);
  // The draw area is a 100x100 area
  Size size = Size(300.300); canvas.drawRect(offset & size, Paint().. color = Colors.red);EndRecording ends the node drawing and returns a Picture
  Picture picture = recorder.endRecording();

  // Create a second circle
  PictureRecorder recorder1 = PictureRecorder();
  Canvas canvas1 = Canvas(recorder1);
  Offset offset1 = Offset(0.0);
  Size size1 = Size(300.300); canvas1.drawOval(offset1 & size1, Paint().. color = Colors.blue); Picture picture1 = recorder1.endRecording();// Initialize a SceneBuilder
  SceneBuilder sceneBuilder = SceneBuilder();
  // Add the Picture generated by the appeal canvas to engine via the method on SceneBuilder
  sceneBuilder.pushOffset(0.0);
  sceneBuilder.addPicture(new Offset(0.0), picture);
  sceneBuilder.addPicture(new Offset(600.800), picture1);
  sceneBuilder.pop();
  // Create sceneBuilder.build
  Scene scene = sceneBuilder.build();

  window.onDrawFrame = () {
    Window. render, which can only be called from onDrawFrame or onBeginFrame
    window.render(scene);
    scene.dispose();
  };
  // A VSync signal is fired and the onDrawFrame callback is fired on the next frame
  window.scheduleFrame();
}
Copy the code

In this example, I created two shapes, a red square and a blue circle. Let’s look at the effect first.

In The Flutter, our Canvas object is required to pass through the PictureRecorder. After calling a series of Canvas operations (for those unfamiliar with Canvas operations, see my previous post Learning basics with Flutter Canvas), We need to call the Recorder.endrecording () to end the Canvas operation and return a Picture object. The Picture object is actually a layer. Then we use the SceneBuilder to add the Picture object. A Scene is the class that controls the drawing of the entire screen. We pass it to window.render() for rasterization into the upper screen. Note that PictureRecorder and SceneBuilder are disposable and cannot be used after calling the Recorder.endrecording and scene.Dispose ().

After reading the above process, we have an idea that the Paint and Composited process of The Flutter are all based on the above process.

compositingBits

Once the layout is completed, the compositingBits process is performed

The compositingBits function is to update the _needsCompositing flag in the dirty composition list. _needsCompositing is used during the paint process to determine whether to use a new layer for rendering, such as cropping. In fact, it itself is not in the common drawing process.

[-> packages/flutter/lib/src/rendering/object.dart:PipelineOwner]

void flushCompositingBits() {
  / / _nodesNeedingCompositingBitsUpdate is a stay compositingBits list again
  _nodesNeedingCompositingBitsUpdate.sort((RenderObject a, RenderObject b) => a.depth - b.depth);
  for (final RenderObject node in _nodesNeedingCompositingBitsUpdate) {
    // Determine if the node needs to be updated
    if (node._needsCompositingBitsUpdate && node.owner == this)
      node._updateCompositingBits();
  }
  // Clear the list
  _nodesNeedingCompositingBitsUpdate.clear();
}
Copy the code

_nodesNeedingCompositingBitsUpdate is a stay to compositingBits list, with a similar layout process, it is through the markNeedsCompositingBitsUpdate to the current node in the list.

[-> packages/flutter/lib/src/rendering/object.dart:RenderObject]

void _updateCompositingBits() {
  // Return directly if the node does not need to be updated
  if(! _needsCompositingBitsUpdate)return;
  final bool oldNeedsCompositing = _needsCompositing;
  _needsCompositing = false;
  // Access the child node
  visitChildren((RenderObject child) {
    child._updateCompositingBits();
    // Set the current '_needsCompositing' to true if the child node also needs compositing
    if (child.needsCompositing)
      _needsCompositing = true;
  });
  // Set the current '_needsCompositing' to true if the current node's isRepaintBoundary or alwaysNeedsCompositing is true
  if (isRepaintBoundary || alwaysNeedsCompositing)
    _needsCompositing = true;
  // Here is an optimization, Set markNeedsPaint if oldNeedsCompositing is not equal to _needsCompositing indicating that the current or descendants of isRepaintBoundary or alwaysNeedsCompositing has been updated
  if(oldNeedsCompositing ! = _needsCompositing) markNeedsPaint(); _needsCompositingBitsUpdate =false;
}
Copy the code

CompositingBits set the _needsCompositing value for the current node. If isRepaintBoundary or alwaysNeedsCompositing is true, If either isRepaintBoundary or alwaysNeedsCompositing is true, then _needsCompositing will also be true. In the process, it updates any rendered objects that have dirty composition bits.

Paint

Let’s take a look at one of the most important processes in rendering — the Paint process.

RepaintBoundary

Before we started the Paint process, we decided on a concept called RepaintBoundary. We knew that modern UI systems would layer the interface so that layers could be reused to reduce the amount of drawing and improve the drawing performance. In Flutter we used RenderObject’s isRepaintBoundary to take care of the layer control.

We want to override isRepaintBoundary=>true in the subclass so that the parent node can rerender without rerendering itself.

Currently in version 2.2.2, All take isRepaintBoundary attributes of the Widget has a TextField, CupertinoTextSelectionToolbar, RenderEditable, SingleChildScrollView, Flow, Android View, UiKitView, PlatformViewSurface, Texture, RenderView, RepaintBoundary, where the RepaintBoundary is open for developers to use.

flushPaint

The Paint procedure builds a layer tree by calling a series of Canvasapis, which starts with a call to flushPaint.

[-> packages/flutter/lib/src/rendering/object.dart:PipelineOwner]

void flushPaint() {
  / /...
  try {
    // The list to render
    final List<RenderObject> dirtyNodes = _nodesNeedingPaint;
    _nodesNeedingPaint = <RenderObject>[];
    for (final RenderObject node indirtyNodes.. sort((RenderObject a, RenderObject b) => b.depth - a.depth)) {assert(node._layer ! =null);
      if (node._needsPaint && node.owner == this) {
        // If the node has already been attached, the current situation is updated, so update the current node and its children directly
        if(node._layer! .attached) {/ / if the layer has been generated, call PaintingContext. RepaintCompositedChild
          PaintingContext.repaintCompositedChild(node);
        } else {
          // Otherwise, call the _skippedPaintingOnLayer method on RenderObject, recursively setting _needsPaint to true to ensure that none of the nodes that should be updated will benode._skippedPaintingOnLayer(); }}}}finally {
    // ...}}Copy the code

Like layout and compositingBits, the Paint procedure maintains a list called _nodesNeedingPaint to update only the nodes that currently need to be updated. The _nodesNeedingPaint is added via the markNeedsPaint method.

markNeedsPaint

[-> packages/flutter/lib/src/rendering/object.dart:RenderObject]

void markNeedsPaint() {
  if (_needsPaint)
    return;
  _needsPaint = true;
  // If the current node isRepaintBoundary is true and owner! =null to add the current node to the _nodesNeedingPaint queue
  if (isRepaintBoundary) {
    if(owner ! =null) { owner! ._nodesNeedingPaint.add(this);
      // Call window.scheduleFrame() to notify that the next frame needs to be received
      owner!.requestVisualUpdate();
    }
  }
  // Mark the parent if it is a RenderObject
  else if (parent is RenderObject) {
    final RenderObject parent = this.parent! as RenderObject;
    parent.markNeedsPaint();
  } else {
    // The current node is the root node
    if(owner ! =null)
      owner!.requestVisualUpdate();
  }
}
Copy the code

Depending on whether isRepaintBoundary is true, add the current node to the needingPaint list if it is currently RepaintBoundary, otherwise recursively call the parent’s markNeedsPaint. Its essence is to determine the area to be drawn according to isRepaintBoundary.

_skippedPaintingOnLayer

[-> packages/flutter/lib/src/rendering/object.dart:RenderObject]

void _skippedPaintingOnLayer() {
  AbstractNode? node = parent;
  while (node is RenderObject) {
    if (node.isRepaintBoundary) {
      if (node._layer == null)
        break; // If the subtree has never been drawn, stop recursion.
      if(node._layer! .attached)break; // Stop recursion if the layer has been inserted into the layer tree
      node._needsPaint = true; } node = node.parent; }}Copy the code

The _skippedPaintingOnLayer handles the fact that the current node and its parent have been detach(removed from the layer tree) at some point in time, and re-labels all removed nodes as _needsPaint.

repaintCompositedChild

[-> packages/flutter/lib/src/rendering/object.dart:PaintingContext]

static void repaintCompositedChild(RenderObject child, { bool debugAlsoPaintedParent = false{})assert(child._needsPaint);
  _repaintCompositedChild(
    child,
    debugAlsoPaintedParent: debugAlsoPaintedParent,
  );
}
static void _repaintCompositedChild(
  RenderObject child, {
  bool debugAlsoPaintedParent = false,
  PaintingContext? childContext,
}) {
  / /...
  // Get the layer of the current node
  OffsetLayer? childLayer = child._layer asOffsetLayer? ;// If the layer is empty, initialize the layer
  if (childLayer == null) {
    child._layer = childLayer = OffsetLayer();
  } else {
    // Otherwise remove the current layer from the layer tree
    childLayer.removeAllChildren();
  }
  // Initialize the current node's childContextchildContext ?? = PaintingContext(child._layer! , child.paintBounds);// Start drawing the current node
  child._paintWithContext(childContext, Offset.zero);
  // Stop drawing
  childContext.stopRecordingIfNeeded();
}
Copy the code

The _repaintCompositedChild basically reinitializes the layer of the current node and then calls _paintWithContext to draw.

_paintWithContext

[-> packages/flutter/lib/src/rendering/object.dart:RenderObject]

void _paintWithContext(PaintingContext context, Offset offset) {
  if (_needsLayout)
    return;
  _needsPaint = false;
  try {
    // Call the current node's paint method
    paint(context, offset);
  } catch (e, stack) {
  }
}
Copy the code

The _paintWithContext does some pre-drawing checks during the development phase, and determines whether or not it needs to be drawn based on the current _needsPaint, and finally calls the paint method to draw.

paint

Paint is the method that starts each node, and the subclass is required to draw the target itself by passing in a PaintingContext and Offset

PaintingContext:PictureRecorderandCanvasDrawing class, and encapsulates some commonly used drawing methods, is drawing the core class
Offset: draw region. The region to be drawn by the current node is calculated by the parent node and drawn in this region during drawing

RenderObject defines only one empty paint method, which requires subclasses to implement, such as the ColoredBox component’s _RenderColoredBox paint implementation:

[-> packages/flutter/lib/src/widgets/basic.dart:_RenderColoredBox]

void paint(PaintingContext context, Offset offset) {
    if(size > Size.zero) { context.canvas.drawRect(offset & size, Paint().. color = color); }if(child ! =null) {
      context.paintChild(child!, offset);
    }
  }
Copy the code

The RenderObject corresponding to ColoredBox is _RenderColoredBox, which is drawn using drawRect and is drawn by Paint().. Color Sets the area color.

The other thing to notice here is that when we call context.canvas, we call the getter method for the current node’s Canvas

Canvas get canvas {
  if (_canvas == null)
    _startRecording();
  return_canvas! ; }void _startRecording() {
  assert(! _isRecording); _currentLayer = PictureLayer(estimatedBounds);When drawing is finished, its endRecording is also called to stop
  _recorder = ui.PictureRecorder();
  // Initialize a canvas with _recorder_canvas = Canvas(_recorder!) ;// Add the initialized PictureLayer to the child node of the current layer_containerLayer.append(_currentLayer!) ; }Copy the code

If the _canvas of the current node is not initialized, then _startRecording is called for a series of initializations. This initializes a PictureLayer, which is a Layer that can draw. Most of the components of our Flutter use it for drawing

compositeFrame

When flushPaint is complete, the Layer tree has been generated. The Layer tree needs to be sent to the GPU for drawing to the screen. FlushPaint after, will immediately call renderView.com positeFrame synthesis of () on the screen

[-> packages/flutter/lib/src/rendering/binding.dart:RenderBinding]

void compositeFrame() {
    / /...
    try {
      // Initialize a SceneBuilder class
      final ui.SceneBuilder builder = ui.SceneBuilder();
      // Pass the SceneBuilder to the layer, which recurses the whole layer tree
      finalui.Scene scene = layer! .buildScene(builder);if (automaticSystemUiAdjustment)
        _updateSystemChrome();
      // Call window.render to draw the screen
      _window.render(scene);
      scene.dispose();
    } finally {
      // ...}}Copy the code

When window.render is called, the drawing will begin. Window. render passes in a Scene that can only be generated with SceneBuilder.

[-> packages/flutter/lib/src/rendering/layer.dart:ContainerLayer]

ui.Scene buildScene(ui.SceneBuilder builder) {
  List<PictureLayer>? temporaryLayers;
  // ..
  updateSubtreeNeedsAddToScene();
  addToScene(builder);
  _needsAddToScene = false;
  final ui.Scene scene = builder.build();
  // ..
  return scene;
}
// Update the _needsAddToScene value of the current node and the child node
void updateSubtreeNeedsAddToScene() {
  super.updateSubtreeNeedsAddToScene();
  Layer? child = firstChild;
  while(child ! =null) { child.updateSubtreeNeedsAddToScene(); _needsAddToScene = _needsAddToScene || child._needsAddToScene; child = child.nextSibling; }}Copy the code

AddToScene is implemented by the Layer subclass itself through the passed SceneBuilder. There are many methods in the SceneBuilder that will pass the Layer into the Flutter engine, such as the commonly used pushOffset

OffsetEngineLayer pushOffset(double dx, double dy, { OffsetEngineLayer oldLayer }) {
    final OffsetEngineLayer layer = OffsetEngineLayer._(_pushOffset(dx, dy));
    return layer;
  }
  EngineLayer _pushOffset(double dx, double dy) native 'SceneBuilder_pushOffset';
Copy the code

The pushOffset can be used to create an offset graph that generates an OffsetEngineLayer from the position information passed in.

Layer

The Layer is a wrapper around some of the SceneBuilder methods in the Flutter Framework. Each Layer corresponds to one or more SceneBuilder methods

The Layer object with the child node does not perform a specific drawing. It calls some Layer objects with the child node to do the drawing. The Layer object with the child node has:

Displacement (OffsetLayer/TransformLayer);
OpacityLayer
Cut class (ClipRectLayer ClipRRectLayer/ClipPathLayer);
Shadow class (PhysicalModelLayer)

The child-free Layer object is the concrete drawing class, but it has no child nodes

The PictureLayer is used to draw, and the components on the Flutter basically use it to draw
TextureLayer is used for external textures, such as video playback
PlatformViewLayer is for iOSPlatformViewUse of embedded textures

Let’s look at the addToScene method of OffsetLayer

void addToScene(ui.SceneBuilder builder, [ Offset layerOffset = Offset.zero ]) {
  engineLayer = builder.pushOffset(
    layerOffset.dx + offset.dx,
    layerOffset.dy + offset.dy,
    oldLayer: _engineLayer as ui.OffsetEngineLayer?,
  );
  addChildrenToScene(builder);
  builder.pop();
}
Copy the code

We pass in the position offset via builder. PushOffset so that the entire drawing of subsequent child nodes will be applied with this offset.

Take a look at the PictureLayer class for drawing

class PictureLayer extends Layer {
  PictureLayer(this.canvasBounds);
  // Attributes used to generate borders during debugging
  final Rect canvasBounds;
  // Save the Picture class for drawing information
  ui.Picture? get picture => _picture;
  ui.Picture? _picture;
  set picture(ui.Picture? picture) {
    markNeedsAddToScene();
    _picture = picture;
  }
  // ...
  @override
  voidaddToScene(ui.SceneBuilder builder, [ Offset layerOffset = Offset.zero ]) { builder.addPicture(layerOffset, picture! , isComplexHint: isComplexHint, willChangeHint: willChangeHint); }/ /...
}
Copy the code

The PictureLayer holds a Picture class that holds the drawing information for an area. It is passed in through the SceneBuilder’s addPicture method when adding to scene.

AddToScene goes from the root node of the Layer, and when we have a Layer object with children, we’re going to call addChildrenToScene and we’re going to call addToScene for all the children, so, Each time the compositeFrame process initializes a SceneBuilder, the SceneBuilder will be used throughout the Layer tree.

Finally, in the compositeFrame method above, generate the Scene object through Builder. Build and send it to the GPU through window.render, so that the whole page is rendered.

Paint on screen with Flutter rendering

Draw the principle

compositingBits

Paint

RepaintBoundary

flushPaint

markNeedsPaint

_skippedPaintingOnLayer

repaintCompositedChild

_paintWithContext

paint

compositeFrame

Layer

Related Posts

The best way to negotiate is to take no objection

React State Hooks Closure trap, which must be learned before using Hooks

Configure routes in Vue scaffolding