Overview of Flutter UI rendering principles

The three trees

What are three trees? Widgets are the core of Flutter. Everything is a Widget, but there are two other elements that work together: Element and RenderObject. Because of their tree-like structure, they are often referred to as three trees.

Widget

One of the most widely used widgets in the development of Flutter applications is the Widget, the basic unit that “describes” the Flutter UI. Flutter uses the same concept (a Widget) in the Widgets layer to represent on-screen drawing, layout (location and size), user interaction, state management, themes, animations, and navigation.

Google designs widgets with some distinctive features:

• Declarative UI — Compared with imperative UI in traditional Native development, declarative UI has many advantages, such as significantly improved development efficiency, significantly enhanced UI maintainability, etc.
• Immutability — All widgets ina Flutter are immutable, that is, their internal members are final, and the parts of a Flutter need to be Stateful widget-state.
• Composition over Inheritance – Widget design follows the good design philosophy of composition over Inheritance, and even small features can be abstracted to achieve complex features through composition.

Widgets can be divided into three functional categories: “Component Widget”, “Proxy Widget”, and “Renderer Widget”, but only the Renderer Widget will be converted into a Render Object for final display on the UI.

You can see it at the beginning of the Widget comment:

/// Describes the configuration for an [Element].
///
/// Widgets are the central class hierarchy in the Flutter framework. A widget
/// is an immutable description of part of a user interface. Widgets can be
/// inflated into elements, which manage the underlying render tree.
///
Copy the code

This comment illustrates the nature of the Widget: the Widget used to configure Element is essentially configuration information for the UI (with some business logic attached).

Element

We know that widgets are essentially UI configuration data (static, immutable), and elements are “instances” generated by widgets, much like JSON and Object. The same configuration (Widget) can generate multiple instances (elements), which may be placed in different locations on the tree.

Similarly, the first sentence in the Element comment reads:

/// An instantiation of a [Widget] at a particular location in the tree. /// /// Widgets describe how to configure a subtree but the same widget can be used /// to configure multiple subtrees simultaneously because widgets are immutable.  /// An [Element] represents the use of a widget to configure a specific location /// in the tree. Over time, the widget associated with a given element can /// change, for example, if the parent widget rebuilds and creates a new widget /// for this location. ///Copy the code

There is actually a real Tree “Element Tree” between the elements. Element has two main responsibilities:

• Maintain the Element Tree according to changes in the UI (Widget Tree), including node insertion, update, deletion, move, etc.
• Coordinator between Widgets and RenderObjects. (It holds internal references to widgets and RenderObjects)

“Component Element” — Composite Element. The corresponding elements of “Component Widget” and “Proxy Widget” fall into this category. The feature is that the Widget corresponding to the child node needs to be created through the build method. Also, each Element of this type has a single child; Renderer Element — Renderer Element, corresponding to Renderer Widget, has a different number of child nodes for each of its subtypes.

RenderObject

There’s a paragraph in the official document

//In Android, the View is the foundation of everything that shows up on the screen. Buttons, toolbars, and
//inputs, everything is a View. In Flutter, the rough equivalent to a View is a Widget. Widgets don’t map 
//exactly to Android views, but while you’re getting acquainted with how Flutter works you can think of them 
//as “the way you declare and construct UI”.
Copy the code

This means that widgets can be loosely thought of as Android views because they both describe the UI, but they are not exactly equal.

In fact, the RenderObject is more like the View in Android, and the RenderObject is the object that actually participates in the drawing. It has a Layout Paint Composite rendering process similar to View, including many methods.

Flutter	RenderObject	Android View
draw	paint()	draw()/onDraw()
layout	performLayout()/layout()	measure()/onMeasure(), layout()/onLayout()
Layout constraints	Constraints	MeasureSpec
Layout Protocol 1	The Constraints parameter to performLayout() indicates the layout Constraints of the parent node on its children	The two parameters of measure() represent the layout limit of the parent node to the child node
Layout Protocol 2	PerformLayout () should call layout() of each child node	OnLayout () should call layout() of each child node
Layout parameters	parentData	mLayoutParams
Request the layout	markNeedsLayout()	requestLayout()
Request drawing	markNeedsPaint()	invalidate()
Add a child	adoptChild()	addView()
Remove the child	dropChild()	removeView()
Associate with window/tree	attach()	onAttachedToWindow()
Disassociate from window/tree	detach()	onDetachedFromWindow()
Access to the parent	parent	getParent()
Touch events	hitTest()	onTouch()
User input event	handleEvent()	onKey()
rotation	rotate()	onConfigurationChanged()

RenderObject is, of course, an abstract class that describes the basic protocol for rendering; some concrete implementations, such as coordinates, are not defined. The implementation of Flutter is the RenderBox, on which most widgets are implemented.

Rendering process

Build Widgets

Take a look at the following code snippet, which represents a simple widget structure:

Container(
  color: Colors.blue,
  child: Row(
    children: [
      Image.network('https://www.example.com/1.png'),
      Text('A'),
    ],
  ),
);
Copy the code

When Flutter needs to draw this code snippet, the framework calls the Build () method and returns a widget subtree that draws the UI based on the current application state. During this process, the build() method may introduce new widgets based on state if necessary. In the example above, the color and child of the Container are typical examples. We can look at the Container source code, and you can see that when the color property is not empty, ColoredBox is added for color layout.

if (color ! = null) current = ColoredBox(color: color! , child: current);Copy the code

Correspondingly, RawImage and RichText are also introduced in the build process for Image and Text. As a result, the resulting widget structure is deeper than the code representation, as shown in Figure 2 in this scenario:

This is why when you use Dart DevTools’s Flutter Inspector to debug the widget tree structure, you will find that the actual structure is deeper than the structure in your original code.

From the Widget to the Element

During the build phase, a Flutter transforms the widgets described in the code into a tree of corresponding Elements, one for each Widget. Each Element represents an instance of a widget at a specific location in the tree hierarchy. There are two basic types of elements:

• ComponentElement, host to other elements.

• RenderObjectElement, an Element that participates in the layout or drawing phase.

RenderObjectElement is the bridge between the underlying RenderObject and the corresponding widget.

Any widget can be referenced to Element through its BuildContext, which is the context for the widget’s location in the tree. Similar to the context in the theme.of (context) method call, it is passed as an argument to the build() method.

Because widgets and the relationships between the nodes above and below them are immutable, any action done to the widget tree (such as replacing Text(‘A’) with Text(‘B’)) returns A new collection of Widget objects. But that doesn’t mean the underlying presentation has to be rebuilt. The Element tree is persistent between frames and therefore plays a crucial role in performance. Flutter relies on this advantage to implement a mechanism that allows the underlying representation to be cached, just as the widget tree is discarded altogether. A Flutter can rebuild parts of the Element tree that need to be reconfigured based on the widgets that have changed.

Layout and Rendering

Few applications draw only a single widget. Therefore, efficiently arranging the structure of widgets and determining the size and location of each Element before rendering is complete is one of the key points of any UI framework.

In the render tree, the base class for each node is RenderObject, which defines an abstract model for layout and rendering. This is all too mundane: it is not always of a fixed size, and does not even follow cartesian coordinates (as shown in the example of this polar coordinate system). Each RenderObject knows about its parent node, but there is little information about its children other than how to access and obtain their layout constraints. This design allows the RenderObject to be efficiently abstracted, handling a wide variety of usage scenarios.

During the construction phase, a Flutter creates or updates an object inherited from the RenderObject for each RenderObjectElement in the Element tree. Renderobjects are actually primitives: RenderParagraphs for rendering text, RenderImages for rendering images, and RenderTransform for applying transformations before rendering child node content are further implementations.

Update the UI

The flow of Flutter rendering at the Framework level (later handed over to the image engine) is described above, which is just a frame flow. Flutter also has some differences in its UI update.

Update the Widget Tree

The immutability of widgets was mentioned earlier, so every frame calls the Build () method to return the Widget tree, and even the StatefulWidget only returns a different Widget tree depending on its state. In theory, there would be a lot of instance creation and destruction, frequent GC, but in practice, as mentioned above, widgets are only UI configuration, not actual rendering, which is not that expensive.

Update the Element Tree

The heavier Element Tree is not completely re-rendered, but is maintained as the Widget Tree changes (insert, delete, update, move…). , the core methods include:

1.Element calls widget.canupdate () to determine whether the new Widget can be used to update Element.

  static bool canUpdate(Widget oldWidget, Widget newWidget) {
    return oldWidget.runtimeType == newWidget.runtimeType
        && oldWidget.key == newWidget.key;
  }
Copy the code

Update () element.update () element.updatechild ()…

@mustCallsuper void update(covariant Widget newWidget) {_widget = newWidget; }Copy the code

//framework.dart @protected Element updateChild(Element child, Widget newWidget, dynamic newSlot) { if (newWidget == null) { if (child ! = null) deactivateChild(child); return null; } Element newChild; if (child ! = null) { assert(() { final int oldElementClass = Element._debugConcreteSubtype(child); final int newWidgetClass = Widget._debugConcreteSubtype(newWidget); hasSameSuperclass = oldElementClass == newWidgetClass; return true; } ()); if (hasSameSuperclass && child.widget == newWidget) { if (child.slot ! = newSlot) updateSlotForChild(child, newSlot); newChild = child; } else if (hasSameSuperclass && Widget.canUpdate(child.widget, newWidget)) { if (child.slot ! = newSlot) updateSlotForChild(child, newSlot); child.update(newWidget); assert(child.widget == newWidget); assert(() { child.owner._debugElementWasRebuilt(child); return true; } ()); newChild = child; } else { deactivateChild(child); assert(child._parent == null); newChild = inflateWidget(newWidget, newSlot); } } else { newChild = inflateWidget(newWidget, newSlot); } assert(() { if (child ! = null) _debugRemoveGlobalKeyReservation(child); final Key key = newWidget? .key; if (key is GlobalKey) { key._debugReserveFor(this, newChild); } return true; } ()); return newChild; }... static bool canUpdate(Widget oldWidget, Widget newWidget) { return oldWidget.runtimeType == newWidget.runtimeType && oldWidget.key == newWidget.key; }Copy the code

• NewWidget == null — The Widget corresponding to the child node has been removed, directly remove the child element (if any);
• Child == null — indicates that the newWidget is newly inserted, creating child nodes (inflateWidget);
• child ! = null — at this point, there are three cases:
  • If child.widget == newWidget, there is no change before and after child.widget. = newSlot Indicates that the child node has been moved between sibling nodes. Change child.slot to updateSlotForChild.
  • CanUpdate determines whether the Child Element can be modified with the newWidget, and if so, calls the update method.
  • Otherwise, remove the Child Element first and create a new Element child node through the newWidget.

Update the RenderObject Tree

.

Dart VM

DartVM’s memory reclamation mechanism uses a multi-generation lockless garbage collector, optimized for the creation and destruction of the large number of Widgets objects common in UI frameworks. The basic flow: DartVM’s memory allocation strategy is very simple, creating objects only requires moving Pointers over the existing heap, and memory growth is always linear, eliminating the need to find available memory segments:

Dart’s thread-like concept is called Isolate. Each Isolate cannot share memory, so this allocation strategy enables Dart to quickly allocate without locking. Dart also uses the multi-generation algorithm for garbage collection. The new generation uses the “half-space” algorithm to collect memory. When garbage collection is triggered, Dart copies the “active” objects in the current half-space to the standby space, and then releases all memory in the current space:

Dart only operates on a small number of “active” objects and ignores a large number of “dead” objects that are not referenced. This algorithm is also well suited to scenarios where a lot of widgets are reconstructed in the Flutter framework.

Reference:

Overview of the Flutter architecture