Author: Jiang Zejun (True Meaning)

Tao uses Flutter in many business scenarios, and the complexity of business scenarios makes Flutter lag and frame skip more obvious in low-end streaming scenarios than native (Android/iOS) development. Through the analysis of the business layer in the Flutter performance problems of each stage of the rendering process conducted a series of depth after optimization, the average frame rate has reached above 50 frames beyond native performance, but the caton rate is still short of the experience of the best effect, encountered difficult to break through the bottleneck of and technical challenges, the need for technology to try and break.

This paper will describe the underlying principles, optimization ideas, optimization strategies of actual scenes, implementation of core technologies, optimization results and other aspects, hoping to bring some inspiration and help to everyone. We also welcome more exchanges and corrections to build a better Flutter technology community.

Apply colours to a drawing mechanism

Native vs Flutter

Flutter itself is based on the Native system, so the rendering mechanism is very close to Native, as Xiao Yu of Google Flutter team shared [1], as shown below:

Rendering process

In the middle of the figure on the left, the Flutter goes through 8 stages after receiving VSync signal. Data will be submitted to GPU after the Compositing stage.

In the Semantics phase, the information that RenderObject marked needs to be semantically updated will be transmitted to the system to realize auxiliary functions. The semantic interface can help visually impaired users understand the UI content, which is not related to the overall drawing process.

Finalize Tree phase will unmount all inactive elements added to _inactiveElements, which is not related to the overall drawing process.

Therefore, the whole Flutter rendering process focuses on the upper right stage:

GPU Vsync

When the Flutter Engine receives the vertical synchronization signal, it notifies the Flutter Framework to beginFrame and enter the Animation phase.

Animation

The transientCallbacks callback is performed. The Flutter Engine tells the Flutter Framework to drawFrame and enter the Build phase.

Build

Build the data structure of the UI component tree to render, that is, create the corresponding Widget and the corresponding Element.

Layout

The purpose is to calculate the true size of the space taken up by each node for layout, and then update the layout information of all dirty Render objects.

Compositing Bits

Update the RenderObject that needs to be updated.

Paint

Generate Layer Tree, which cannot be used directly. Compositing into a Scene is also required for Rasterize processing. Generally, there are many levels of Flutter, and it is very inefficient to directly transfer each layer to GPU. Therefore, Composite will be made first to improve efficiency. After rasterization, it will be handed over to the Flutter Engine.

Compositing

Synthesize Layout Tree into Scene and create raster images of the current state of Scene, that is, Rasterize them and submit them to Flutter Engine. Finally, Skia submits data to GPU through Open GL or Vulkan interface. The GPU is displayed after processing.

Core rendering stage

Widget

Most of what we write is widgets. Widgets are basically data structures in a component tree that are a major part of the Build phase. The depth of the Widget Tree, setState rationality of the StatefulWidget, unreasonable logic in the build function, and the use of related widgets that call saveLayer are often performance issues.

Element

The Widget is associated with the RenderObject to generate contextual information about the Widget’s Element. Flutter traverses the Element to generate the RenderObject view tree to support the UI structure.

RenderObject

RenderObject determines the Layout information in the Layout phase, and the corresponding Layer is generated in the Paint phase to show its importance. So most of the plotting performance optimization in Flutter occurs here. The data constructed from the RenderObject tree is added to the LayerTree required by the Engine.

Performance Optimization

Understanding the underlying rendering mechanism and the core rendering stage, optimization can be divided into three layers:

The optimization details of each layer are not detailed here, but the actual scene is mainly described in this paper.

Streaming scenarios

Flow component principle

Under the native development, often using RecyclerView/UICollectionView list scenario development; Under the development of Flutter, the Flutter Framework also provides a component of the ListView, which is essentially a SliverList.

The core source

Let’s take a look at SliverList’s core source code:

class SliverList extends SliverMultiBoxAdaptorWidget {

  @override
  RenderSliverList createRenderObject(BuildContext context) {
    final SliverMultiBoxAdaptorElement element = context as SliverMultiBoxAdaptorElement;
    return RenderSliverList(childManager: element);
  }
}

abstract class SliverMultiBoxAdaptorWidget extends SliverWithKeepAliveWidget {

  final SliverChildDelegate delegate;

  @override
  SliverMultiBoxAdaptorElement createElement() => SliverMultiBoxAdaptorElement(this);

  @override
  RenderSliverMultiBoxAdaptor createRenderObject(BuildContext context);
}
Copy the code

SliverList is a RenderObjectWidget (RenderObjectWidget).

Let’s first look at its RenderObject core source code:

class RenderSliverList extends RenderSliverMultiBoxAdaptor { RenderSliverList({ @required RenderSliverBoxChildManager childManager, }) : super(childManager: childManager); @override void performLayout(){ ... Final SliverConstraints constraints = this.constraints; final double scrollOffset = constraints.scrollOffset + constraints.cacheOrigin; final double remainingExtent = constraints.remainingCacheExtent; final double targetEndScrollOffset = scrollOffset + remainingExtent; final BoxConstraints childConstraints = constraints.asBoxConstraints(); . insertAndLayoutLeadingChild(childConstraints, parentUsesSize: true); . insertAndLayoutChild(childConstraints,after: trailingChildWithLayout,parentUsesSize: true); . collectGarbage(leadingGarbage, trailingGarbage); . } } abstract class RenderSliverMultiBoxAdaptor extends RenderSliver ... { @protected RenderBox insertAndLayoutChild(BoxConstraints childConstraints, {@required RenderBox after,... }) { _createOrObtainChild(index, after: after); . } RenderBox insertAndLayoutLeadingChild(BoxConstraints childConstraints, {@required RenderBox after,... }) { _createOrObtainChild(index, after: after); . } @protected void collectGarbage(int leadingGarbage, int trailingGarbage) { _destroyOrCacheChild(firstChild); . } void _createOrObtainChild(int index, { RenderBox after }) { _childManager.createChild(index, after: after); . {if} void _destroyOrCacheChild (RenderBox child) (childParentData. KeepAlive) {/ / in order to better performance not keepAlive, go else logic.... } else { _childManager.removeChild(child); . }}}Copy the code

Check RenderSliverList source, found that for the child to create and removed by its parent class RenderSliverMultiBoxAdaptor. And RenderSliverMultiBoxAdaptor is done through _childManager SliverMultiBoxAdaptorElement namely, the entire process of SliverList drawing layout size by the parent node limit are given.

In streaming scenarios:

• In the process of sliding is through SliverMultiBoxAdaptorElement createChild entering the visual area of the creation of a new child; (That is, each item card of the business scenario)
• In the process of sliding is through SliverMultiBoxAdaptorElement removeChild is not in the visible area of the old child to remove.

We’ll look at SliverMultiBoxAdaptorElement core source code:

class SliverMultiBoxAdaptorElement extends RenderObjectElement implements RenderSliverBoxChildManager { final SplayTreeMap<int, Element> _childElements = SplayTreeMap<int, Element>(); @override void createChild(int index, { @required RenderBox after }) { ... Element newChild = updateChild(_childElements[index], _build(index), index); if (newChild ! = null) { _childElements[index] = newChild; } else { _childElements.remove(index); }... } @override void removeChild(RenderBox child) { ... final Element result = updateChild(_childElements[index], null, index); _childElements.remove(index); . } @override Element updateChild(Element child, Widget newWidget, dynamic newSlot) { ... final Element newChild = super.updateChild(child, newWidget, newSlot); . }}Copy the code

By looking at the SliverMultiBoxAdaptorElement source code can be found that are actually is through the operation of the child parent Element updateChild.

Next, let’s look at Element’s core code:

abstract class Element extends DiagnosticableTree implements BuildContext {
  @protected
  Element updateChild(Element child, Widget newWidget, dynamic newSlot) {
    if (newWidget == null) {
      if (child != null)
        deactivateChild(child);
      return null;
    }
    Element newChild;
    if (child != null) {
      ...
      bool hasSameSuperclass = oldElementClass == newWidgetClass;;
      if (hasSameSuperclass && child.widget == newWidget) {
        if (child.slot != newSlot)
          updateSlotForChild(child, newSlot);
        newChild = child;
      } else if (hasSameSuperclass && Widget.canUpdate(child.widget, newWidget)) {
        if (child.slot != newSlot)
          updateSlotForChild(child, newSlot);
        child.update(newWidget);
        newChild = child;
      } else {
        deactivateChild(child);
        newChild = inflateWidget(newWidget, newSlot);
      }
    } else {
      newChild = inflateWidget(newWidget, newSlot);
    }
    ...
    return newChild;
  }

  @protected
  Element inflateWidget(Widget newWidget, dynamic newSlot) {
    ...
    final Element newChild = newWidget.createElement();
    newChild.mount(this, newSlot);
    ...
    return newChild;
  }

  @protected
  void deactivateChild(Element child) {
    child._parent = null;
    child.detachRenderObject(); 
    owner._inactiveElements.add(child); // this eventually calls child.deactivate() & child.unmount()
    ...
  }
}
Copy the code

RenderObjectElement; RenderObjectElement; RenderObjectElement;

abstract class RenderObjectElement extends Element { @override void mount(Element parent, dynamic newSlot) { super.mount(parent, newSlot); . _renderObject = widget.createRenderObject(this); attachRenderObject(newSlot); . } @override void attachRenderObject(dynamic newSlot) { ... _ancestorRenderObjectElement = _findAncestorRenderObjectElement(); _ancestorRenderObjectElement? .insertChildRenderObject(renderObject, newSlot); . } @override void detachRenderObject() { if (_ancestorRenderObjectElement ! = null) { _ancestorRenderObjectElement.removeChildRenderObject(renderObject); _ancestorRenderObjectElement = null; }... }}Copy the code

By looking at the source code trace above, we know:

In streaming scenarios:

• A new child is created while sliding into the viewable area by creating a new Element and mounting it to the Element Tree. Then create corresponding RenderObject, calls the _ancestorRenderObjectElement? . InsertChildRenderObject;
• Remove the old child that is not visible during sliding, remove the corresponding Element from Element Tree unmount Then call the _ancestorRenderObjectElement removeChildRenderObject.

In fact this _ancestorRenderObjectElement SliverMultiBoxAdaptorElement, we look at the SliverMultiBoxAdaptorElement:

class SliverMultiBoxAdaptorElement extends RenderObjectElement implements RenderSliverBoxChildManager { @override void insertChildRenderObject(covariant RenderObject child, int slot) { ... renderObject.insert(child as RenderBox, after: _currentBeforeChild); . } @override void removeChildRenderObject(covariant RenderObject child) { ... renderObject.remove(child as RenderBox); }}Copy the code

Actually calls are ContainerRenderObjectMixin method, we’ll look at ContainerRenderObjectMixin:

mixin ContainerRenderObjectMixin<ChildType extends RenderObject, ... { void insert(ChildType child, { ChildType after }) { ... adoptChild(child); // attach render object _insertIntoChildList(child, after: after); } void remove(ChildType child) { _removeFromChildList(child); dropChild(child); // detach render object } }Copy the code

ContainerRenderObjectMixin maintains a two-way chain table to hold the current children RenderObject, So in the process of sliding create and remove synchronization in ContainerRenderObjectMixin double linked list to add and remove.

Finally concluded:

• A new child is created while sliding into the viewable area by creating a new Element and mounting it to the Element Tree. Then create corresponding RenderObject, by calling the SliverMultiBoxAdaptorElement. InsertChildRenderObject attach to the Render Tree, And synchronization add RenderObject SliverMultiBoxAdaptorElement mixin by double linked list;
• Remove the old child that is not visible during sliding, remove the corresponding Element from Element Tree unmount . Then by using SliverMultiBoxAdaptorElement removeChildRenderObject corresponding RenderObject from mixins by removing and synchronous double linked list RenderObject from Render Tree detach.

Rendering principle

Through the analysis of the core source code, we can classify the elements of the streaming scene as follows:

Let’s look at the overall rendering process and mechanism when the user swipes up to see more merchandise cards and triggers the loading of the next page of data for display:

• When sliding up, the cards at the top of 0 and 1 move out of the Viewport Area (Visible Area + Cache Area), which we define as entering the Detach Area, After entering the Detach Area, Detach the RenderObject from the Render Tree, unmount the Element from the Element Tree, and synchronously remove the Element from the bidirectional list.
• Monitor the sliding calculation position of the ScrollController to determine whether to start Loading the next page of data. Then the bottom Loading Footer component will enter the visual area or cache area. Need to SliverChildBuilderDelegate childCount + 1, the last child back Loading Footer component, at the same time call setState SliverList refresh to the whole. Update will call performRebuild to rebuild, and the middle part will be updated in the user’s visual area. Then create the Loading Footer component corresponding to the new Element and RenderObject and synchronously add them to the bidirectional list.
• When the loading ends and data is returned, setState will be called again to refresh the entire SliverList. Update will be called performRebuild to rebuild the SliverList. The middle part will be updated in the user’s visual area. Then the Loading Footer component detach the corresponding RenderObject from the Render Tree, unmount the corresponding Element from the Element Tree, and synchronously remove it from the bidirectional list.
• The new item at the bottom goes into the viewable area or cache, and the corresponding new Element and RenderObject need to be created and added simultaneously to the bidirectional list.

Optimization strategy

The above scenario, in which the user slides up to see more merchandise cards and triggers the loading of the next page of data for display, can be optimized in five directions:

Load More

The calculation is performed continuously by listening to the ScrollController slide, preferably without judgment, automatically recognizing the need to load the next page of data and then launching the loadMore() callback. ReuseSliverChildBuilderDelegate loadMore, and add new and item footerBuilder Builder at the corresponding levels, and the default includes Loading Footer component, In SliverMultiBoxAdaptorElement. CreateChild (int index,…). Determine if you need to dynamically call back loadMore() and automatically build the Footer component.

Local refresh

Referring to the previous smoothness optimization of the long list by Idle fish [2], setState is called to refresh the entire SliverList after the data on the next page comes back, resulting in the entire middle part of the SliverList being updated in the user’s visual area. In fact, only the newly created part needs to be refreshed. Optimize SliverMultiBoxAdaptorElement. Update (SliverMultiBoxAdaptorWidget newWidget) part of the implementation of the local refresh, the diagram below:

Element & RenderObject reuse

Referring to xianyu’s previous fluency optimization in long list [2] and Google Android RecyclerView ViewHolder reuse design [3], when a new item is created, ViewHolder similar to Android RecyclerView can be used to hold and reuse components. Based on the analysis of the principle of rendering mechanism, widgets in Flutter can actually be understood as a component tree data structure, that is, more data expression of component structure. We need to cache the Element and RenderObject component types of the removed item. When creating a new item, we will first take them out of the cache and reuse them. Without breaking the Key design of Flutter itself, only reuse elements and RenderObjects with the same Key if a Key is used on an item. However, in the streaming scenario, the list data is different data, so the Key is used in the streaming scenario, which cannot be reused. If you reuse Element and RenderObject, you are not advised to use keys for item components.

We added a cache state to the classification of elements in the original streaming scenario:

The GC inhibition

Dart has its own GC mechanism, similar to Java generational collection, which can suppress GC in the sliding process and customize the GC collection algorithm. In view of this discussion with Flutter experts at Google, Dart does not have multithreaded garbage collection as Java does. Single-threaded garbage collection is faster and lighter. At the same time, the Flutter Engine needs to be deeply modified. Consider the benefits are not large temporarily.

asynchronous

The Flutter Engine restricts the use of Platform apis by non-main isolates, and puts all the logic that does not interact with Platform threads into the new ISOLates. Frequent creation and recycling of isolates may also affect the performance. Flutter compute<Q, R>(isolates.ComputeCallback<Q, R> callback, Q message, {String debugLabel}) creates a new Isolate each time it is invoked. After the task is executed, it is reclaimed and a thread pool-like Isolate is implemented to process non-view tasks. After the actual test, the improvement is not obvious.

Realization of core technology

We can classify the code that calls the link as follows:

The core code

@override void update(covariant ReuseSliverMultiBoxAdaptorWidget newWidget) { ... If (_isPartialRefresh(oldDelegate, newDelegate)) {... int index = _childElements.lastKey() + 1; Widget newWidget = _buildItem(index); // do not create child when new widget is null if (newWidget == null) { return; } _currentBeforeChild = _childElements[index - 1].renderObject as RenderBox; _createChild(index, newWidget); } else { // need to rebuild performRebuild(); }}Copy the code

Element & RenderObject reuse

Call link optimization

• Create: in ReuseSliverMultiBoxAdaptorElement createChild method reads the corresponding component type _cacheElements cache Element for reuse; If there are no reusable Elements of the same type, create a corresponding Element and RenderObject.
• Removed: In ReuseSliverMultiBoxAdaptorElement removeChild method will remove the RenderObject removed from the double linked list, Detach from Element deactive and RenderObject, and update Element _slot to NULL for reuse. The corresponding Element is then cached in a linked list of component types for _cacheElements.

_createChild(int index, Widget newWidget){ Type delegateChildRuntimeType = _getWidgetRuntimeType(newWidget); if(_cacheElements[delegateChildRuntimeType] ! = null && _cacheElements[delegateChildRuntimeType].isNotEmpty){ child = _cacheElements[delegateChildRuntimeType].removeAt(0); }else { child = _childElements[index]; }... newChild = updateChild(child, newWidget, index); . }Copy the code

@override
void removeChild(RenderBox child) {
 ...
 removeChildRenderObject(child); // call removeOnly
 ...
 removeElement = _childElements.remove(index);
 _performCacheElement(removeElement);
 }
Copy the code

The core code

@override void createChild(int index, { @required RenderBox after }) { ... Widget newWidget; if(_isBuildFooter(index)){ // call footerBuilder & call onLoadMore newWidget = _buildFooter(); }else{ newWidget = _buildItem(index); }... _createChild(index, newWidget); . }Copy the code

Overall structure design

• The core optimization capability is concentrated in the Element layer to provide the underlying capability;
• ReuseSliverMultiBoxAdaptorWidget as a base class returns after optimization by default Element;
• Will be unified by the inherited from loadMore and FooterBuilder ability of SliverChildBuilderDelegate ReuseSliverChildBuilderDelegate for exposure to the upper;
• If there are separate custom Widget flow components, the inheritance from RenderObjectWidget directly to the switch for ReuseSliverMultiBoxAdaptorWidget, For example, custom single column list component (ReuseSliverList), waterfall component (ReuseWaterFall) and so on.

Optimization results

Based on the previous series of deep optimizations and the change of Flutter Engine to UC Hummer, optimization variables of streaming scene were separately controlled, and fluency data were obtained by PerfDog to conduct fluency test comparison:

It can be seen that the overall performance data has been improved by optimization. Combined with the test data before Engine replacement, the frame rate has been improved by 2-3 frames on average, and the lag rate has decreased by 1.5 percentage points.

conclusion

use

And the way of the use of native SliverList, widgets into corresponding to reuse components (ReuseSliverList/ReuseWaterFall/CustomSliverList), The delegate if need footer and loadMore use ReuseSliverChildBuilderDelegate; If you don’t need to use native SliverChildBuilderDelegate can directly.

Paging scenario

return ReuseSliverList( // ReuseWaterFall or CustomSliverList delegate: ReuseSliverChildBuilderDelegate( (BuildContext context, int index) { return getItemWidget(index); }, // Build footer footerBuilder: (BuildContext context) {return DetailMiniFootWidget(); }, // add loadMore listener addUnderFlowListener: loadMore, childCount: dataofWidgetList.length));Copy the code

No paging scenario

return ReuseSliverList( // ReuseWaterFall or CustomSliverList
delegate: SliverChildBuilderDelegate(
  (BuildContext context, int index) {
    return getItemWidget(index);
  }, 
  childCount: dataOfWidgetList.length
)
);
Copy the code

Pay attention to the point

Do not add a Key to the item/footer component, otherwise it is considered to reuse only the same Key. Because of the reuse of Element, although the Widget representing the result of the component tree data is updated each time, the StatefulElement State is generated when the Element is created and reused, consistent with the design of Flutter itself. Therefore, you need to retrieve State cached data from the Widget in the didUpdateWidget(covariant T oldWidget).

Reuse Element Lifecycle

The state of each item can be called back, and the upper layer can do logic processing and resource release, etc. For example, the data cached by State in didUpdateWidget(Covariant T oldWidget) is retrieved from the Widget that can be placed in onDisappear or play automatically.

Void onAppear() {} void onDisappear() {}}Copy the code

The resources

[1] Xiao Yu:Flutter Performance Profiling and Theory: files.flutter-io.cn/events/gdd2…

[2] : Xianyu Cloud: He doubled the smoothness of the long list of Xianyu apps

[3] : Google Android RecyclerView.ViewHolder:RecyclerView.Adapter#onCreateViewHolder:developer.android.com/reference/a…

Follow the official account of alibaba Mobile Technology, 3 mobile technology practices & Dry Goods for you to think about every week!