1. The Flutter architecture

The architecture of Flutter is divided into three layers :Framework, Engine, and Embedder.

1. Dart is used to implement the Framework, including Material Design style Widgets,Cupertino(for iOS) style Widgets, text/image/button and other basic Widgets, rendering, animation, gestures, etc. The core code of this part is: Flutter package under the flutter repository, IO, Async, UI and other packages under the Sky_engine repository (DART: UI library provides the interface between the Flutter framework and the engine).

2.Engine is implemented in C++, including Skia,Dart and Text. Skia is an open source two-dimensional graphics library that provides a common API for a variety of hardware and software platforms.

3.Embedder is an Embedder layer that allows the Flutter to be embedded to various platforms. The main tasks here include rendering Surface Settings, thread Settings, plug-ins, etc. From this we can see that the platform-dependent layer of Flutter is very low. Platforms (such as iOS) only provide a canvas, and all the remaining render related logic is inside Flutter, which makes it very cross-end consistent.

2. Draw the Flutter view

For developers who use the framework most, I will start with the entry functions of a Flutter and work my way down to analyze how a Flutter view is drawn.

The simplest implementation of the main() function in the Flutter application is as follows

// Parameter app is the first widget to be displayed when the Flutter application starts.
void runApp(Widget app){ WidgetsFlutterBinding.ensureInitialized() .. scheduleAttachRootWidget(app) .. scheduleWarmUpFrame(); }Copy the code

1.WidgetsFlutterBinding

WidgetsFlutterBinding inherited from BindingBase and mixed with a lot of Binding, The Binding basically listens for and processes events on the Window object (containing information about the current device and system, as well as some callbacks to the Flutter Engine). These events are then wrapped, abstracted, and distributed according to the Framework’s model.

The WidgetsFlutterBinding is the glue that binds the Flutter Engine to the upper Framework.

  1. GestureBinding: provides a window onPointerDataPacket callback, binding Framework gestures subsystem, is the binding Framework model and the underlying events entrance.

  2. ServicesBinding: Provides window.onPlatformMessage callback for binding platform message channels, which handle native and Flutter communication.

  3. SchedulerBinding: Provides window.onBeginFrame and Window. onDrawFrame callbacks, listens for refresh events, and binds the Framework to draw scheduling subsystems.

  4. PaintingBinding: Binds the drawing library to handle image caching.

  5. SemanticsBinding: A bridge between the semantic layer and the Flutter engine, mainly providing low-level support for ancillary functions.

  6. RendererBinding: provides a window. OnMetricsChanged, window. OnTextScaleFactorChanged callback, etc. It is the bridge between the render tree and the Flutter engine.

  7. WidgetsBinding: Provides callbacks to window.onLocaleChanged, onBuildScheduled, etc. It is the bridge between the Flutter Widget layer and the Engine.

WidgetsFlutterBinding. The ensureInitialized () is responsible for the initialization of a global WidgetsBinding singleton, the code is as follows

class WidgetsFlutterBinding extends BindingBase with GestureBinding.ServicesBinding.SchedulerBinding.PaintingBinding.SemanticsBinding.RendererBinding.WidgetsBinding {
  static WidgetsBinding ensureInitialized() {
    if (WidgetsBinding.instance == null)
      WidgetsFlutterBinding();
    returnWidgetsBinding.instance; }}Copy the code

See this WidgetsFlutterBinding mixed with (with) a lot of Binding, let’s look at the parent class BindingBase:

abstract class BindingBase {... ui.SingletonFlutterWindow getwindow => ui.window;// Get the window instance
  @protected
  @mustCallSuper
  void initInstances(){ assert(! _debugInitialized); assert(() { _debugInitialized =true;
      return true;
    }());
  }
}
Copy the code

Window => UI. Window links to the host operating system interface, that is, the Flutter Framework links to the host operating system interface. The system has an instance of Window, which can be obtained from the Window property.

// The window is of type FlutterView, which has a PlatformDispatcher attribute
ui.SingletonFlutterWindow get window => ui.window;
/ / initialize the PlatformDispatcher. The instance was introduced into, to complete the initialization
ui.window = SingletonFlutterWindow._(0, PlatformDispatcher.instance);
// SingletonFlutterWindow class structure
class SingletonFlutterWindow extends FlutterWindow {.../ / is actually give platformDispatcher onBeginFrame assignment
  FrameCallback? get onBeginFrame => platformDispatcher.onBeginFrame;
  set onBeginFrame(FrameCallback? callback) {
    platformDispatcher.onBeginFrame = callback;
  }
  
  VoidCallback? get onDrawFrame => platformDispatcher.onDrawFrame;
  set onDrawFrame(VoidCallback? callback) {
    platformDispatcher.onDrawFrame = callback;
  }
  
  / / window. ScheduleFrame is actually call platformDispatcher. ScheduleFrame ()
  voidscheduleFrame() => platformDispatcher.scheduleFrame(); . }class FlutterWindow extends FlutterView {
  FlutterWindow._(this._windowId, this.platformDispatcher);
  final Object _windowId;
  // PD
  @override
  final PlatformDispatcher platformDispatcher;
  @override
  ViewConfiguration get viewConfiguration {
    return platformDispatcher._viewConfigurations[_windowId]!;
  }
}
Copy the code

2.scheduleAttachRootWidget

ScheduleAttachRootWidget then calls the WidgetsBinding attachRootWidget method, which is responsible for adding the root Widget to the RenderView as follows:

 void attachRootWidget(Widget rootWidget) {
    final bool isBootstrapFrame = renderViewElement == null;
    _readyToProduceFrames = true;
    _renderViewElement = RenderObjectToWidgetAdapter<RenderBox>(
      container: renderView,
      debugShortDescription: '[root]'.child: rootWidget, ).attachToRenderTree(buildOwner! , renderViewElementasRenderObjectToWidgetElement<RenderBox>?) ;if (isBootstrapFrame) {
      SchedulerBinding.instance!.ensureVisualUpdate();
    }
  }
Copy the code

The renderView variable is a RenderObject, which is the root of the render tree. The renderViewElement variable is the Element object corresponding to the renderView. As you can see, this method does the whole process of associating the root widget to the root RenderObject and then to the root Element.

RenderView get renderView => _pipelineOwner.rootNode! as RenderView;
Copy the code

RenderView is PipelineOwner. PipelineOwner is PipelineOwner, which plays an important role in Rendering pipelines.

Collect “Dirty Render Objects” as the UI changes and drive Rendering Pipeline to refresh the UI.

PipelineOwner is a bridge between RenderObject Tree and RendererBinding.

Final call attachRootWidget, execution will call RenderObjectToWidgetAdapter attachToRenderTree method, this method is responsible for creating the root element, namely RenderObjectToWidgetElement, We also associate the Element with the widget, creating an Element tree corresponding to the widget tree. If the Element is already created, set the widget associated with the root Element to new so that you can see that the element is created only once and reused later. BuildOwner is a widget framework management class that keeps track of which widgets need to be rebuilt. The following code

RenderObjectToWidgetElement<T> attachToRenderTree(BuildOwner owner, [RenderObjectToWidgetElement<T> element]) {
  if (element == null) { owner.lockState(() { element = createElement(); assert(element ! =null);
      element.assignOwner(owner);
    });
    owner.buildScope(element, () {
      element.mount(null.null);
    });
  } else {
    element._newWidget = this;
    element.markNeedsBuild();
  }
  return element;
}
 
Copy the code

3.scheduleWarmUpFrame

In runApp, when attachRootWidget is called, the last line calls scheduleWarmUpFrame() of the WidgetsFlutterBinding instance, which is implemented in SchedulerBinding. This method locks event distribution until the draw is complete. This means that the Flutter will not respond to events until the draw is complete. This ensures that no new redraw will be triggered during the draw.

Here is a partial implementation of the scheduleWarmUpFrame() method (exsanguine code omitted) :

void scheduleWarmUpFrame(){... Timer.run(() { handleBeginFrame(null); 
  });
  Timer.run(() {
    handleDrawFrame();  
    resetEpoch();
  });
  // Lock the event
  lockEvents(() async {
    awaitendOfFrame; Timeline.finishSync(); }); . }Copy the code

HandleBeginFrame () and handleDrawFrame() are called in this method

HandleBeginFrame () and handleDrawFrame() are transientCallbacks. The latter executes the persistentCallbacks and postFrameCallbacks queues.

1. TransientCallbacks: Used to store temporary callbacks, usually animation callbacks. Can pass SchedulerBinding. Instance. ScheduleFrameCallback add callbacks. PersistentCallbacks: persistentCallbacks hold persistentCallbacks that do not require new frames to be drawn. PersistentCallbacks cannot be removed once registered. SchedulerBinding. Instance. AddPersitentFrameCallback (), the callback in processing the layout and drawing work. PostFrameCallbacks: at the end of the Frame is called only once, after the call will be system removed, by SchedulerBinding. Instance. AddPostFrameCallback register (). Be careful not to trigger a new Frame in such a callback, as this can cause a loopCopy the code

The real rendering and rendering logic is implemented in RendererBinding. If you look at the source code, you’ll find the following code in its initInstances() method:

void initInstances(){...// omit extraneous code
  addPersistentFrameCallback(_handlePersistentFrameCallback);
}
void _handlePersistentFrameCallback(Duration timeStamp) {
  drawFrame();
}
void drawFrame(){ assert(renderView ! =null);
  pipelineOwner.flushLayout(); / / layout
  pipelineOwner.flushCompositingBits(); // Preprocess the RenderObject to see if it needs to be redrawn
  pipelineOwner.flushPaint(); / / redraw
  renderView.compositeFrame(); // Send the bits to be drawn to the GPU
  pipelineOwner.flushSemantics(); // this also sends the semantics to the OS.
}
Copy the code

RendererBinding is a mixin, and the WidgetsBinding with RendererBinding is a WidgetsBinding, so we need to see if this method can be overridden in a WidgetsBinding.

@override
void drawFrame(){...// omit extraneous code
  try {
    if(renderViewElement ! =null)
      buildOwner.buildScope(renderViewElement); 
    super.drawFrame(); // Call RendererBinding's drawFrame() methodbuildOwner.finalizeTree(); }}Copy the code

In the call RendererBinding. DrawFrame () method will be called before buildOwner. BuildScope () (drawing) for the first time, This method will rebuild() elements marked as “dirty”. Let’s look at WidgetsBinding again. Create a BuildOwner object in initInstances() and execute BuildOwner! .onBuildScheduled = _handleBuildScheduled; , which assigns _handleBuildScheduled to the buildOwnder onBuildScheduled property.

The BuildOwner object, which keeps track of which widgets need to be rebuilt and other tasks that should be used in the Widgets tree, maintains an internal list of _dirtyElements that are marked “dirty.”

As each element is created, its BuildOwner is identified. A page has only one buildOwner object, which manages all elements of that page.

// WidgetsBinding
void initInstances(){... buildOwner! .onBuildScheduled = _handleBuildScheduled; . } ()); } when calling buildOwner. OnBuildScheduled (), will go the following process./ / WidgetsBinding class
void _handleBuildScheduled() {
  ensureVisualUpdate();
}
/ / SchedulerBinding class
void ensureVisualUpdate() {
    switch (schedulerPhase) {
      case SchedulerPhase.idle:
      case SchedulerPhase.postFrameCallbacks:
        scheduleFrame();
        return;
      case SchedulerPhase.transientCallbacks:
      case SchedulerPhase.midFrameMicrotasks:
      case SchedulerPhase.persistentCallbacks:
        return; }} When the schedulerPhase is idle, scheduleFrame is called and passeswindowScheduleFrame () in the performDispatcher. ScheduleFrame () go to register a VSync listening invoid scheduleFrame(){...window.scheduleFrame(); . }Copy the code

4. Summary

The Flutter from startup to display images on the screen mainly goes through: First listen for events that handle window objects, wrap these event handlers into Framework models for distribution, create the Element tree with widgets, render with scheduleWarmUpFrame, and lay out and draw with Rendererbinding. Finally, the scene information is sent to the Flutter engine by calling uI.window.render (scene). The Flutter engine finally calls the render API to draw the image on the screen.

I have briefly rearranged the sequence diagram drawn by the Flutter view as follows

3. Monitor Flutter performance

After a certain understanding of view rendering, how can WE control and optimize the performance of view rendering? Let’s first take a look at the two performance monitoring tools provided by Flutter officials

1.Dart VM Service

1.observatory

observatory: In engine/shell/testings/observatory can find its concrete implementation, it opened a ServiceClient, used to get dartvm running state. The flutter When the app starts, it generates a current observatory server address

Flutter: socket connected service in the Dart VM service Protocol v3.44 listening on http://127.0.0.1:59378/8x9XRQIBhkU=/Copy the code

For example, after selecting Timeline, performance analysis can be performed, as shown in the figure

2.devTools

DevTools also provides some basic checks, not as detailed as Observatory. Visibility is strong

You can install it by using the following command

flutter pub global activate devtools
Copy the code

After the installation is complete, run the devtools command to open it and enter the DartVM address

The page is displayed

Timeline in DevTools is performance. The page is as follows after we select it, and the operation experience is much betterDart VM Service (vm_service) Dart VM Service (vm_service) Dart VM Service (VM_service)

Dart is a set of Web services provided within the Dart VIRTUAL machine over jSON-RPC 2.0.

However, we don’t need to implement the data request parsing ourselves. The official Dart SDK has been written for us to use: vm_service. When vm_service is started, it will start a WebSocket service locally. The service URI can be obtained in the corresponding platform:

1) Android in FlutterJNI. GetObservatoryUri ();

2) iOS in FlutterEngine. ObservatoryUrl.

After we have the URI, we can use the vm_service service. There is an SDK written for us: vm_service

 Future<void> connect() async {
    ServiceProtocolInfo info = await Service.getInfo();
    if (info.serverUri == null) {
      print("service protocol url is null,start vm service fail");
      return;
    }
    service = await getService(info);
    print('socket connected in service $info');
    vm = awaitservice? .getVM(); List<IsolateRef>? isolates = vm? .isolates; main = isolates? .firstWhere((ref) = > ref.name?.contains('main') = =true); main ?? = isolates? .first; connected =true;
  }

  
  Future<VmService> getService(info) async {
    Uri uri = convertToWebSocketUrl(serviceProtocolUrl: info.serverUri);
    return await vmServiceConnectUri(uri.toString(), log: StdoutLog());
  }
Copy the code

To get the frameworkVersion, call callExtensionService of a VmService instance and pass ‘flutterVersion’ to get the current flutter framework and engine information

Future<Response? > callExtensionService(String method) async {
    if (_extensionService == null&& service ! =null&& main ! =null) { _extensionService = ExtensionService(service! , main!) ;await_extensionService? .loadExtensionService(); }return_extensionService! .callMethod(method); }Copy the code

To get memory information, call getMemoryUsage of a VmService instance to get the current memory information

  Future<MemoryUsage> getMemoryUsage(String isolateId) =>
      _call('getMemoryUsage', {'isolateId': isolateId});
Copy the code

To obtain the FPS of the Flutter APP, there are several ways that we can use to view FPS and other performance data during the development of the Flutter APP, such as DevTools. For details, see the Debugging Flutter apps and Flutter performance profiling documents.

// Register when you need to listen on FPS
void start() {
  SchedulerBinding.instance.addTimingsCallback(_onReportTimings);
}
// Remove it when no listener is needed
void stop() {
  SchedulerBinding.instance.removeTimingsCallback(_onReportTimings);
}
void _onReportTimings(List<FrameTiming> timings) {
  // TODO
}
Copy the code

2. Crash logs are captured and reported

There are two main aspects of flutter crash log collection:

1) Exception of flutter dart code (including APP and Framework code)

2) Crash logs of flutter Engine (usually flash back)

Dart has the concept of a Zone, sort of like a sandbox. Different Zone code contexts are not affected by each other, and zones can create new sub-zones. The Zone can redefine its print, timers, microtasks, and most importantly how Uncaught errors are handled as uncaught exceptions

runZoned(() {
    Future.error("asynchronous error");
}, onError: (dynamic e, StackTrace stack) {
    reportError(e, stack);
});
 
Copy the code

1. Capture the Flutter framework exception

Register the FlutterError. OnError callback to collect exceptions thrown outside the Flutter framework.

FlutterError.onError = (FlutterErrorDetails details) {
    reportError(details.exception, details.stack);
};
Copy the code

2. The Flutter engine catches exceptions

The exception of the Flutter engine, for example Android, is mainly caused by libfutter.so.

This part can be handled directly by native crash collection SDKS such as Firebase Crashlytics, Bugly, xCrash, etc

We need to pass the Dart exception and stack to the Bugly SDK via the MethodChannel.

After collecting exceptions, you need to refer to the symbols table to restore the stack.

To check the version of the flutter engine, run the following command:

The output of flutter –version is as follows:

Flutter 2.23. • channel stable • https://github.com/flutter/flutter.git
Framework • revision f4abaa0735 (4Have a line)2021-07-01 12:46:11 -0700
Engine • revision 241c87ad80
Tools • Dart 2.134.
Copy the code

The revision for Engine is 241C87AD80.

Second, find the symbols.zip file corresponding to the CPU ABI on Flutter infra and download it. After decompression, you can get the debug so file — libflutter. Bugly, for example, offers uploading tools

java -jar buglySymbolAndroid.jar -i xxx

4. Optimization of Flutter performance

In business development, we should learn to use DevTools to detect engineering performance, which will help us achieve more robust applications. In the process of investigation, I found that the rendering time of the video detail page was time-consuming, as shown in the figure

1. Build time optimization

The Build time of the VideoControls is 28.6ms, as shown in the figure

Therefore, our optimization plan here is to improve build efficiency, reduce the starting point of Widget Tree traversal, and try to deliver the setState refresh data to the underlying node, so extract the child components in VideoControl that trigger the refresh into independent widgets. SetState is delivered inside the extracted Widget

After optimization, it is 11.0ms, and the overall average frame rate reaches 60fps, as shown in the figure

2. Paint time optimization

Did paint process can be optimized under the next analysis part, we opened debugProfilePaintsEnabled variable analysis can see Timeline display paint level, as shown in figure

We found that the frequently updated _buildPositionTitle is in the same layer as the other widgets. The optimization point we thought of here is to use the RepaintBoundary to improve paint efficiency. It provides a new layer of isolation for content that changes frequently. The new layer paint does not affect other layers

Take a look at the optimized effect, as shown in the figure

3. Summary

During the development of Flutter, we used DevTools to troubleshoot localization page rendering problems in two main ways:

1. Improve build efficiency and send setState refresh data to the underlying node as far as possible.

2. Improve paint efficiency. RepaintBoundry creates a separate layer to reduce repainting areas.

Of course, performance tuning of Flutter goes beyond this. There are many details that can be optimized for each build/layout/paint process.

5. To summarize

1. Review

This article mainly introduced the technology of Flutter from three dimensions. To explain the rendering principle, we reviewed the source code and found that the whole rendering process of Flutter was a closed loop. The Framework, Engine and Embedder each had their own functions. The Framework handed the DART code to Engine to translate into cross-platform code, which was then called back to the host platform via Embedder. Performance monitoring is constantly inserting our sentinels into this cycle, looking at the whole ecosystem and getting abnormal data to report; Performance optimization Through a project practice, we learned how to use tools to improve our efficiency in locating problems.

2. The advantages and disadvantages of

Advantages:

We can see that Flutter forms a closed loop during view drawing, and both ends basically maintain consistency, so our development efficiency is greatly improved, and performance monitoring and optimization are convenient.

Disadvantages:

1) Declarative development is not very friendly to dynamically manipulate view nodes. It is not as imperative as native programming, or as easy as front-end dom node acquisition

2) To achieve dynamic mechanism, there is no good open source technology to reference