Author: Xianyu Technology – Fuju

preface

Performance stability is the life of App. Flutter brings a lot of innovation and opportunities. However, while enjoying the benefits brought by Flutter, the team also meets the challenges brought by many new things.

This article on the memory optimization process of some practical experience with you to do a share.

Once the Flutter comes online

The Idle Fish uses a hybrid stack management scheme to embed Flutter into the existing App. In terms of product experience, we have achieved better experience than Native. This is mainly due to Flutter’s cross-platform rendering advantage, and partly because the pages we’ve reimplemented in Dart have shed a lot of historical baggage.

After the launch of all aspects of technical indicators, have reached or even exceeded some expectations. Some stability indicators that we are most worried about, such as crash, are also in the stable range. However, after a while we found significant anomalies in the ABORT rate data that was killed by the system due to high memory. Performance and stability issues are critical, so we quickly started troubleshooting problems.

Locating and troubleshooting problems

The obvious problem is excessive memory consumption. Memory consumption in App is relatively complex, how to locate the culprit in the complex business? With a little observation, we determined that the Flutter problem was obvious relative to the valence. You need to better locate memory problems. Using existing tools is very helpful. Fortunately, we have plenty of performance analysis tools available at both the Native layer and the Dart layer.

Tools to analyze

Here’s a quick overview of how we used the tools to look at the phone data to help analyze the problem. It is important to note that this article is not about how to use the tools, so it is just a brief list of common tools used in the section.

Xcode Instruments

Instruments is a great tool for iOS memory screening, which makes it easy to see memory usage in real time, needless to say.

Xcode MemGraph + VMMap

MEMGraph, released after XCode 8, is XCode’s memory debugging tool that allows you to visualize memory in real time. Even more conveniently, you can export MemGraph to work with command line tools to get more structured information.

Dart Observatory

This is the official Dart language debugging tool, which also includes tools similar to Xcode’s Instruments. The Dart VM starts in Debug mode and accepts Debug requests on specific ports. The official documentation

The observations

I made a lot of observations throughout the process, and here are some typical data representations.

Through Xcode Instruments, we see that CG Raster Data is a little high. This Raster Data is actually the memory consumption of rasterizing images.

We exported the MemGraph of the App memory exception scenario and executed the VMMap command to obtain the following results:

vmmap --summary Runner[40957].memgraph

vmmap Runner[40957].memgraph | grep 'IOKit'
Copy the code

We focus on resident and dirty memory. IOKit takes up a lot of memory.

Combined with Xcode Raster Data and IOKit’s high memory consumption, we began to suspect that the problem was caused by a memory leak. The memory of the Dart Image object is further observed through the Dart Observatory.

The result, as I’m sure many of you have already thought, is that the App has obvious memory problems, which are probably related to multimedia resources. With accurate data clues from the tool, we get a general direction for further research.

The number of weird Dart images exploded

Image object leak?

Previously, we observed with tools that the excessive number of Image objects in the Dart layer directly led to very large memory pressure, and we initially suspected that there was a memory leak of the Image. However, after further confirmation, we found that the picture was not really leaked.

The Dart language uses Garbage Collection to manage allocated memory, and Garbage Collection at the VM level should be trusted most of the time. But from a practical point of view, the large memory spikes caused by the explosion in the number of images intuitively make GC a bit late. In Debug mode we manually trigger the GC using the Dart Observatory, and eventually the image objects will be reclaimed without reference.

At this point, we can almost confirm that the image object does not leak. So what’s causing the GC to be slow? Is it the Dart language itself?

Garbage Collection not in time?

For this reason, I need to learn about the implementation of the Dart memory management mechanism garbage collection. About the detailed memory problems, my team member @Jiangxiu has posted a related article for reference: Memory article

Instead of going into the details of Dart garbage collection implementation, I’ll talk about Flutter as it relates to Dart.

  1. Framework (Dart) refers specifically to the code for Flutter written by Dart.

  2. The Dart VM executes the Dart language-related library for Dart code, which is provided as the Dart SDk implemented in C. The Dart Api of the C interface is exposed. It contains the Dart compiler, runtime, and so on.

  3. The FLutter Engine is implemented by C++ to drive the FLutter Engine. He is mainly responsible for cross-platform rendering implementation, including access to the Skia rendering engine; Dart language integration; And some code related to Native layer adaptation and Embeder. Flutter. Framework on iOS and Flutter. Jar on Android are the result of building the engine code.

There is no awareness of GC in the Dart code.

There is a limit to what we can do with the Dart SDK, the Dart language, because the Dart language itself is a standard, and if Dart does have a problem we need to work with the Dart maintenance team to fix it. The Dart language was designed to be transparent to the user and not rely on the specific algorithms and strategies that the GC implements. However, we still need to look at the source code for the Dart SDK to understand what GC looks like.

Since we have previously confirmed that this is not a memory leak, our investigation into GC delays will focus on the Flutter Engine and Dart CG portal.

Flutter and Dart Garbage Collection

Since GC doesn’t feel timely, regardless of consumption, we can at least try to trigger more GC to reduce peak memory stress. But I’m canvassed the dart_api. H (/ SRC/third_party/dart/runtime/include/dart_api. H) after the interface file, but did not find an explicit provide trigger GCS interface.

But this method, Dart_NotifyIdle, was found:

/** * Notifies the VM that the embedder expects to be idle until |deadline|. The VM * may use this time to perform garbage collection or other tasks to avoid * delays during execution of Dart code in the future. * * |deadline| is measured in microseconds against the system's monotonic time. * This clock can be accessed via Dart_TimelineGetMicros().  * * Requires there to be a current isolate. */
DART_EXPORT void Dart_NotifyIdle(int64_t deadline);
Copy the code

This interface means we can explicitly notify Dart when idle, and you can then use that time (before dealine) to do GC. Note that GC is not guaranteed to be performed immediately, and it’s understandable that we’re asking Dart to do GC, depending on Dart’s own strategy.

In addition, I found a method called Dart_NotifyLowMemory:

/** * Notifies the VM that the system is running low on memory. * * Does not require a current isolate. Only valid after  calling Dart_Initialize. */
DART_EXPORT void Dart_NotifyLowMemory(a);
Copy the code

However, the Dart_NotifyLowMemory method has nothing to do with GC. It is a method that terminates the redundant ISOLATE when the memory is low. You can simply clean up some unnecessary threads.

If you look at the Flutter Engine code, you will find that the Flutter Engine actually collaborates with the Dart layer on GC through Dart_NotifyIdle. We can see the following code in the Flutter Engine source animator.cc:

//Animator is responsible for the drawing of refresh and notification frames. frame_scheduled_) { // We don't have another frame pending, so we're waiting on user input // or I/O. Allow the Dart VM 100 ms. delegate_.OnAnimatorNotifyIdle(*this, dart_frame_deadline_ + 100000); } / / the delegate will call here bool RuntimeController: : NotifyIdle (int64_t deadline) {if (! root_isolate_) { return false; } tonic::DartState::Scope scope(root_isolate_.get()); //Dart API Dart_NotifyIdle(deadline); return true; }Copy the code

The logic here is straightforward: NotifyIdle tells the Dart layer that GC is ready if no frame rendering task is currently available. Note that Dart does not go back to GC only in this case, but that Flutter does GC as much as possible in this way, working with Dart to do GC at a more reasonable time.

There was enough reason to give this interface a try, so we manually requested GC in some memory-stressed scenarios. Online Abort is significantly better, but peak memory is not improved. We need to go further to find the root cause.

The truth about the number of pictures exploding

In order to determine the problem of delayed release of a large number of images, we need to track the whole process of Flutter images from initialization to destruction.

We trace the life cycle of the Image object from the Dart layer. We can see that all images inside the Flutter are acquired by the ImageProvider. When the Image is acquired, the ImageProvider calls the Resolve interface. This interface will first query the ImageCache to read the Image, and if there is no cache it will fetch the instance of the new Image.

Key code:

  ImageStream resolve(ImageConfiguration configuration) {
    assert(configuration ! =null);
    final ImageStream stream = new ImageStream();
    T obtainedKey;
    obtainKey(configuration).then<void>((T key) {
      obtainedKey = key;
      stream.setCompleter(PaintingBinding.instance.imageCache.putIfAbsent(key, () => load(key)));
    }).catchError(
      (dynamic exception, StackTrace stack) async {
        FlutterError.reportError(new FlutterErrorDetails(
          exception: exception,
          stack: stack,
          library: 'services library',
          context: 'while resolving an image',
          silent: true.// could be a network error or whatnot
          informationCollector: (StringBuffer information) {
            information.writeln('Image provider: $this');
            information.writeln('Image configuration: $configuration');
            if(obtainedKey ! =null)
              information.writeln('Image key: $obtainedKey'); }));return null; });return stream;
  }
Copy the code

Rough logic

  1. Resolve requests to get the image.
  2. Query whether ImageCache exists.Yes->3 NO->4
  3. Returns an existing picture object
  4. Generating a new Image object and starting to load it does not seem particularly complicated, but here I will mention the implementation of the Flutter ImageCache.

Flutter ImageCache

The original version of Flutter ImageCache was actually very simple, based on LRU algorithm caching implemented with Map. There is nothing wrong with this algorithm and implementation, but it is important to note that ImageCache caches an ImageStream object, which is an asynchronously loaded image object. There is no limit to the total amount of memory that can be used by the cache, but a default maximum of 1000 objects. (Flutter has a memory size limit added to the logic of Flutter 0.5.6 beta.) One problem with caching asynchronously loaded objects is that there is no way to know how much memory will be consumed until the image load decoding is complete. At least this problem is not addressed in the Cache implementation of Flutter. Dart source code for imagecache.dart.

The Flutter itself provides the ability to customize the Cache, so the first step in optimizing the ImageCache is to adjust the Cache size to fit the physical memory of the Flutter model and set the appropriate limit for the ImageCache. About ImageCache, you can refer to the official document and this issue. I won’t talk about it here.

Flutter Image lifecycle

Going back to our Image object trace, it is clear that a new Image will be generated in the event of a cache miss. Digging deeper into the code, you can see that the Image object is generated by this code:


Future<Codec> instantiateImageCodec(Uint8List list) {
  return _futurize(
    (_Callback<Codec> callback) => _instantiateImageCodec(list, callback, null)); }String _instantiateImageCodec(Uint8List list, _Callback<Codec> callback, _ImageInfo imageInfo)
  native 'instantiateImageCodec';
Copy the code

There is a native keyword, which is Dart’s ability to call C code. If we look at the source code, we can see that the final initialization is a C++ Codec object. The specific code for Flutter Engine codec.cc. It basically starts a decoding task in the IO thread and sends the final image object back to the UI thread after the IO completes. The detailed introduction of Flutter threads has been described in another article of mine. Here is the link for those who are interested. In-depth understanding of the Flutter Engine thread model. After this code and thread analysis, we get a rough flow chart:

In other words, decoding tasks in the IO thread, IO task queue are C++ lambda expressions, holding the actual decoding object, also hold memory resources. When there are too many IO thread tasks, there are too many IO tasks waiting to be executed, and these memory resources are held by closures waiting to be released. This is why there is an intuitive problem with memory spikes caused by memory being released late. This also explains why IOKit is the largest part of the VMMap virtual memory data we received earlier.

In this way, we found the key clue. In the case of cache failure, a large number of Image objects were initialized, which resulted in heavy tasks for IO thread, and IO held a large number of memory resources for Image decoding. With this inference, I added the monitoring code for the number of tasks and C++ image objects to the Task Runner of the Flutter Engine to confirm that there was indeed an overload of IO Task threads, with the peak reaching 100+IO operations in extreme cases.

At this point the question seems more and more clear, but why do I/O tasks trigger so much? The above logic may take up a lot of memory if the IO thread is overloaded. There is nothing wrong with a request to generate a new picture object, and that is the design. For example, the main thread blocks a large number of tasks, which inevitably causes the interface to stall, but it is not the main thread itself. We need to find out what causes new object creation inflation to actually overload IO threads.

The root of a large number of requests

Under the previous clues, we continue to look for the root of the problem. In the actual App operation process, we found that the more pages Push, the faster the image generation. The more pages you have, the faster the request, so it doesn’t seem like a big deal. But the number of visible images is always within a certain range, and the frequency of object creation should not be accelerated as pages grow. Subconsciously, we began to wonder if an invisible Image Widget was constantly requesting images. As a result, the Cache fails to hit and a large number of new images are generated.

I started investigating image loading requests for each page. We knew that everything in Flutter was a Widget, and the page was a Widget managed by the Navigator. I added monitoring code to the Widget’s lifecycle method (see the official Flutter documentation for details). As I expected, the Resolve Image of the page that was not visible at the bottom of the Navigator stack also continued. This directly caused the Image object to pop and the IO thread to be overloaded, resulting in memory spikes.

At last, it seems, we have found the root cause. The solution is not hard. There is no need to make extra image load requests when the page is not visible, and the peak drops accordingly. After some code optimization and testing, the problem was fundamentally solved. After optimization went online, we saw a qualitative improvement in the data. Some of you might wonder why invisible widgets are called to their associated lifecycle methods. I recommend reading the official Flutter documentation about widgets. I don’t have enough space to cover them here. widgets

At this point, we have solved one of the more serious memory problems. Memory optimization is complicated and can be clicked more, but I will continue to briefly share some other aspects of optimization.

Screenshot cache optimization

File caching + preloading strategy

We take embedded Flutter and use a set of hybrid stack modes to manage the logic of Native and Flutter pages jumping to each other. Since FlutterView exists in the singleton form in App, we used screenshots to transition the page switching process for better user experience.

As we all know, images are very memory intensive objects, so how can we minimize memory consumption without degrading the user experience? If we kept a screenshot for every page push, memory would grow linearly, which would not be good enough.

Memory and space in most cases is a mutual conversion relationship, optimization is a lot of time to find a reasonable compromise. In the end, I adopted the strategy of preloading and caching, in which at most two screenshots were stored in memory at the same time on the page, and other files were stored and preloaded in advance when needed. Brief flow chart:

This reduces the space complexity from O(n) to O(1) without affecting the user experience. This optimization further saves unnecessary memory overhead.

Additional optimizations for screenshots

  • The resolution of screenshots can be adjusted adaptively according to the memory of the current device to minimize memory consumption.
  • In extreme memory cases, remove all screenshots from memory (stored files can be recovered) and place them in PlaceHolder form. De-escalation strategies to avoid getting killed and ensure usability in extreme situations.

Page bottom strategy

There is a common problem for e-commerce apps, users will constantly push pages into the stack, we can not prevent users from this behavior. We could of course kill the old page and reload it every time we go back, but this user experience is just as unacceptable as a Web page. We need to maintain the state of the page to ensure the user experience. This inevitably leads to linear memory growth, which inevitably kills. The goal of our optimization is to increase the maximum number of pages a user can push.

For Flutter page optimization, in addition to optimizing the memory consumption of each page, we implemented a downsizing strategy to ensure the usability of the App: old pages were destroyed in extreme cases and re-created when needed. This does degrade the user experience, and in extreme cases, the degraded experience is still better than Crash.

FlutterViewController singleton destructor

Another topic I want to discuss is FlutterViewController. Flutter is currently designed to run in singleton mode, which should not be a problem for apps completely redeveloped with Flutterc. But for hybrid apps, the extra resident memory is a real problem.

In fact, the underlying implementation of the Flutter Engine takes the issue of destructor into account and has related interfaces. However, in the Embeder layer (specifically, the FlutterViewController Message Channels), there are some circular references in the implementation that cannot be released even when the Native layer does not reference the FlutterViewController.

After a period of time, I tried to remove the circular reference. These circular references are mainly concentrated in FlutterChannel. I freed the FlutterViewController successfully after undoing it, and you can clearly see that the resident memory has been freed. However, I found that releasing the FlutterViewController caused some Skia Image Objects to leak, because Skia Objects had to be released in the thread it created (see skia_gpu_object.cc source code for details), thread synchronization issues. I have an issue on GitHub for your reference. FlutterViewController release issue

At present, we have fed back this optimization to the Flutter team, looking forward to their official support. I hope we can explore and study together.

Further discussion

In addition, there are many aspects of Flutter memory that can be studied. Here are a few of the problems that have been observed so far.

  1. During my memory analysis, I found that the Boring SSL library underlying Flutter had identified memory leaks. Although this leak is relatively slow, it still has an impact on the long-term operation of the App. I put forward an issue on GitHub to follow up, and relevant personnel have already followed up. SSL leak issue

  2. Regarding image rendering, there is currently room for optimization with Flutter, especially with image tailoring on demand. In most cases, there is no need to extract the entire bitmap into memory. We can scale the image reasonably for the display area size and screen resolution to achieve the best performance consumption.

  3. While analyzing the MemGraph of Flutter memory, I found that the Skia engine was consuming a lot of memory for TextLayout. At present, I have not found the specific reason, and there may be room for optimization.

conclusion

In this article, I briefly discuss the team’s current attempts and explorations with Flutter application memory. A short article can not contain all the content, only introduced a few typical cases for analysis, I hope to discuss with you. Welcome interested friends to study together, if you have better ideas, I am very happy to see your share.

Idle fish is looking forward to your joining us

Welcome to join idle Fish to explore more possibilities of Flutter. Resume: [email protected]

The resources

  • (github.com/flutter/flu… Development of the document
  • (github.com/flutter/eng… Engine document
  • IO /) flutter IO
  • (www.dartlang.org/) Dart language development documentation