Introduction:

When using the image component of Flutter, have you ever considered how Flutter loads a web image? And what improvements can be made to the built-in image components?

The problem

  1. How is the Flutter network image requested?

  2. Is this how the image is displayed after a successful request? How does each frame of a GIF support presentation?

  3. How do I support disk caching of images?

Next, let’s explore the inner workings of the Flutter image component with questions

The source code analysis in this paper is based on the FLUTTER version 1.22 and only involves dart end, not c-layer image decoding

Image’s core class diagram and its relationships

Draw another one yourself

  • Image, it is a statefulWidget, flutter Image at the core of the entry class, contains a network, file, and assert that the memory that several major function, subcontract correspondence network pictures, picture files, APP built-in assert pictures, stream parsing images from a file
  • _ImageState, because Image is a statefulWidget, the core code is in _ImageState
  • ImageStream, a bridge between image resources, ImageState and ImageStreamCompleter
  • ImageInfo, a store of native information for images
  • ImageStreamCompleter, which can be interpreted as a frame that parses an image and calls back the parsed data to the presenter, has two main implementation classes
    • OneFrameImageStreamCompleter single frame image parser (seemingly not in use)
    • MultiFrameImageStreamCompleter multiframe images parser, all in the source images are default to this
  • ImageProvider, an image loader, has different implementations for different loading methods
    • NetworkImage Loads images from the network
    • MemoryImage loads images from a binary stream
    • AssetImage loads the image in the asset
    • FileImage loads images from a file
  • ImageCache, the ImageCache that flutter contains, has only memory cache. The official cache contains a maximum of 100 images and a maximum memory of 100MB
  • Load in a fast sliding ScrollAwareImageProvider, avoid images

The loading process of network pictures

Image.network(imgUrl, // image link width: w, height: h),)Copy the code

As mentioned above,Image is a StatefulWidget, so the core logic is to look at the corresponding ImageState,ImageState inherits from State, the life cycle of State we know, The first initialization is done in the order InitState()->didChangeDependencies->didUpdateWidget()-> Build ()

The InitState of ImageState does nothing, the image request is initiated in didChangeDependencies, okay

// ImageState->didChangeDependencies @override void didChangeDependencies() { We don't parse _updateInvertColors(); // From here, provier, stream,completer all appear _resolveImage(); If (tickermode.of (context)) _listenToStream(); else _stopListeningToStream(); super.didChangeDependencies(); }Copy the code

Look again at the _resolveImage method in ImageState

Void _resolveImage() {// ScrollAwareImageProvider specifies the proxy mode. It's also an inherited ImageProvider, Provider = ScrollAwareImageProvider<dynamic>(context: _scrollAwareContext, imageProvider: widget.image, ); // Call ImageProvider resolve, Picture request the main process of final ImageStream newStream = provider. Resolve (createLocalImageConfiguration (context, size: widget. The width! = null && widget.height ! = null ? Size(widget.width, widget.height) : null, )); assert(newStream ! = null); _updateSourceStream(newStream); }Copy the code

Let’s move on to the Resolve method of the ImageProvider

// Create an ImageStream. // Create an ImageStream. Create a Key. The Key is implemented by the provider itself. This Key is used in ImageCache. Encapsulate the rest of the process in a Zone that catches both synchronous and asynchronous exceptions, @NonVirtual ImageStream Resolve (ImageConfiguration Configuration) {assert(Configuration! = null); final ImageStream stream = createStream(configuration); // Create a key, wrap the subsequent process in the zone, source code I do not paste, _createErrorHandlerAndKey(configuration, (T key, ImageErrorListener errorHandler) { resolveStreamForKey(configuration, stream, key, errorHandler); }, (T? key, dynamic exception, StackTrace? stack) async { await null; // wait an event turn in case a listener has been added to the image stream. final _ErrorImageCompleter imageCompleter =  _ErrorImageCompleter(); stream.setCompleter(imageCompleter); InformationCollector? collector; assert(() { collector = () sync* { yield DiagnosticsProperty<ImageProvider>('Image provider', this); yield DiagnosticsProperty<ImageConfiguration>('Image configuration', configuration); yield DiagnosticsProperty<T>('Image key', key, defaultValue: null); }; return true; } ()); imageCompleter.setError( exception: exception, stack: stack, context: ErrorDescription('while resolving an image'), silent: true, // could be a network error or whatnot informationCollector: collector, ); }); return stream; }Copy the code

In 1.22, the default provider was ScrollAwareImageProvider. ScrollAwareImageProvider overwrites resolveStreamForKey. There is the logic for scrolling control loading, but ultimately the resolveStreamForKey of ImageProvier is called

// ImageProvier -> resolveStreamForKey @protected void resolveStreamForKey(ImageConfiguration configuration, ImageStream stream, T key, ImageErrorListener handleError) {// Streem already has completers, from the cache, if (stream.completer! = null) { final ImageStreamCompleter? completer = PaintingBinding.instance! .imageCache! .putIfAbsent( key, () => stream.completer! , onError: handleError, ); assert(identical(completer, stream.completer)); return; } // If it's the first time, create a new completer, and then the load is executed. The second entry to putIfAbsent, final ImageStreamCompleter? completer = PaintingBinding.instance! .imageCache! .putIfAbsent( key, () => load(key, PaintingBinding.instance! .instantiateImageCodec), onError: handleError, ); // Assignment, notice this, I'll talk about this later when I show the picture if (completer! = null) { stream.setCompleter(completer); }}Copy the code

Load is the method used to load images. Different providers have different implementations. We will focus on the implementation of NetworkImage in Provier

// NetworkImage @override ImageStreamCompleter load(image_provider.NetworkImage key, image_provider.DecoderCallback decode) { // Ownership of this controller is handed off to [_loadAsync]; it is that // method's responsibility to close the controller's stream when the image // has been loaded or an error is thrown. final StreamController<ImageChunkEvent> chunkEvents = StreamController<ImageChunkEvent>(); / / MultiFrameImageStreamCompleter is frame parser, the default is used it is for this, so the default support GIF return MultiFrameImageStreamCompleter (codec: _loadAsync(key as NetworkImage, chunkEvents, decode), // load images asynchronously. Chunkevents. stream, // Callback for loading process scale: key.scale, debugLabel: key.url, informationCollector: () { return <DiagnosticsNode>[ DiagnosticsProperty<image_provider.ImageProvider>('Image provider', this), DiagnosticsProperty<image_provider.NetworkImage>('Image key', key), ]; }); }Copy the code

Then look at NetworkImage’s _loadAsync

// It is clear that Future< uI.codec > _loadAsync(NetworkImage Key, StreamController<ImageChunkEvent> chunkEvents) image_provider.DecoderCallback decode, ) async { try { assert(key == this); final Uri resolved = Uri.base.resolve(key.url); final HttpClientRequest request = await _httpClient.getUrl(resolved); headers? .forEach((String name, String value) { request.headers.add(name, value); }); final HttpClientResponse response = await request.close(); if (response.statusCode ! = HttpStatus. Ok) {/ / request fails, an error throw image_provider.Net workImageLoadException (statusCode: response. StatusCode, uri: resolved); } / / binary stream data callback final Uint8List bytes = await consolidateHttpClientResponseBytes (response, onBytesReceived: (int cumulative, int? total) { chunkEvents.add(ImageChunkEvent( cumulativeBytesLoaded: cumulative, expectedTotalBytes: total, )); }); if (bytes.lengthInBytes == 0) throw Exception('NetworkImage is an empty file: $resolved'); Return decode(bytes); } catch (e) { // Depending on where the exception was thrown, the image cache may not // have had a chance to track the key in the cache at all. // Schedule a microtask to give the cache a chance to add the key. scheduleMicrotask(() { PaintingBinding.instance! .imageCache! .evict(key); }); rethrow; } finally { chunkEvents.close(); }}Copy the code

Now that the first question is answered, how does an image data request get called back to ImageState and displayed in the interface?

Network picture data callback and display process

To look at callbacks and displays, we start with the build method of the destination ImageState

RenderImage is the widget that renders the image. RenderImage is the final render. Image, then _imageInfo? When is image assigned? Widget result = RawImage( image: _imageInfo? .image, debugImageLabel: _imageInfo? .debuglabel, width: widget.width, height: widget.height, scale: _imageInfo?. Scale?? 1.0, color: widget.color, colorBlendMode: widget.colorBlendMode, fit: widget.fit, alignment: widget.alignment, repeat: widget.repeat, centerSlice: widget.centerSlice, matchTextDirection: widget.matchTextDirection, invertColors: _invertColors, isAntiAlias: widget.isAntiAlias, filterQuality: widget.filterQuality, );Copy the code

Remember from part 1 that _updateSourceStream(newStream); Methods? In this method, a listener is set for ImageStrem

// Set listener _imagestream.addListener (_getListener()); // ImageStreamListener ImageStreamListener _getListener({bool recreateListener = false}) { if(_imageStreamListener == null || recreateListener) { _lastException = null; _lastStack = null; _imageStreamListener = ImageStreamListener(_handleImageFrame, widget.loadingBuilder == null ? Null: _handleImageChunk, // Image loading intertune onError: Widget.errorBuilder! = null // Image loading error intermodulation? (dynamic error, StackTrace stackTrace) { setState(() { _lastException = error; _lastStack = stackTrace; }); } : null, ); } return _imageStreamListener; }Copy the code

So let’s look at the _handleImageFrame of the image Estate

// Very simple, So setState, you can see it's assigned _imageInfo void _handleImageFrame(ImageInfo ImageInfo, bool synchronousCall) { setState(() { _imageInfo = imageInfo; _loadingProgress = null; _frameNumber = _frameNumber == null ? 0 : _frameNumber + 1; _wasSynchronouslyLoaded |= synchronousCall; }); }Copy the code

So when is this _imageStreamListener called back? Remember the first step loading process MultiFrameImageStreamCompleter last step?

/ / MultiFrameImageStreamCompleter is to support multiple frames GIF parser, is also a OneFrameImageStreamCompleter, But have no MultiFrameImageStreamCompleter ({required Future < UI. The Codec > Codec, the required double scale, String? debugLabel, Stream<ImageChunkEvent>? chunkEvents, InformationCollector? informationCollector, }) : assert(codec ! = null), _informationCollector = informationCollector, _scale = scale { this.debugLabel = debugLabel; Codec. Then <void>(_handleCodecReady, onError: (Dynamic error, StackTrace stack) {// Catch error and report}); // Listen for the callback if (chunkEvents! = null) { chunkEvents.listen(reportImageChunkEvent, onError: (dynamic error, StackTrace stack) { reportError( context: ErrorDescription('loading an image'), exception: error, stack: stack, informationCollector: informationCollector, silent: true, ); }); }}Copy the code

Here to answer the second question, GIF how each frame support, the key is the class MultiFrameImageStreamCompleter, then watch _handleCodecReady MultiFrameImageStreamCompleter

void _handleCodecReady(ui.Codec codec) { _codec = codec; assert(_codec ! = null); If (hasListeners) {/ / see how the function name, parsing the next frame and implement _decodeNextFrameAndSchedule (); }}Copy the code

MultiFrameImageStreamCompleter _decodeNextFrameAndSchedule ()

Future < void > _decodeNextFrameAndSchedule () is async {try {/ / get the next frame, this step in C processing _nextFrame = await _codec! .getNextFrame(); } catch (exception, stack) { reportError( context: ErrorDescription('resolving an image frame'), exception: exception, stack: stack, informationCollector: _informationCollector, silent: true, ); return; } // The number of frames is not equal to 1, indicating that the picture has multiple frames. .frameCount == 1) { // This is not an animated image, just return it and don't schedule more // frames. _emitFrame(ImageInfo(image: _nextFrame! .image, scale: _scale, debugLabel: debugLabel)); return; } // If there is only one frame, _scheduleAppFrame will eventually go to _emitFrame _scheduleAppFrame(); }Copy the code

Then look at MultiFrameImageStreamCompleter _emitFrame

SetImage void _emitFrame(ImageInfo ImageInfo) {setImage(ImageInfo); _framesEmitted += 1; }Copy the code

The ImageStreamCompleter setImage

@protected void setImage(ImageInfo image) { _currentImage = image; if (_listeners.isEmpty) return; // Make a copy to allow for concurrent modification. final List<ImageStreamListener> localListeners = List<ImageStreamListener>.from(_listeners); For (final ImageStreamListener listener in localListeners) {try {// Where onlisteners are registered Return to ImageStream addLister listener.onImage(image, false); } catch (exception, stack) { } } }Copy the code

ImageStream addLister

Void addListener(ImageStreamListener listener) {void addListener(ImageStreamListener listener) {void addListener(ImageStreamListener listener) { // So listener.onimage (image, false); // ImageStream's completer is created in ImageStream. This will eventually call back to the _imageStreamListener in ImageState if (_completer! = null) return _completer! .addListener(listener); _listeners ?? = <ImageStreamListener>[]; _listeners! .add(listener); }Copy the code

At this point, the picture is the display process has been analyzed, the second question has been answered.

Fill the picture memory cache source analysis

First of all, flutter memory cache only has memory cache by default, which means that if the flutter process restarts, the image will need to be reloaded.

The memory cache in 1.22 is divided into three main parts, which is an increase over 1.17

  • _pendingImages loading cache. What does this do? Assuming Widget1 loads image A and Widget2 loads image A at this point, the Widget uses the cache being loadedCopy the code
  • _cache The image cache that has been successfully loaded is understandableCopy the code
  • _liveImages Live image cache. The code basically adds a layer of caching to the CacheImage cache. After the CacheImage is cleared,Copy the code

    When loading an image for the first time, it will be in _pendingImages first. Note that the image has not been successfully loaded, so if there is reuse, the _pendingImages will be hit. When the image request is successful, a copy will be saved in _cache and _liveImages. The _pendingImages is removed. When the maximum number in the cache is exceeded, it is removed from the _cache according to LRU rules

How do I support disk caching of images

After looking at the entire process, you should have some ideas about disk caching. The first is to customize the ImageProvider, which writes the image data to the disk cache after the image data request is successful. However, for hybrid projects, it is better to replace the network request method of image, and use channel and native (Android, ios) image library to load the image. This allows you to reuse the disk cache of the native image library, but it also has the disadvantage of being less efficient because of multiple copies of memory and channel communication.

conclusion

This article only analyzes the loading and presentation of Image.Net Work, and only touches on the Dart side of the code. In general, the whole process is not complicated, other principles such as image. Memory and image. File are the same, the difference is that their ImageProvider is different, we can also customize the ImageProvider to achieve their desired effect.