Loading of Flutter picture

Painted painted levels: fostered fostered fostered

Tags: “Flutter” “Image” author: Dadato Brief Correction: QiShare team

---

* preface:

Reading this article you will get: 1. How does Flutter image load? 2, Flutter image load source code implementation process? 3. What are the optimization points of Flutter image loading? *

Flutter Image

In Flutter, an Image is the widget that displays the Image and is used to retrieve the Image from the ImageProvider. Image Supports JPEG, WebP, GIF, Animated WebP/GIF, PNG, BMP, and WBMP.

Image structure is as follows:

You can see multiple loading modes at the top of the image.

Flutter image loading method

1.Image.asset

Images obtained from AssetBundle using key;

The two methods are as follows:

Image(height: 100, width: 100, image: AssetImage(happy.png), )
Copy the code

Image.asset( happy.png, width: 100, height: 100.)Copy the code

To do this, of course, you need to configure the image path in the pubspec.yaml file.

2,Image.network

Get images from web urls;

Image.network('https://p0.ssl.qhimg.com/t0183421f63f84fccaf.gif',fit: BoxFit.fill);
Copy the code

3,Image.file

Get images from local files

Image.file(File('/sdcard/happy.png')),
Copy the code

4,Image.memory

Used to fetch images from the Uint8List;

new Image.memory(Uint8List bytes),
Copy the code

Bytes refers to image data in memory, which is converted to image objects.

Comparison of Unit8List with other language data structures in Flutter:

flutter	java	swift	C
Uint8List	byte[]	FlutterStandardTypedData	char[]

Other related common ways to load images

5, CacheNetworkImage

A cached network image, belonging to the cached_network_image library

new CachedNetworkImage(
    fit:BoxFit.fill,
    width:200,
    height:100,
    imageUrl:'https://p0.ssl.qhimg.com/t0183421f63f84fccaf.gif',
    placeholder:(context, url) => new ProgressView(),
    errorWidget:(context, url, error) => new Icon(Icons.error),
);
Copy the code

6, FadeInImage. MemoryNetwork

Default placeholder map and fade effect

import 'package:transparent_image/transparent_image.dart';

FadeInImage.memoryNetwork(
    placeholder: kTransparentImage, //kTransparentImage belongs to the transparent_image library
    image: 'https://p0.ssl.qhimg.com/t0183421f63f84fccaf.gif',);Copy the code

7, IconIcons refer to the URL

new Icon(Icons.android,size: 200,);
Copy the code

The resolution of the Flutter loading images

The Flutter loads images at the appropriate resolution for the current device. Specify different pixel devices like proportional images can be assigned to the asset folder like this:

• … icon/happy.png
• … / 2.0 x/happy. PNG
• … / 3.0 x/happy. PNG

The main resource corresponds to a 1.0 x resolution image by default; On devices with a device pixel ratio of 1.8… / 2.0 x/happy. PNG; For devices with a pixel ratio of 2.7, choose… / 3.0 x/happy. PNG.

Each item in the asset declaration in pubspec.yaml is identified as corresponding to the actual file. When the main resource is missing, however, loads are sought in order of resolution from lowest to highest. The loading scheme here can be compared with the image loading logic in the Android system.

Flutter packaging applications, resources will be in accordance with the key – in the form of value in the apk assets/flutter_assets/AssetManifest json file, load resources parsing json file first, choose the most suitable, according to the pictures to load Assetmanifest.json assetmanifest.json assetmanifest.json

{
    "assets/happy.png": ["Assets / 2.0 x/happy. PNG"."Assets / 3.0 x/happy. PNG"]}Copy the code

Android

On Android, you can get the asset from AssetManager and find the openFd based on the key.

The key is obtained by pluginregistry.registrar lookupKeyForAsset and FlutterView getLookupKeyForAsset.

Pluginregistry.registrar is used to develop the plug-in, while FlutterView is used to develop the view for the platform app.

pubspec.yaml

flutter:
  assets:
    - icons/happy.png
Copy the code

Java plugin code

AssetManager assetManager = registrar.context().getAssets();
String key = registrar.lookupKeyForAsset("icons/happy.png");
AssetFileDescriptor fd = assetManager.openFd(key);
Copy the code

iOS

IOS development uses MainBundle to get assets.

Method using FlutterPluginRegistrar lookupKeyForAsset and lookupKeyForAsset: fromPackage: access to the file path; FlutterViewController lookupKeyForAsset and lookupKeyForAsset: fromPackage: method to get the file path;

FlutterPluginRegistrar is then used to develop the plug-in and FlutterViewController is used to develop the view of the platform app.

Objective-C plugin

NSString* key = [registrar lookupKeyForAsset:@"icons/happy.png"];
NSString* path = [[NSBundle mainBundle] pathForResource:key ofType:nil];
Copy the code

Of course pubspec.yaml configuration is consistent.

Source code analysis

There are four ways to load images. Let’s take a look at how the framework layer loads images. We take Image.network as an example to follow up the relevant source code implementation.

Image.net Work’s method is as follows:

Image.network(
   String src, {
   Key key,
   double scale = 1.0.this.frameBuilder,
   this.loadingBuilder,
   this.semanticLabel,
   this.excludeFromSemantics = false.this.width,
   this.height,
   this.color,
   this.colorBlendMode,
   this.fit,
   this.alignment = Alignment.center,
   this.repeat = ImageRepeat.noRepeat,
   this.centerSlice,
   this.matchTextDirection = false.this.gaplessPlayback = false.this.filterQuality = FilterQuality.low,
   Map<String.String> headers,
 }) : image = NetworkImage(src, scale: scale, headers: headers),
      assert(alignment ! =null),
      assert(repeat ! =null),
      assert(matchTextDirection ! =null),
      super(key: key);
Copy the code

This method creates an image widget that displays the ImageStream from the network. The image that loads the NetworkImage is created by NetworkImage. SRC, scale, headers cannot be empty. Other parameters are not required. NetworkImage inherits from ImageProvider, so image is ImageProvider. ImageProvider is an abstract class. Its implementation classes include NetworkImage, FileImage, ExactAssetImage, AssetImage, MemoryImage, and AssetBundleImageProvider.

Image source code is as follows:

class Image extends StatefulWidget {
/// Image for display
  finalImageProvider image; .@override
  _ImageState createState() => _ImageState();
}
Copy the code

_ImageState class

class _ImageState extends State<Image> with WidgetsBindingObserver { ImageStream _imageStream; ImageInfo _imageInfo; .@override
void initState() {
    super.initState();
    WidgetsBinding.instance.addObserver(this);
}
  
@override
void didChangeDependencies() {
    _updateInvertColors();
    _resolveImage();// Parse the image from here
    // Set and remove callbacks that listen for picture changes
    if (TickerMode.of(context))
      _listenToStream();
    else
      _stopListeningToStream();
    super.didChangeDependencies();
}
  
void _resolveImage() {
    // Call ImageProvider resolve according to ImageConfiguration to obtain the ImageStream object
    finalImageStream newStream = widget.image.resolve(createLocalImageConfiguration( context, size: widget.width ! =null&& widget.height ! =null ? Size(widget.width, widget.height) : null)); _updateSourceStream(newStream); }... }Copy the code

Its lifecycle method methods include initState(), didChangeDependencies(), build(), deactivate(), Dispose (), didUpdateWidget(), and more. When it is inserted into the render tree, the initState() function is called, followed by didChangeDependencies(). You can see in the code that the method _resolveImage() is called, which creates newStream, a new object for ImageStream. Widget. image is the ImageProvider, and the resolve method is called as follows:

ImageStream resolve(ImageConfiguration configuration) {
  final ImageStream stream = ImageStream();
  T obtainedKey;
  bool didError = false;
  Future<void> handleError(dynamic exception, StackTrace stack) async {
    if (didError) {
      return;
    }
    didError = true;
    await null; // Wait for event polling in case listeners are added to the image stream.
  
    final_ErrorImageCompleter imageCompleter = _ErrorImageCompleter(); stream.setCompleter(imageCompleter); . }... Future<T> key;try {
        key = obtainKey(configuration);
      } catch (error, stackTrace) {
        return;
      }
      key.then<void>((T key) {
        obtainedKey = key;
        final ImageStreamCompleter completer = 
        PaintingBinding.instance.imageCache.putIfAbsent(key, () => load(key), onError: handleError);
        if(completer ! =null) {
          stream.setCompleter(completer);
        }
      }).catchError(handleError);
    
return stream;
Copy the code

ImageStreamCompleter Is the base class used to manage the classes that dart: UI loads. The object of ImageStreams is rarely constructed directly, but is automatically configured by the ImageStreamCompleter. The image manager ImageStreamCompleter in ImageStream is created by method. ImageCache is a singleton implemented in the Flutter framework for image caching that is created when the Dart VM is loaded. ImageCache can cache up to 1000 images and 100MB of memory space. The maximumSize can be adjusted using [maximumSize] and [maximumSizeBytes].

PaintingBinding.instance.imageCache.putIfAbsent(key, () => load(key), onError: handleError);
Copy the code

You can see two key methods in the source code: putIfAbsent and Load.

putIfAbsent

ImageStreamCompleter putIfAbsent(Objectkey, ImageStreamCompleter loader(), {ImageErrorListener onError }) { ImageStreamCompleter result = _pendingImages[key]? .completer;// Since the image has not been loaded, nothing needs to be done.
    if(result ! =null)
      return result;
    // Remove the corresponding ImageProvider by Key from the cache list so that it can be moved to the most recently used location below.
    final _CachedImage image = _cache.remove(key);
    if(image ! =null) {
      _cache[key] = image;
      return image.completer;
    }
    try {
      result = loader();
    } catch (error, stackTrace) {
      ......
    }
    void listener(ImageInfo info, bool syncCall) {
      // Images that cannot be loaded do not occupy the cache size.
      final intimageSize = info? .image ==null ? 0 : info.image.height * info.image.width * 4;
      final _CachedImage image = _CachedImage(result, imageSize);
      // If the image is larger than the maximum cache size and the cache size is not zero, increase the cache size to the image size plus 1000.
      // Point to ponder: when does constant addition cause a crash?
      if (maximumSizeBytes > 0 && imageSize > maximumSizeBytes) {
        _maximumSizeBytes = imageSize + 1000;
      }
      _currentSizeBytes += imageSize;
      final _PendingImage pendingImage = _pendingImages.remove(key);
      if(pendingImage ! =null) {
        pendingImage.removeListener();
      }
      _cache[key] = image;
      _checkCacheSize();
    }
    if (maximumSize > 0 && maximumSizeBytes > 0) {
      final ImageStreamListener streamListener = ImageStreamListener(listener);
      _pendingImages[key] = _PendingImage(result, streamListener);
      / / remove [_PendingImage removeListener] on listening in
      result.addListener(streamListener);
    }
    return result;
  }
Copy the code

load

/// Pull the image of a network image_provider.NetworkImage Implementation.
class NetworkImage extends image_provider.ImageProvider<image_provider.NetworkImage> implements image_provider.NetworkImage {...@override
  ImageStreamCompleter load(image_provider.NetworkImage key) {
     
    final StreamController<ImageChunkEvent> chunkEvents = StreamController<ImageChunkEvent>();
     
    return MultiFrameImageStreamCompleter(
        
      codec: _loadAsync(key, chunkEvents),
        
      chunkEvents: chunkEvents.stream,
      scale: key.scale,
      informationCollector: () {
        return <DiagnosticsNode>[
          DiagnosticsProperty<image_provider.ImageProvider>('Image provider'.this),
          DiagnosticsProperty<image_provider.NetworkImage>('Image key', key), ]; }); }Copy the code

loadAsync

Future<ui.Codec> _loadAsync(
    NetworkImage key,
    StreamController<ImageChunkEvent> chunkEvents,
  ) async {
    try {
      final Uri resolved = Uri.base.resolve(key.url);
      final HttpClientRequest request = await_httpClient.getUrl(resolved); headers? .forEach((String name, String value) {
        request.headers.add(name, value);
      });
      final HttpClientResponse response = await request.close();
      if(response.statusCode ! = HttpStatus.ok)throw image_provider.NetworkImageLoadException(statusCode: response.statusCode, uri: resolved);
        // Convert the response information returned by the network into a Uint8List bytes in memory. There is the logic to unzip gzip.
      final Uint8List bytes = await consolidateHttpClientResponseBytes(
        response,
        onBytesReceived: (int cumulative, inttotal) { chunkEvents.add(ImageChunkEvent( cumulativeBytesLoaded: cumulative, expectedTotalBytes: total, )); });if (bytes.lengthInBytes == 0)
        throw Exception('NetworkImage is an empty file: $resolved');
        
      return PaintingBinding.instance.instantiateImageCodec(bytes);
    } finally{ chunkEvents.close(); }}Copy the code

Convert the response information returned by the network into the Uint8List bytes in memory, and finally return an instantiated image Codec object, Codec, Dart file _instantiateImageCodec calls native methods to handle it.

MultiFrameImageStreamCompleter

This object is an implementation of ImageStreamCompleter, which manages the multi-frame picture stream and manages the decoding and scheduling of image frames.

This class handles two types of frames:

• Image frame: The image frame of an animated image.

• App Frame: A frame drawn to the screen by the Flutter engine to be displayed on the application GUI.

It won’t put all of the code, in image_stream. Dart files visible class MultiFrameImageStreamCompleter.

MultiFrameImageStreamCompleter({
    @required Future<ui.Codec> codec,
    @required double scale,
    Stream<ImageChunkEvent> chunkEvents,
    InformationCollector informationCollector,
  }) : assert(codec ! =null),
       _informationCollector = informationCollector,
       _scale = scale {
    codec.then<void>(_handleCodecReady, onError: (dynamic error, StackTrace stack) {
     ..........
    });
Copy the code

_handleCodecReady

Here the codec asynchronous callback submethod

void _handleCodecReady(ui.Codec codec) {
    _codec = codec;
    if(hasListeners) { _decodeNextFrameAndSchedule(); }}Copy the code

_decodeNextFrameAndSchedule

Codec decodes the number of frames of the image and determines if the image has only one frame, which is PNG, JPG and other static images.

Future<void> _decodeNextFrameAndSchedule() async {
    try {
      _nextFrame = await _codec.getNextFrame();
    } catch (exception, stack) {
      ........
      return;
    }
    if (_codec.frameCount == 1) { // Only one frame is available.
      _emitFrame(ImageInfo(image: _nextFrame.image, scale: _scale));
      return;
    }
    _scheduleAppFrame();
  }

  void _scheduleAppFrame() {
    if (_frameCallbackScheduled) {
      return;
    }
    _frameCallbackScheduled = true;
    SchedulerBinding.instance.scheduleFrameCallback(_handleAppFrame);
  }
Copy the code

_emitFrame(ImageInfo(image: _nextFrame.image, scale: _scale));

void _emitFrame(ImageInfo imageInfo) {
    setImage(imageInfo);
    _framesEmitted += 1;
  }
  

  @protected
  void setImage(ImageInfo image) {
    _currentImage = image;
    if (_listeners.isEmpty)
      return;
    // Make a copy to allow concurrent modification.
final List<ImageStreamListener> localListeners = List<ImageStreamListener>.from(_listeners);
      
    for (ImageStreamListener listener in localListeners) {
      try {
        listener.onImage(image, false);
      } catch(exception, stack) { .......... }}}Copy the code

The core logic of setImage is to notify all registered listeners that the image has changed and can be updated. At this point we go back to the _listenToStream method called by the didChangeDependencies method in the _ImageState class that we started with, and finally call the _handleImageFrame method, Change the image information _imageInfo and image frame number change _frameNumber, and finally execute setState(() {}) to refresh the UI.

void _listenToStream() {
    if (_isListeningToStream)
      return;
    _imageStream.addListener(_getListener());
    _isListeningToStream = true; } ImageStreamListener _getListener([ImageLoadingBuilder loadingBuilder]) { loadingBuilder ?? = widget.loadingBuilder;return ImageStreamListener(
      _handleImageFrame,
      onChunk: loadingBuilder == null ? null : _handleImageChunk,
    );
  }

  void _handleImageFrame(ImageInfo imageInfo, bool synchronousCall) {
    setState(() {
      _imageInfo = imageInfo;
      _loadingProgress = null;
      _frameNumber = _frameNumber == null ? 0 : _frameNumber + 1;
      _wasSynchronouslyLoaded |= synchronousCall;
    });
  }
Copy the code

This completes the loading of a web image.

Here should be a flow chart more concise and clear expression.

conclusion

Image loading display framework provides a variety of ways, we carried on the analysis of the image network loading. From the source point of view of the network picture loading process has a general understanding. The points found can be optimized. Here are the optimized points:

1, see the network image is only in the ImageCache management class memory cache, when the application process is restarted or to download the image again, here can be optimized, such as save to the local disk external storage.

2, get the picture to load into the memory, whether there is a compression of the picture, this processing is best to adapt to the current platform without too much change in the picture definition.

Looking forward to the iterative optimization points in the next chapter.

---

Recommended articles:

Dispatch_semaphore_t Swift 5.1 (11) – Method Swift 5.1 (10) – properties iOS App background keep alive in Swift CGAffineTransform iOS performance monitoring (1) CPU power monitoring iOS performance monitoring (2) Main thread lag monitoring iOS performance monitoring (3) Method Time monitoring