Flutter uses Metal to drive the GPU on iOS

preface

Flutter1.17 is released

About the 1.17 release

Application performance and package size optimization

The main focus of this release is to continue overall optimization efforts in terms of performance and memory improvements. Simply upgrade your application to this version, and users will see faster animations, smaller applications, and lower memory usage. In this release, the default navigation case (opaque navigation bar) will be 20%-37% faster. For simple iOS animations, you can reduce CPU/GPU usage by up to 40%, depending on the hardware (detailed in PR 14104 and PR 13976).

This release also provides a larger application size improvement, the result of several bug fixes that resulted in larger packages. For example, Android’s FlutterGallery example, which was 9.6MB at the end of 2019, is now 8.1MB, a decrease of 18.5%.

In terms of memory usage, this version reduces the memory used to quickly scroll through large images by 70%, which also indirectly improves application performance, depending on the amount of memory on the device.

Metal support improves iOS performance by 50%

Apple’s support for Metal on iOS gives almost direct access to the underlying GPU and is apple’s recommended graphics API. On iOS devices that fully support Metal, Flutter is currently preferred by default, which makes your Flutter application run faster most of the time, increasing rendering speed by about 50% on average (depending on the amount of rendering effort).

For devices that do not fully support Metal (versions prior to the A7 processor or devices running iOS versions prior to 10), Flutter uses OpenGL as in the past to provide local rendering speed for older devices.

New Widgets: NavigationRail, DatePicker, etc

In this release, Flutter adds NavigationRail as a new widget that provides responsive application navigation. It was designed and developed by the Google Material Design team. The NavigationRail is excellent because it’s easy to switch the application BottomNavigato from another way to increase the application’s visible screen size.

For other components, please refer to the top link for further understanding;

Material Text Proportion: Modernizes the theme of the Flutter text

In this release, the Flutter team completed the implementation of the Type Scale portion of the 2018 Material Design specification without breaking existing Flutter applications. Opt-in support for new configurations was added in PR 22330 in October 2018, but not for new names. The existing text style name is not changed, because doing so is a major API change that could affect most applications. This version updates the TextThemeAPI to match the current Material specification, but keeps the old name so that your code doesn’t break. However, the old name is deprecated, so you will receive a warning when using it to remind you to adopt the new name when developing.

Google font is used for Flutter

Text and fonts go hand in hand, so if you’re interested in the new Material Text Scale implementation, you might also be interested in the new Flutter V1.0 Google fonts.

Google Fonts allows developers to easily try and use any font found in Fonts.google.com in their applications. When the application is ready for release, the developer decides whether the user will receive the font by downloading it from the API, or if it is pre-bundled with the application package.

Accessibility and internationalization

In this release, we’ve done all the work, including accessibility fixes for scrolling, text fonts, and other input widgets. You’ll see a full list of accessibility issues in this version of GitHub. We recommend that you test the accessibility of your application, and we have updated the documentation in this release with some recommended best practices.

Tools: Dart DevTools porting to Flutter, Android Quick Start, and more

This release enables us to replace the current version of Dart DevTools with the new Flutter version. If you want to try it out, start DevTools and click the “beaker” icon in the upper right corner of DevTools.

Devtools has many other interesting updates waiting to be discovered.

For details, please refer to the linked article for practice; Practice will lead you to a better introduction than any article;

Flutter and Metal in 1.17

Here is the key translation content of the article

The Flutter engine will migrate to use the Metal rendering API instead of OpenGL on iOS.

Why use Metal?

A Flutter user interface is displayed on the screen by accessing the platform’s GPU using the accelerated graphics API. This API is a version of OpenGL ES on iOS. With the introduction of iOS 12, OpenGL ES and related apis were officially rejected by Apple in 2018. Metal is now the recommended API for low level, low overhead rendering on iOS and Mac. While the cross-platform aspects of OpenGL are certainly useful, the Flutter team is also bullish on the Metal solution. The unpopularity of OpenGL ES also seems to have contributed to the poor maintenance of OpenGL drivers on iOS. There has been an uptick in ios-specific OpenGL issues, which are increasingly difficult to debug and unlikely to be fixed. For these reasons, migrating to Metal on iOS is imperative.

On what version does Flutter use Metal?

The Metal rendering of Flutter will work on apple A7 devices or devices running iOS10 or later. OpenGL is degraded when Metal is unavailable.

Why can’t Flutter use Metal on all iOS models?

Flutter currently runs on iOS 8 and later. This includes devices where Metal itself is not available. Specifically, Metal is only available on Apple A7 and later iOS devices. This would rule out devices like the iPhone 5C that can be upgraded to iOS 10 but only have an Apple A6 SOC. A policy requiring Metal to be used everywhere means we will have to drop some models. This is not an adaptation that Flutter is now prepared to consider.

Also problematic are the few devices that support iOS versions between 8 and 10 that have An Apple A7 SOC (or higher) but have not yet been updated to the latest iOS version. The Flutter is also technically capable of using Metal on these devices, but not yet. This is because the Metal API on earlier versions of iOS was not expressive enough about the features needed for Flutter, particularly the Skia rendering library used by Flutter. This requires creating solutions or alternatives for situations such as dual source mixing. In order to reduce the implementation and testing burden, iOS 10 was chosen as the benchmark. This decision may be amended if necessary. (This means that anything above A7 can theoretically be rendered using Metal)

Does this change the lowest version of iOS that Flutter supports?

Not at all. Flutter will use Metal on newer devices and iOS versions, but the OpenGL back end remains intact. Used when Metal is not available, is an alternative.

As a Flutter developer, what do I need to do to migrate to iOS using Metal?

No specific adaptation by the developer is generally required, except in cases where it is very unlikely that your application will make assumptions about Flutter under the engine using OpenGL. The updated Flutter engine does not require any changes to the Flutter application or the way it is embedded.

Is there an impact on binary size due to the addition of Metal support?

right The size of the uncompressed Flutter engine binary will increase by about 250KB. Incremental reduction of this size will continue, but to make a significant reduction, you need to remove the OpenGL back end. This is not something the Flutter team is currently considering.

Will camera preview, video playback, WebView, maps and other plugins continue to work?

right These plug-ins do not need to be updated and will continue to work as expected. All plug-ins in the Flutter Plugins repo have been tested to work with the Metal back end.

How can I tell if my application has modified Flutter using OpenGL under the engine?

The rendering API used by Flutter is an implementation detail of the Flutter engine. However, Flutter does provide the underlying platform with the ability to provide textures for inline synthesis in the Flutter hierarchy. Developers (or plug-ins) provide and update these textures by using a texture widget in their application and writing platform-specific code. Inline combinations of textures provided to the Flutter engine may fail if the platform-specific code for manipulating textures provided to the Flutter engine does not specify that textures can interoperate with the Metal API. For guidance on ensuring that textures interoperate with metals, follow the documentation on how to “Create Interoperable textures”. Specifically, describe the texture pixels in creating buffer, besides kcppixelbufferopenglcompatibilitykey key, also must specify kcppixelbuffermetalcompatibilitykey keys.

If the application does not use the texture widget, no action is required. If so, the textures provided to the Flutter must be checked to ensure that they interoperate with Metal.

This is a very unique use case and there is no plugin for Flutter Plugins repo that uses this mechanism.

If my application uses OpenGL to provide textures for inline composition in the Flutter application, does this apply to the Metal backend?

right As mentioned earlier, you can continue to update textures using OpenGL as long as the textures provided to the Flutter are interoperable with Metal, and the Flutter will compose the same textures using Metal.

How to determine whether the Flutter application is rendered using Metal instead of OpenGL at runtime?

The fluttViewControllers view will have a Camertalayer class. Flutter will use Metal or OpenGL. That is, there cannot be some Flutter application instances that use both Metal and OpenGL in one process.

Will Flutter use metal on iOS emulators?

No, not at the moment. Flutter will continue to use the software back end on the iOS emulator as it has always done.