In 2019, Flutter launched several official versions, supported more and more terminals, and used more and more projects. Flutter is undergoing a process of trial from a small scale to a large scale application. More and more R&D teams have joined in the study of Flutter. As one of the major Internet companies, JINGdong has also actively participated in the cross-end study of Flutter. This paper will introduce the application scheme of JD on Flutter and relevant optimization results.

Why consider the Flutter technology solution

In fact, JINGdong has been studying and practicing cross-end development solutions for a long time, using Hybrid App as the earliest technology solution, and gradually switching to RN technology stack since early 2015. At present, it should be one of the companies with the most extensive application of RN technology platform and relatively complete supporting facilities in the industry. Since mid-2018, we have also looked at Flutter technology. The most attractive features of Flutter are its high performance and compatibility. These two points are also the relative shortcomings of RN technology. High performance refers to rendering performance in complex scenes and interactions, and compatibility refers to consistency of layout and experience across different terminal platforms, which is especially important on the highly fragmented Android platform.

Jd’s practice in Flutter

As Google officially released a preview version of Flutter at the end of 2018, more and more R&D teams within JD.com sought to develop business with Flutter. We officially launched development and released the JDFlutter engine internally. On top of the official Flutter engine, we have made additional optimizations and feature extensions:

• The Flutter engineering retrofit: Optimizes the Flutter development environment and DART code management to seamlessly integrate into existing apps and support automated DART compilation and packaging for easy development and debugging.
• Routing and multi-page management: Centralized routing management is implemented for native and flutter pages, two-way parameter transfer and jump, and shared memory optimization is implemented.
• Extended UI Component library: The officially supported Material and Cupertino styles were not sufficient, so we implemented a custom component library in-house.
• Native capability expansion: the official native capability has been extended, encapsulating the opening of basic capabilities including network, login, buried point and so on, and providing 50+ native extension API.
• Dynamic support on The Android side: Dynamic support is implemented on the Android side, and hot update services can be online. Dynamic is not supported on iOS.

JDFlutter is currently developed by JD Mall, JD Video, JD Home, JD Logistics, 7Fresh and other apps.

JDFlutter frame design

The overall frame structure of JDFlutter consists of three parts: basic frame, components and tools, as shown in the figure:

​

Basic framework

The JDFlutter basic framework is divided into three layers, including JDFlutter basic layer, general business layer and business layer.

• Base layer: provides the basic component support of Flutter, including component management, state management, etc. The base layer is completely independent and has no business dependency.
• General business layer: provides support for general business components, such as login components, payment components, etc. The generic business layer depends on the base layer.
• Business layer: that is, the concrete business logic implementation layer, according to business needs to carry on the combination of different components, to achieve the rapid development of business pages.

Core components

• Component management: Components communicate with each other through standard protocol interfaces to reduce component coupling and facilitate maintenance and component upgrade.
• State management: separation of data and interface, unified state management, data changes to drive interface changes, more conducive to data persistence and preservation, but also conducive to the reuse of UI components;
• Hybrid Router: Mainly resolves the problem of cross-hop between the Flutter and Native to reduce memory overhead and share the same Flutter Engine.

Tool is introduced

• Build and publish: Optimize the build logic of Flutter, manage dependencies on Flutter native dependencies, package Flutter and native code, and automate build and publish.
• Resource management: Manage image resources and convert them into Flutter classes for easy reading of resources, similar to R class of Andorid.
• Template code generation: Reduce the code writing of Flutter and automatically generate the framework template code of Flutter components to improve code writing efficiency;
• JSON conversion: Convert JSON data into Flutter code and provide AN API for converting Flutter objects from JSON, reducing the need to write and parse Flutter code.

JDFlutter business development practice

JDFlutter provides a whole-process development solution for business R&D teams:

Mixed configuration engineering

There are two situations of Flutter and native hybrid development. First, students who develop Flutter business need to interact with native, so they need to have a hybrid compilation environment of Flutter and native. Second, students who use the native SDK to develop their business need to integrate and package with the Flutter business. At this time, they need to be transparent to the Flutter to reduce their dependence on the Flutter compilation environment. In addition, they only need to rely on the native compilation environment. Next, we’ll focus on configuring a hybrid build environment for Android and iOS.

Android Configuration

Create a Flutter Module

 flutter create -t module --org com.example my_flutter
Copy the code

Add the following configuration information to settings.gradle of the root project

// MyApp/settings.gradle
include ':app'                        // assumed existing content
setBinding(new Binding([gradle: this]))              // new
evaluate(new File(                                   // new
settingsDir.parentFile,                              // new
  'my_flutter/.android/include_flutter.groovy'       // new
)) 
Copy the code

Added flutter dependencies to native App modules

dependencies {
  implementation project(':flutter')}Copy the code

This allows native projects to compile together.

For details, please refer to the official documentation: github.com/flutter/flu…

This method can meet the requirements of the Flutter, but it is not very convenient. After the development of the project, we need to compile in the Android Studio project, which is quite troublesome. Therefore, we can also modify the Flutter project settings.gradle. Run a hybrid project containing native code directly in the Flutter development environment as follows

// MyApp/settings.gradle //projectName native module name //projectPath native projectPath include":$projectName"
project(":$projectName").projectDir = new File("$projectPath")
Copy the code

You can also run Futter Run to start the Flutter project. However, note that the gradle build environment is consistent with the native build environment of the Flutter. Inconsistency may result in a compilation error.

IOS Platform Configuration

Create a flutter module

flutter create -t module my_flutter
Copy the code

Go to the iOS project directory and initialize the POD environment (skip this step if the project already uses Cocoapods)

pod init
Copy the code

Edit the Podfile

# Add new code to Podfile
flutter_application_path = '/{flutter module directory}/my_flutter'
eval(File.read(File.join(flutter_application_path, '.ios'.'Flutter'.'podhelper.rb')), binding)
Copy the code

Install the pod

pod install
Copy the code

Open the project (***.xcWorkspace) to configure the Build Phase and add the build option for compiling the Dart code

Open the iOS project, select the Build Phases option for the project, click the + button in the upper left corner, select New Run Script Phase, and add the following shell Script to the input box:

"$FLUTTER_ROOT/packages/flutter_tools/bin/xcode_backend.sh" build
"$FLUTTER_ROOT/packages/flutter_tools/bin/xcode_backend.sh" embed
Copy the code

Build PUB private server warehouse

The components used in Flutter development are generally shared within companies to avoid duplication of development, while Flutter component sharing requires the use of a PUB repository. Since internal business components are not suitable for uploading to the pub official warehouse, a private server warehouse needs to be set up to address the need for all business R&D teams to share Flutter components.

Interested students can study the source of the official pub repository pub.dartlang.org/, which relies heavily on the Google Cloud environment, Can also be based on https://github.com/kahnsen/pub_server to build a simple version of the private servers warehouse, in order to meet the upload and download function, the pub agreement is relatively simple, we can increase protocol interface to realize more functions in source code.

Run pub_server

~ $ git clone https://github.com/dart-lang/pub_server.git
~ $ cd pub_server
~/pub_server $ pub get
...
~/pub_server $ dart example/example.dart -d /tmp/package-db
Listening on http://localhost:8080

To make the pub client use this repository configure your shell via:

    $ export PUB_HOSTED_URL=http://localhost:8080
Copy the code

Publishing a Flutter component requires modifying pubspec.yaml to add the following

Name: hello_plugin //plugin name description: A new Flutter plugin. XXX <[email protected]>// Author and mailbox homepage: https://localhost:8080 // Introduction page of the component publish_to: http://localhost:8080// Repository upload addressCopy the code

During the upload, you can use the following command to check for code errors and display the uploaded directory structure

pub publish --dry-run
Copy the code

If you have files that you do not want to upload, you can add a.gitignore file to the root directory to ignore the following

/build
Copy the code

Add the following information to the dependencies configuration of the Flutter component under Dependencies: in the project pubspec.yaml

Dependencies: hello_plugin: Hosted: Name: hello_plugin URL: http://localhost:8080 version: 0.0.2Copy the code

In this way, Flutter components can be shared within the company. If you do not want to build your own PUB repository, you can also use Git dependencies

Dependencies: hello_plugin: git: url: git://github.com/hello_plugin.git //Copy the code

Development and commissioning of Flutter business

This is not possible if the project is a hybrid project running in Android Studio or Xcode, but debugging can also be done:

Install the App to be debugged into your phone (install the DEBUG version), connect to your computer, and run the following command to synchronize the Flutter code to the device’s host App

$ cd flutterProjectPath/
$ flutter attach
Copy the code

After the command is executed, the system waits for the device connection status, opens the host App, and enters the Flutter page. If the following information is displayed, the synchronization is successful

zbdeMacBook-Pro:example zb$ flutter attach
Waiting for a connection from Flutter on MI 5X...
Done.
Syncing files to device MI 5X...                             1.2s

🔥  To hot reload changes while running, press "r". To hot restart (and rebuild state), press "R".
An Observatory debugger and profiler on MI 5X is available at: http://127.0.0.1:54422/
For a more detailed help message, press "h". To detach, press "d"; to quit, press "q".

Copy the code

Open http://127.0.0.1:54422 to view debugging information. If there are code changes, press R to synchronize the page in real time. If the changes do not take effect in real time, press R to restart the Flutter application.

JDFlutter thermal update practice

Most cross-end frameworks, such as React Native, Weex, and H5, can be hot-patched and live at any time to fix unexpected online problems. The architecture is flexible. This flexibility is expected to be difficult to achieve due to the AOT design of the Flutter, but it is still technically feasible. As we mentioned in our previous Flutter introduction article, the Design of the Flutter is designed to support thermal repair, but only on Android. The latest official architecture has support for hotfix architecture, you can update to version 1.2.1 to check, but the official features are still weak, can not achieve version control and rollback flexibility, so JDFlutter is not adopted.

Let’s start with a look at how Google’s official hotfix is designed:

Flutter1.2.1 introduced Dynamic Patch

Package structure of Flutter App

1. icudtl.dat
2. isolate_snapshot_data
3. isolate_snapshot_instr

The initialization process of a Flutter package

How does this code load when the Flutter page starts? That is about to speak of from the initialization of Flutter, in front of the page to start the need to call FlutterMain. StartInitialization to do initialization:

As you can see, this initialization is required on the main thread, and the following three main things are done:

• Some environment data is configured, such as the path of each core package, which is mainly provided to other modules for global calls

  

• Check the integrity of a Flutter package under asset. If the core package is missing, an exception will be thrown. During the development process, we often failed to package some files due to configuration, and then crash was triggered here. The specific code is as follows:

  

• Unzip some assets into the data partition. Here are some snippets of code. It can also be read by assetManager under asset. Because frequent use of assetManager to read asset is likely to cause multithreading congestion. Once blocked, the whole Flutter business cannot be rendered. Therefore, decompress some core resource libraries. Instead of unextracting all the resources (like images)

  

  

Start task to decompress the asset library into the app_flutter directory corresponding to app data in the data partition. The folder structure is as follows:

The res_timestamp file is used to mark some time stamps. The algorithm is relatively fixed and generated according to the installation time of the client and the version code of the app. That is to say, this value is fixed when the user opens the Flutter page. Delete the existing package of App_FLUTTER and unzip it again

Operation principle

The above is the analysis of the loading of the Flutter program. The final display of the Flutter page needs to be presented in the Native component of the Flutter View, which will bind to the underlying Flutter Native View and eventually run the Dart code of the data partition described above to render the UI. If a Flutter Activity is used, the Flutter View is displayed in full screen by default. If you need to customize the page, you need to design your own Activity

Thermal repair experiment

With this in mind, a hot fix is already in sight. Replace the decompression of the app_flutter package, kill the process, and then reload the Flutter page. Here’s a simple experiment:

Use adb to push some modified and compiled DART code into the app_flutter directory:

• Open the Flutter page first. By default, the package under Asset is loaded and unzipped to the data partition

• Modify a Flutter project and compile the code, and finally in the project directory my_flutter/android/Flutter/build/intermediates/Flutter/release seen in packaging generated file

  

• In this way, only the Flutter_assets directory and isolate_SNAPshot_data file in the file directory contain business codes and pictures, while the other parts are basically unchanged. Therefore, these two directories will be replaced here. You can experiment by using the ADB push command to push resource files to the corresponding data partition

The adb push my_flutter/android/Flutter/build/intermediates/Flutter/release/isolate_snapshot_data/data/data/app package name /app_flutterCopy the code

• Close the Flutter page, kill the process in Task, return to the Flutter page and open it again to see the effect of the changes. Image resources are stored in the Flutter_asset directory. You can also update images by placing them in this directory

The above experiment verifies that the scheme is basically feasible, but it is only a simple replacement, and there are still many problems in the actual use of replacement. So how did Google design it?

Google Hotfix design

Hot repair procedure

In the Flutter SDK 1.2.1, Google provides a ResourceUpdater for package inspection and download decompression. The upgrade procedure is as follows:

• At page initialization, check the fixed download update directory for business upgrade packages. From the code, you must turn this feature on in the manifest and set DynamicPatching

  

  Logically, the upgrade package will only be downloaded when the onResume or App is restarted. The overall download is completed by HTTP request. You can refer to the implementation section of DownloadTask in ResourceUpdater, which will not be detailed here.

• Every time init triggers a check for the app_flutter package in the data partition. If the package does not exist, it will be extracted from the aASet directory. The upgrade package replacement is done in this step. If not, extract it from the asset directory.

  

• Of course in the check to upgrade packages, do check for updates some configuration, mainly manifest. Json file, it will contain buildNumber/baselineChecksum fields, CRC32 verification is also performed on files such as “isolate_SNAPshot_data “, “isolate_snapshot_instr”, and” Flutter_assets /isolate_snapshot_data”

  

• The updated version timestamp is determined by reading patchNumber from the configured manifest.json file and the file download time, and will be generated again after the file is overwritten.

  The path of the upgrade package is as follows

  

Configuring the Server

Part of the article introduces how to open the function of upgrading patch. Since upgrading involves the server, how does Google associate the patch with the server? In fact, the principle is relatively simple. We need to configure the meta attribute of the manifest file of the client, add PatchServerURL, which is the address of our service, and download mode PatchDownloadMode and load mode PatchInstallMode. The default is ON_NEXT_RESTART (next initialization)

The whole process

Existing defects

• Too customized, all in the engine to complete, it is difficult to adapt to some special needs customization;
• Now more mainstream upgrade process, such as grayscale and whitelist functions are not supported;
• The dimension of the version number cannot be controlled, and the version rollback cannot be performed.

How does JDFlutter achieve thermal repair

Realize the principle of

The overall implementation principle of JDFlutter is actually the same as That of Google. At present, this solution is the simplest without modifying the engine. However, we do not use the upgrade architecture of Google. The advantage is that the overall compatibility is stronger and more flexible.

1. The server supports the whitelist and grayscale upgrade package delivery based on the unique identifier of the client.
2. Optimize the download and replace process. The upgrade package of Flutter is usually 4-5m and is obtained from the network. The failure rate is high. The replacement process of Flutter also involves file operation. After a JDFlutter client downloads the package, it does not replace the file directly. Instead, it changes the file name and decompress it to the app_FLUTTER directory, and then changes the file to the standard Flutter name when the business page is reopened or reinitialized. There are no performance issues with this operation and the old version of the file is backed up so that the code can be rolled back;
3. There are many concurrent Flutter pages running at the same time. It is necessary to avoid the situation that some intermediate states may occur due to the upgrade, making services or pages unavailable.
4. The rollback code can be controlled if the upgrade fails or there is a problem with the service package after downloading.
5. If a large number of online exceptions occur, you can specify a degradation policy for the CORRESPONDING Flutter service to quickly degrade the service to the H5 page.

Thermal repair planning

In the future, JDFlutter will continue to explore and validate thermal remediation to meet the rapid development needs of JD’s business. For the current scheme, we considered the following optimization points:

• Upgrade of Flutter service packet difference: The existing upgrade mode is full package coverage, and the upgrade package is still large even after compression, which affects the success rate of upgrade and user traffic. Some DIff tools will be used later to compare and generate differential patches, deliver them through the server, and merge them into a complete package on the client. However, the final version will be fragmented after too many upgrades. It is difficult to maintain the relationship before the version.
• Update the page in a timely manner after the upgrade: Current solutions (including the standard Google upgrade solution) cannot update the page after downloading or replacing the service package. You need to restart the process to refresh the page. In the future we will optimize the engine to refresh pages at any time by releasing underlying resources and reloading them.

future

Google Flutter is an excellent cross-development technology that has come a long way. Community ecology and framework maturity are also fast catching up to RN. We believe that Flutter+RN will become the pinnacle of cross-platform development in the near future.

Team to introduce

Jingdong ARES Cross-end team, as the multi-end technology platform team of JINGdong Technology and Data Center, focuses on cross-end development technology framework and platform construction, including but not limited to RN, Flutter, small program and other technology stacks. At present, it has been widely used in jingdong core apps such as JINGdong Mall, JINGdong Finance, Jingdong Home, jingdong Shopping, etc., to help business teams develop their own businesses at low cost and fast, so as to cope with the rapidly changing market.