Author: Idle fish technology – Dust rustle
The introduction
During the long hybrid engineering period of the transition from Native to Flutter, it is a good choice to use Native’s more sophisticated controls in Flutter for a smooth transition. This paper hopes to introduce the way of using AndroidView and the solution of dual-end embedded Native component based on it.
1. Use tutorials
1.1. DemoRun
The idea of embedding maps may exist in many apps, but the current map SDKS do not provide libraries for Flutter, and it is not practical to develop a map of your own. In this scenario, using the hybrid stack format is a good choice. We can directly embed a Map in the Native drawing tree. However, the View embedded in this solution is not in the drawing tree of Flutter, which is a violent and inelegant way and requires a lot of effort to use.
An elegant solution is to use the AndroidView control that Flutter provides. Here is a simple demo embedded in Autonavi map, let us follow this application scenario, look at the use of AndroidView and the implementation principle.
1.2. AndroidView usage
AndroidView works in a similar way to MethodChannel, which is relatively simple and consists of three steps:
Step 1: Use AndroidView at the appropriate location in the Dart code, passing in a viewType String that uniquely identifies the Widget and associates it with the Native View.
The second step: Add code to the native side and write a PlatformViewFactory. The main job of the PlatformViewFactory is to create a View in the create() method and pass it to Flutter. This will be explained later)
Step 3: Use the registerViewFactory() method to register the PlatformViewFactory that you just wrote. This method takes two arguments. The first argument should correspond to the viewType that was previously written to the Flutter side. The second argument is the PlatformViewFactory that you just wrote.
The configuration of autonavi map part here is omitted, the official has more detailed documents, you can go to Autonavi developer platform for reference.
The above is all the operations using AndroidView, the overall look is relatively simple, but really to use, there are still two problems can not be ignored:
- Who decides the final display size of a View?
- How are touch events handled?
Let the small idle fish to give you one answer.
2. Principle explanation
In order to solve the above two problems, we must first understand the nature of the so-called “pass View”.
2.1. What is the essence of “passing View”?
To solve this problem, there is a natural need to read the source code, from a deeper level to see the whole process of this transfer, you can sort out a flow chart like this:
We can see that Flutter ends up with a textureId returned by the Native layer. According to Native’s knowledge, ky H textureId is the ID corresponding to the drawing data of the view rendered on the Native side. With this ID, the corresponding drawing data can be directly found and used in the GPU. Then how does Flutter make use of this ID?
In the previous in-depth understanding of Flutter interface development, we also introduced the drawing process of Flutter. And I’m going to give you a little bit of a rearrangement here
The Framework layer of Flutter will finally submit a layerTree to the Engine layer. The pipeline will traverse each leaf node of the layerTree. Each leaf node will eventually call the Skia Engine to complete the drawing of interface elements. Call glPresentRenderBuffer (IOS) or glSwapBuffer(Android) to complete the on-screen operation.
There are many types of Layer, and AndroidView uses one of them, TextureLayer. TextureLayer is described in more detail in the previous Texture surrounding Flutter, which will not be covered here. When a TextureLayer is iterated, it calls an Engine method, SceneBuilder::addTexture(), passing in textureId as an argument. Finally, when drawing, Skia will directly find the corresponding drawing data in GPU according to textureId and draw it on the screen.
So can anyone who gets this ID do this? The answer is of course no, Texture data is stored in the thread that created its EGLContext, so it won’t be available to you if you work on another thread. A few concepts need to be introduced here:
- Display object: Provides reasonable information about the pixel density and size of the Display
- Presentation: It provides Android with the ability to draw on the corresponding Context and Display objects, usually used for dual-screen Display.
Instead of explaining Presentation, we need to understand that Flutter implements external textures through Presentation. When creating the Presentation, we pass in the corresponding Context of FlutterView and create a virtual display object. Enable Flutter to find texture data created by Native using ID directly.
2.2. Who decides the final display size of View?
As you can imagine from the above process, the display size seems to be determined by two parts: the size of the AndroidView and the size of the AndroidView. So who actually decides, let’s do an experiment?
Create a New Flutter project and change the middle to AndroidView.
//Flutter
class _MyHomePageState extends State<MyHomePage> {
double size = 200.0;
void _changeSize() {
setState(() {
size = 100.0;
});
}
@override
Widget build(BuildContext context) {
return new Scaffold(
appBar: new AppBar(
title: new Text(widget.title),
),
body: Container(
color: Color(0xff0000ff),
child: SizedBox(
width: size,
height: size,
child: AndroidView(
viewType: 'testView',
),
),
),
floatingActionButton: new FloatingActionButton(
onPressed: _changeSize,
child: newIcon(Icons.add), ), ); }}Copy the codeAdd the corresponding code on the Android side. To better see the crop effect, use ImageView.
//Android
@Override
public PlatformView create(final Context context, int i, Object o) {
final ImageView imageView = new ImageView(context);
imageView.setLayoutParams(new ViewGroup.LayoutParams(500.500));
imageView.setBackground(context.getResources().getDrawable(R.drawable.idle_fish));
return new PlatformView() {
@Override
public View getView(a) {
return imageView;
}
@Override
public void dispose(a) {}}; }Copy the code
The final size of a RenderObject can be determined in two ways. One is specified by the parent node. Another option is to size the parent node within the range specified by the parent node. Open the corresponding source code, you can see that there is a very important property sizedByParent = true, that is to say, the size of the AndroidView is determined by its parent node, we can use containers, SizedBox and other controls to control the size of the AndroidView.
AndroidView’s drawing data is provided by the Native layer, so what happens when the actual pixel size of the View rendered in Native is larger than the size of the AndroidView? Usually, the way to deal with this situation is nothing more than two choices, one is cutting, the other is scaling. Flutter keeps its usual practice, all the widgets that are out of the bounds are displayed in a cropped manner, and the situation described above is treated as out of the bounds.
When the actual pixel size of the View is smaller than that of the AndroidView, you will notice that the View does not get smaller (the background color of the Container is not visible) and empty areas will be filled with white. This is the reason for SingleViewPresentation: : onCreate, will use a FrameLayout as rootView.
2.3. How are touch events transmitted
Android should be familiar with event flow, top-down passing, bottom-up processing or backflow. Flutter also uses this rule, but AndroidView handles gestures through two classes:
MotionEventsDispatcher: is responsible for encapsulating events into Native events and transmitting them to Native;
AndroidViewGestureRecognizer: responsible for identify the corresponding gestures, which has two attributes:
CachedEvents and forwardedPointers are distributed only if the PointerEvent pointer property is within the forwardedPointers property, otherwise they are in cacheEvents. The implementation here is designed to resolve conflicts in events, such as sliding events, which can be handled through gestureRecognizers (see the official notes here).
/// For example, with the following setup vertical drags will not be dispatched to the Android view as the vertical drag gesture is claimed by the parent [GestureDetector].
///
/// GestureDetector(
/// onVerticalDragStart: (DragStartDetails d) {},
/// child: AndroidView(
/// viewType: 'webview',
/// gestureRecognizers: <OneSequenceGestureRecognizer>[].
/ / /),
/ / /)
///
/// To get the [AndroidView] to claim the vertical drag gestures we can pass a vertical drag gesture recognizer in [gestureRecognizers] e.g:
///
/// GestureDetector(
/// onVerticalDragStart: (DragStartDetails d) {},
/// child: SizedBox(
/ / / width: 200.0.
/ / / height: 100.0.
/// child: AndroidView(
/// viewType: 'webview',
/// gestureRecognizers: <OneSequenceGestureRecognizer>[ new VerticalDragGestureRecognizer() ],
/ / /),
/ / /),
/ / /)
Copy the codeSo to sum up, this part of the process can be summed up very simply: The initial stage of events from Native to Flutter is beyond the scope of this paper. Flutter processes events according to its own rules. If AndroidView wins the event, the event will be encapsulated into the corresponding event at the Native end and sent back to Native through the method channel. Native then processes it according to its own rules for handling events.
3. Summary
3.1. Limitations of the scheme
To sum up, this solution is Google’s solution to the contradiction between the growing business needs of developers and the backward ecosystem, which is the main contradiction that a new ecosystem must face. The easiest way to solve this problem, of course, is to allow developers to use controls that are already well established in the old ecosystem. Of course, this can temporarily solve the problem of the incomplete development of Flutter ecosystem, but using this solution inevitably requires the writing of two-end code (even there is no corresponding control for iOS at present, of course, it will be updated later), which cannot achieve true cross-end.
To summarize, there are performance drawbacks to this solution. In the third comment of the AndroidView class, it is officially stated that this is an expensive solution. Avoid using the Flutter control when it can be implemented. If you have read the article “Flutter external Textures” before, you should know that the process cost of implementing Flutter external textures from GPU->CPU->GPU is relatively high, which will cause significant performance defects in the scenario of heavy use. We bypass the intermediate CPU step by some means, and implement this technology in APP for processing image resources.
3.2. Practical application
At present, the migration of idle fish from Native to Flutter has encountered the problem that the local image resources of Native cannot be accessed on the Flutter side. Since Flutter and Native will coexist for a long time, it is certainly possible to copy another resource to store it according to the rules of Flutter. But it inevitably increases package size and is difficult to manage.
Our solution to this problem is to take the Texture approach from AndroidView and optimize it. Realize the normalization of image resources of Native and Flutter. In addition to loading Native images in the Native resource directory, Native’s image library can also be used to load network images.
The reason for us to do so is that our image library in Native side is relatively complete and we have experienced a lot of online tests. At this stage, we do not want to put too much energy into the repetition of wheel building, and the idea of processing online image resources and local image resources is actually the same. Therefore, we choose to integrate the picture resources uniformly. After communicating with the official team and improving it, we will synchronize with you. Please pay attention to our public account.
3.4. Reference
Amap SDK documentation
Never thought – Flutter is attached to texture
Analysis of principle of double screen display