This is the twentieth article in a series that covers the implementation logic of PlatformView on Android and explains why the PlatformView keyboard is always problematic on Android.

The paper also explains why iOS is relatively stable.

Article summary address:

A complete series of articles on Flutter

A series of articles on the world outside Flutter

1、为什么有 PlatformView

Because Flutter implementation is similar in concept to WebView on Android, Flutter is implemented by converting Widget Tree to texture and rendering the widgets via Skia. This makes for excellent cross-platform performance but also introduces irreversible compatibility issues.

1.1. Native platform controls cannot be integrated

Just like the WebView, the Flutter UI is not converted into an Android control, but rendered directly on the SurfaceView by the Flutter Engine using Skia.

This means that the Flutter UI will never contain Android Native controls by default. This means that common controls such as WebView or MapView cannot be integrated into Flutter.

To solve this problem, Flutter creates a control logic called AndroidView. Developers can use this Widget to embed Android Native components into the Flutter UI.

1.2, the implementation of AndroidView

AndroidView Widget needs to be combined with Flutter to display fully: Draw the content that needs to be rendered by AndroidView to the VirtualDisplays in Flutter, and then draw the screen to the corresponding memory of The VirtualDisplay.

VirtualDisplay is similar to a VirtualDisplay area and needs to be invoked together with DisplayManager. It is generally used in secondary screen display or recording scenarios. VirtualDisplay renders the contents of the VirtualDisplay area on a Surface.

As shown above, simply put, the content of the native control is drawn into memory. Then the Flutter Engine gets its rendering data from the corresponding textureId and displays it.

By taking textures from the VirtualDisplay output and mixing them with Flutter’s original UI rendering tree, Flutter can graphically insert Android native controls into its own Flutter Widget tree.

1.3. Is there any other way to achieve this?

On iOS, a method similar to VirtualDisplay is not used. Instead, the Flutter UI is composed by dividing the Flutter UI into two transparent textures: one under the iOS view and one on top of it.

The advantage of this is that the Flutter UI that needs to be rendered below the iOS platform view will eventually be drawn onto the texture underneath it. The Flutter UI that needs to be rendered on top of the “platform” will eventually be drawn with the texture above it. They just need to be combined at the end.

This approach is usually better because it means that the Android Native View can be added directly to the Flutter UI hierarchy.

However, Android does not support this mode because on iOS there is a callback notification after the frame renders. For example, when the iOS view moves down 2px, we can also render all other Flutter controls in its list down 2px.

On Android, however, there is no system API for synchronous output rendering. Forcing Android to use the Flutter UI in this way will eventually cause problems such as AndroidView not synchronizing with the Flutter UI.

For a detailed discussion of this alternative method, see flutter. Dev /go/ NSHC

2. Related problems and solutions

Although it was possible to embed Android controls into the Flutter UI using VirtualDisplay earlier, this VirtualDisplay intervention has other troubling issues to deal with.

2.1. Touch events

PlatformViews does not receive touch events by default.

AndroidView is actually rendered in VirtualDisplay, and whenever the user clicks on the “AndroidView” they are actually “clicking on the Flutter texture being rendered. User generated touch events are sent directly to the Flutter View, not the AndroidView they actually clicked on.

2.1.1 solutions

  • AndroidViewUse the click test logic in the Flutter Framework to detect whether the user’s touch is in an area that needs special processing.

Detailed Description of the complete Development of Flutter (13)

  • When the touch is successful, a message is sent to the Android Embedding that contains the details of the touch event.

  • In Android embedding, the coordinates of the event match the coordinates of the AndroidView in VirtualDisplay, and a MotionEvent is created to describe the touch. And forwards it to the real AndroidView in the internal VirtualDisplay for response.

2.1.2 limitations

  • The implementation logic will distribute the new MotionEvent directly to the AndroidView, and if that View is derived from another View, the touch message may be sent to the wrong location.

  • During the transformation of MotionEvent, some information may be lost due to different mechanisms.

2.2. Text input

In general, AndroidView cannot get text input because the VirtualDisplay location will always be considered unfocused.

Android does not currently provide any API to dynamically set or change the focus Window of a Flutter. The focused Window of a Flutter usually holds the “real” Flutter texture and UI and is directly visible to the user.

InputConnections (how to enter text in Android) are usually discarded in unfocused views.

2.2.1 Solutions

  • Flutter rewrite the checkInputConnectionProxy method so that Android will think Flutter View is as AndroidView and input method editor (IME) agent, This allows Android to get InputConnections from the Flutter View and apply them to AndroidView.

  • In Android Q, InputMethodManager (IMM) is changed to each Window instantiating itself instead of a global singleton. So the naive “set up proxy” mode doesn’t work at Q. To further solve this problem, Flutter creates a subclass of Context that returns the same content as the IMM in the Flutter View. This eliminates the need for the actual Window to be returned when the IMM is queried. This means that when Android requires IMM, VirtualDisplay will still use the IMM of Flutter View as a proxy.

  • When AndroidView is asked to provide an InputConnection, it checks whether AndroidView is indeed the target of the input. If so, the InputConnection in AndroidView will be fetched and returned to Android.

  • Android considers the Flutter View to be focused and usable, so AndroidView’s InputConnection can be successfully acquired and used.

2.2.2 WebView keyboard input in Platforview

WebView inputs are more complex on versions prior to Android N because they have their own internal logic to create and set up input connections that do not fully follow Android protocols. In the Flutter_webView plug-in, you also need to add additional workaround to enable text input in the WebView.

  • Set a proxy View that matches theWebViewListen for input connections on the same thread. Without this feature,WebViewWill consume all internallyInputConnectionWithout notifying the Flutter View agent.
  • In the proxy thread, return to the Flutter View to create the input.
  • Reset the input connection back to the Flutter thread when the WebView loses focus. This prevents text input from getting “stuck” in the WebView.

2.2.3 Limitations

  • Often this logic depends on the internal behavior of Android and can be vulnerable, such as keyboard input exceptions for Huawei devices in version 1.12.

  • Some text functions are still unavailable, for example: The Copy and Share dialog boxes are currently unavailable.

3, summarize

The implementation mode of PlatformView increases the vitality and vitality of Flutter, but it also introduces many problems, such as the high number of #webview-keyboard, #webview and #platform-views related issues. And as the documentation for the webview_FLUTTER plugin states:

This plugin relies on the new mechanism of Flutter to embed Android and iOS views. Since the mechanism is currently in developer preview, the plug-in should also be considered developer preview.

Keyboard support for WebView_FLUTTER is also not yet ready for production, as keyboard support in Webview is currently in the experimental stage.

So by now you should know why Flutter PlatforView is so difficult to implement on Android and has so many keyboard input problems.

Since then, chapter 20 has come to an end! (/ / / del / / /)

Resources to recommend

  • Making: github.com/CarGuo
  • Open Source Flutter complete project:Github.com/CarGuo/GSYG…
  • Open Source Flutter Multi-case learning project:Github.com/CarGuo/GSYF…
  • Open Source Fluttre Combat Ebook Project:Github.com/CarGuo/GSYF…
  • Open Source React Native project: github.com/CarGuo/GSYG…