Author: Idle fish technology — small artisan

The Flutter page cannot directly locate elements using Native testing tools, which makes automated testing difficult. Although Google has officially launched the Flutter Driver and Integration test, there are the following problems in practical use:

• Not suitable for mixed stack apps, although appium has the relevant driver, it cannot switch the environment. • Element localization is relatively weak. • Depends on VMService and needs to build a Profile or Debug package.

For the above reasons, we did not directly use the official Google tools. Instead, we chose to expand the testing capabilities of Flutter pages based on Native testing tools. This paper analyzes the principle and implementation of Flutter Driver and Integration test, and briefly introduces the attempt scheme of Idle Fish in UI automatic test.

A, Flutter driver

When we first got into the automatic test of Flutter, we tried to use the Appium framework to drive APP. When we used the inspect function to dump page elements, we found that many elements would be merged into a region block, and then we could only locate them by xpath when clicking, it would be difficult to locate some specific elements. Xpath is also easy to change, and the code is not maintainable. Because of the above reasons, we began to investigate the Flutter driver, a test tool provided by Flutter official. When we first used this framework, we found that it was only suitable for pure Flutter applications, but not for mixed stack applications. However, the element positioning capability provided by the underlying framework might be useful to us, so we analyzed its source code. The framework’s schematic diagram 1 is shown below.The process interaction of the whole flutter driver framework is relatively simple. When the test script is running, it first uses Flutterdriver.connect () to connect VMService to obtain the relevant ISOLATE. The operation process and data acquisition are then transmitted through websocket. All operations on the test script side are serialized as JSON strings and passed to Ioslate through websocket to be converted into commands for execution on the APP side. For example, if we want to obtain the text content of a component, the resulting JSON structure is as follows:

{" jsonrpc ":" 2.0 ", "id" : 5, "method" : "ext. Flutter. Driver", "params" : {" finderType ":" ByValueKey." "keyValueString":"counter", "keyValueType":"String", "command":"get_text", "isolateId":"isolates/4374098363448227" } }Copy the code

After understanding the above principles, a protocol format can be constructed to drive the FLUTTER test in any language and test framework. Therefore, we encapsulated the protocol and used Python to drive the FLUTTER test. This allows you to test flutter pages using UIAutomator2 and Facebook-WDA to meet the test requirements of the Flutter hybrid stack application. The final implementation code demo is as follows.

from flutter_driver.finder import FlutterFinder from flutter_driver.flutter_driver import FlutterDriver import uiautomator2 as u2 if __name__ == "__main__": d = u2.connect() driver = FlutterDriver(d) if pageFlutter is True: # If it is flutter, Connect ("com.it592.flutter_app") Finder = flutterfinder.by_value_key ("input") driver.tap(finder) time.sleep(1) print(driver.getText(FlutterFinder.by_value_key("counter"))) else: d(text="increase").click()Copy the code

We tried to use this framework and found that the underlying capabilities of flutter Driver were relatively weak and could not fully meet our requirements. The main problems are as follows:

• Cannot batch manipulate elements, and will throw an exception if the Finder locates more than one element. • Most of the time, developers do not write key, element positioning is not so convenient. • Because Flutter does not have the inspect dump element, scripts can only be written with source code, resulting in high maintenance costs. • The project has been officially abandoned, so no new features are expected.

​

Second, the integration_test

As mentioned earlier, Flutter officially abandoned the maintenance of the Flutter Driver and launched a new test framework called Integration_test. Will this framework support mixed stack applications? The official documentation states that “the package allows self-actuating tests of Flutter code on devices and emulators”. The underlying element operations and positioning of Integration_test are driven by Flutter_test, which has the following advantages:

• Test scripts can use various Flutter apis. • When you pack IPA and APK, you can run tests on Firebase Test Lab and other devices without additional drivers. • Integration_test has no correlation between tests for each page, allowing for single-page-level testing.

However, since the underlying elements are positioned in the same way as the Flutter driver, the problems with the Flutter driver still exist, as well as other limitations:

• Test scripts are packaged into the APP and need to be repackaged each time the script is modified. • Not friendly to end-to-end testing, requiring additional functions to wait for data to finish loading. • Not suitable for full-link level page testing. • Poor scalability

Based on the above problems, it does not meet our use requirements, so we just do a simple pre-study, without in-depth understanding and application.

Xianyu UI automated test scheme

After studying the relevant test framework launched by Flutter official, we began to think about the future of Idle Fish UI automation. Whether to build the wheel on the shoulders of the authorities or extend Flutter testing capabilities by reusing existing native test automation capabilities. After considering the cost and maintenance difficulty of test scripts, we chose to use image processing technology to expand the test capability support of the native automation framework for Flutter pages. The whole test scheme architecture is shown in Figure 2.The elements of Flutter are not completely unrecognizable by UIAutomator2 and Facebook-WDA, so only unrecognizable elements need to be handled when writing test scripts. For elements whose name, label and xpath are not easy to change, we preferentially use the native positioning ability for positioning operations, while image processing technology is directly used for positioning operations for other elements. When dealing with elements that cannot be located using native ability, we prefered to use OCR word matching for location, which has high accuracy and is not easily affected by resolution. For pure images, we used image search for location. For some common element controls such as commodity card, price, icon, head picture, etc., we build a training set and use image classification to judge the type of elements, so as to realize the positioning of common controls. The biggest problem with UI automation is that as versions iterate, so do test scripts. Therefore, the robustness and maintainability of scripts should be considered in the process of scheme selection and script writing. In the actual script development, we encapsulated the page elements into separate classes and separated them from the test logic, thus ensuring that only the corresponding page elements need to be modified in the later element iteration and reducing maintenance costs.This scheme has been used for UI operations related to automated testing of idle fish performance, and there is no need to distinguish what type of page is currently on when scripting. Our script has run stably over 500 times with a success rate of over 98%.

Four,

It can be seen from Figure 4 that the support of the flutter Driver and integration Test for the mixed stack is not mature enough. However, the Flutter Driver can be extended to some extent. For pure flutter applications, this scheme can basically meet the test requirements. Integration test is relatively less mature. For the test of hybrid stack application, the scenario switching cost of hybrid stack may still need to be taken into account. Some EXPANSION using OCR technology may cost less and benefit more.

Five, thank you

Thanks to SLM and TMQ, we can focus on our business.

• flutter driver (flutter. Dev/docs/cookbo…

​