It takes about 1.8 minutes to read this article.

preface

One question I’ve seen before is: if you have a monthly release that requires regression to a feature before it goes live, should you automate testing that feature if the automated script to implement that feature only takes a day? There is no absolute answer to this question, and it has a lot to do with the specifics of the actual project. We should have a correct concept of automated testing in mind: “The fundamental purpose of automated testing is to improve efficiency and reduce costs.”

Before implementing automated testing, we need to think about the following:

First, does the project really need automated testing, and what is the input/output ratio?

Second, what automation solution is more suitable?

Finally, how do you automate?

In actual projects, many students think too much about the third question and do some research on the second question, but often lack of thinking about the first question. Each team should carefully consider the costs and benefits of automated testing from the perspective of its own project, and choose an approach that is appropriate for its project to maximize the input-output ratio.

For more information about our team’s thinking on these issues, see another article, “Exploring the Road to Terminal Automation Testing.” Once the benefits of automated testing have been established, the next step is to choose a test solution to keep development and maintenance costs as low as possible.

Here are some thoughts after a year or so of practice.

Automated UI testing based on Appium

  • Consider what are the major changes to the application under test and whether it is really suitable for UI automation testing. If the application UI changes infrequently, mainly in the underlying logic, then this application is better suited for UI automation testing. On the other hand, if the UI changes a lot, then the UI automation script maintenance costs will be high and the input-output ratio of automated testing will not be high. So we settle after a year or so of practice a set of basic stable automation suite, mainly covers the application of some of the main process of UI haven’t changed much, basic can achieve higher input-output ratio, this part before with separate written an article: “terminal test automation quest”.

  • The type of application under test should be considered, because different types of application have an important impact on the selection of automation framework. If it is a game application, many images may be directly rendered by OpenGL, and Appium cannot find elements in the images directly rendered by OpenGL. At this time, we can consider the way of image recognition to make a judgment. For example, netease’s Airtest framework is more suitable for automated testing of game UI.

  • What are the objectives of UI automation testing and whether there is a requirement for test run time? If UI automation’s goal is to quickly return to, in a short period of time required for a large number of return to the operation of the case, may not share the UI test automation to do at this time, because the UI test automation to run the same test cases generally than artificial execution time longer, so it’s hard to run in a short time a large number of test cases. However, if there is no time requirement, such as BVT running at regular intervals every night or developing smoke tests after integration of one module at a time (about 3 minutes on average, mainly measuring the main flow), time efficiency is not considered.

  • Since it’s UI automation based on Appium, there’s a lot of freedom in language choice. At this time, students in the department can consider the ability to choose a language with low cost of learning and implementation. We choose Python here.

Competitive Product Comparison Test Based on UIAutomator (Performance Test)

  • Since UIAutomator does not need to rely on code for automated testing, it is suitable for automated analysis of competing products.

  • Cross-process has strong compatibility, performance data collection can be accessed through third-party apps, and no special adaptation is required. Here, we are connected to netease’s Emmagee, which can easily collect the data of memory, CPU, fluency and other aspects of the target application.

Once the test plan is decided, you need to consider how to implement it. Students who have experience in automated test development should know that it is not difficult to develop automated test scripts, but the maintenance of test scripts is more difficult. The idea of test script design is to improve the reusability and stability of the test script as much as possible, reduce the maintenance cost of the script, and increase the income.

Recommended reading:

Automated quality assessment dimensions

Exploring the path of terminal automation testing

Want to understand some sense, meet some interesting things — off the island