It’s time to talk about automation again, and it feels like “automation” is now a topic that we testers can’t avoid, either actively or passively.

Take initiative. You feel the need to do some testing, so you learn about it and apply it at work.

Passive, maybe you want to change your job. When you open the recruitment JD of various companies, you will see “automation” again, again have to learn.

What is automated testing

What I mean by that is: behavior that replaces our manual testing.

For example, common Web automation and interface automation is to perform page operations and interface operations on our behalf and verify that the results are as expected.

Second, why do automated tests

Automation has certainly had its advantages in recent years:

  • Instead of doing a lot of repetitive work, testers can spend more time working on new features or more comprehensive use case designs.
  • Can greatly improve the efficiency of regression testing, suitable for the agile development process.
  • It makes better use of unattended time to execute tests more frequently, especially in the current mode of running tests outside of working hours and analyzing failed use cases during working hours.
  • It can efficiently implement certain types of tests that cannot be done manually or are costly, such as system stability tests with 7×24 hour continuous operation of critical business and stress tests with high concurrency scenarios.
  • It also ensures consistency and repeatability of each test execution and validation, avoiding human omissions or omissions.

Weaknesses? That must exist:

  • Automated testing does not replace manual testing, it can only replace manual testing in the execution of high frequency, mechanical repeat steps, play a complementary role.
  • Automation is not intelligent, we will only follow the established steps in the code to execute, the middle of any accident, there is no processing power.
  • Automated test case development requires much more work than a single manual test, so the cost of automated testing can only be recovered if the test case is executed more than or equal to 5 times.
  • Automated testing usually finds few bugs and is primarily used for regression.
  • The efficiency of testing largely depends on the design of automated test cases and the quality of implementation. It is better not to have unstable automated tests.
  • In the early days of automated testing, use-case development is often inefficient.
  • Automated test developers must have some programming skills, which can be a challenge to traditional manual test engineers and require additional learning costs.

So, when it comes to automated testing, you can’t generalize. It all depends on whether your project is right for you.

What project is suitable for automated testing

1. The demand is stable and will not change frequently

If requirements change frequently, the cost of your automated maintenance will skyrocket. Just think about the case you just debugged, the resulting page or interface logic changes, and you have to retune.

2. The development and maintenance cycle is long, requiring frequent regression testing

We are talking about software products, such as wechat app and Taobao Web system. Software products generally have a long life cycle, with multiple releases, each with a large number of regression testing requirements.

For project software (e.g., developing a system for XX company and delivering it in 2 months), it depends.

In general, short-term, one-off projects are not suitable for automated testing, and the ROI is not high. Manual exploration is more appropriate.

For medium – and long-term projects, this can be done:

  • Automated testing of more stable software functions,
  • Manual testing of features that change significantly or whose requirements are temporarily unclear,

The ultimate goal is to cover 80% of your regression tests with 20% of your effort.

3. Scenarios where the same tests need to be run repeatedly on multiple platforms

This is actually quite common, such as multi-browser execution on the Web side, multi-system version or multi-model testing on the APP side, etc.

Individual test cases in these projects need to be executed multiple times to maximize the return on investment for automated testing.

4. Some test items cannot be achieved through manual tests, or manual costs are too high

Usually for performance and stress tests.

For example, if a project requires a benchmark performance test of 10,000 concurrent users, do you really want 10,000 users to operate the software under test according to your password? For example, for a 7×24 hour stability test, do you also need a group of users to operate the software under test day and night?

This is where automated testing techniques have to be used, using machines to simulate.

5. The tester has some programming ability

It can be difficult to implement automated testing if the test team members have no background in development programming:

  • Short-term benefits: the cost of learning in the early stage is usually quite large, and it is difficult to make substantial help to the actual project in the short term. At this point, if management does not have the right expectations for automated testing, it is likely to stop automated testing.
  • Put the cart before the horse: if testers cannot see automated testing in the right way, they may spend a lot of energy on the learning and practice of automated testing techniques, and neglect the design of test cases, which is fatal.

So, look at “automated testing” in context. It does free up some of the tester’s labor to do some tests that humans can’t do, but it doesn’t work for all testing scenarios.

If the cost of maintaining an automated test outweighs the cost savings, the gains are often outweighed.