Why do you need automated tests?

  1. Instead of manual repetition, test engineers can spend more time designing comprehensive test cases and testing new functions.
  2. Improve the efficiency of regression testing, suitable for agile development process
  3. Better use of non-working hours to execute tests and analyze failed use cases during working hours
  4. Efficient implementation of certain types of tests that cannot be done or are costly for manual testing, such as: 7*24 hours continuous system stability testing for critical business and stress testing for high concurrency scenarios [instead of complex manual testing and long testing]
  5. It can also ensure the consistency and repeatability of the steps and verification of each test, avoiding human omission or negligence [Ensure the consistency of operation and traceability of results]

 

Some disadvantages of automated testing

  1. It cannot completely replace manual testing, but can only replace the repeated steps with high frequency and mechanization in manual testing.
  2. Unable to cope with the changes of the system under test, the maintenance cost of automated test cases is high; Because automated testing only performs pre-defined test steps and verifies test results, it has no ability to deal with obvious errors and unexpected events during execution [high maintenance costs, unable to cope with system changes and emergencies].
  3. The development workload of automated test cases is much larger than that of manual word tests, so the cost can be recovered only when the effective execution times of completed test cases are ≥5 [the development of use cases is large and the input-output ratio is difficult to improve]
  4. Manual testing found more defects, automated testing only found defects in the regression test range.
  5. The efficiency of automated testing depends greatly on the design and implementation quality of automated test cases. Unstable automated test cases are worse than no automation.
  6. In the early stage of the implementation of automated testing, the efficiency of use case development is very low. After the system is mature and test engineers are skilled, the initial use cases should be reconstructed.
  7. The combination of people who know the business + people who know the automation technology can effectively conduct automated testing
  8. Automated test developers must have programming ability

 

What projects are suitable for automated testing?

First, demand is stable and will not change frequently

The high frequency of requirement changes can cause the maintenance cost of automated test cases to skyrocket

 

Second, long development and maintenance cycles require frequent regression testing

Software products are better suited for automated testing than software projects

  • First, software products have a long life cycle, with continuous iterations and regression testing requirements for each release
  • Second, the advantages of automated testing can be better demonstrated when the execution ratio of automated test cases is higher than 1:5

Automated testing of software projects needs to be case-by-case

  • Short-term projects: Automation is not recommended, even if it is technically feasible, but the ROI is not high; Choose manual exploratory testing, where finding defects is the first priority
  • Medium – and long-term projects: automated testing of stable functions, manual testing of functions with large changes or unclear requirements, and the ultimate goal is to cover 80% regression tests with 20% of the effort

 

Third: scenarios where the same tests need to be run repeatedly on multiple platforms

 

Fourth: some test items cannot be achieved through manual testing, or manual costs are too high

For all performance and stress testing, it is difficult to implement manually, such as:

  • Benchmark performance test of 10,000 concurrent users
  • 7*24 hour stability test

At this point, a machine must be used to simulate a scenario in which a large number of users repeatedly operate the software under test

 

Fifth: the development of the tested software is more standardized, which can ensure the reliability of the system

To achieve stable automated testing, the development process of the software under test must be standardized. For example, if there is no rule to find the name of the control on the GUI, it will cause the unstable identification and positioning of the automatic control on the GUI, affecting the efficiency

In addition, the automation of some use cases must require developers to reserve testability interfaces in the product, otherwise subsequent automation will be difficult to implement.

For example, the user login operation may need the image verification code, need to develop a path to bypass the image verification code, or need to use optical character recognition (OCR) technology to identify the image verification code, but its recognition rate will be very low, directly affecting the stability of the use case