“Haphazard automation,” says one, “leads to faster chaos.” Not just faster, but worse, bigger mess. When used properly, automation can be an amazing productivity booster and system quality enhancer in a test team. The key to automated testing is getting it right, which is the hardest part initially. Below, we’ll take a look at the most common automated testing challenges and what to do about them.

Challenge 1: Get management approval

As with any corporate department, employees need to know what budgets allow and don’t allow. Testers may already know automation in both business and technical advantage, shorten time to market, increase test coverage and accuracy, reduce test cost per hour, quickly find errors, etc.), but the testers how to persuade the financial department and the head of the software development, for the automated test for the necessary time and money?

“Automated testing improves productivity and accuracy, and this is where the business case must be made. The cost of a single defect… Can offset the price of one or more tool licenses.” – Randall Rice

We agree with Randall Rice that automation pays for itself. To demonstrate to management that the financial benefits are substantial, show them a simple breakdown of ROI for automated testing. It’s also important to be open and honest with others and other stakeholders about the upfront work and resource demands of automation.

Challenge 2: Choose and use the right tools

For a variety of reasons, many teams don’t make it through this phase. They may not have the expertise of a particular tool, they may not be able to find the ideal tool, the toolset is not broad enough, and so on. If you don’t have enough basic knowledge of how to use the tool, you have the following options:

  • Take an online course
  • Let the tool maker do the training
  • Hire a consultant to get a better grip
  • outsourcing

If existing tools don’t meet your needs, consider finding a multi-tool solution. Remember, it’s impossible to test everything completely, but you can use tools to test the most important parts.

Finally, if the tool costs more than you budgeted, do a quick cost and benefit analysis and show it. This measures the cost of previous mistakes and shows how much time and money could have been saved if the tool had been used.

Challenge 3: Determine the startup strategy

Ok, so now you have all the tools and support to start automating, but automate what exactly? And how? The tools themselves don’t tell you what to automate, which can be as frustrating as a new parent discovering their child has arrived without a parenting manual. Will it produce a generation of great automated tests, or will it be so damaged that it can no longer be managed? Of course, we’re hoping for the former! But there’s really no way to automate everything, so you have to be strategic. Two approaches can be used to help solve this problem: risk-based testing and automated pyramids.

Risk-based automated testing

It prioritizes testing of components that are most at risk of failure and that have the greatest negative consequences if said failure occurs. Consider here:

  • Potential false economic impact
  • Probability of failure
  • Service Level Agreement (SLA)
  • With or without property risk

This should give you a good way to prioritize automated test cases.

Automated test pyramid

Another highly recommended approach is to follow the automated pyramid. This topic was discussed more extensively in a recent article, and is briefly outlined here.

Ice cream cones may taste sweet and tempting, but adopting the ice cream cone approach could ruin the automation process! Following the ice cream cone approach leads to a high error rate because it emphasizes automation at the UI level, which uses tests that are more easily broken. But if you put effort into automated unit testing, errors can be found and eliminated almost immediately during software development.

Challenge 4: Setting realistic expectations for automation

No matter how good your tools and processes are, remember that testing is never done. Automated testing is not a panacea for a buggy system and should not replace, but be used in conjunction with, non-automated testing. Some tests cannot be automated at all, but there are also automated tests that can find errors that cannot be found otherwise.

Automated testing is really just checking the system automatically, and still requires manual non-automated testing. Also, remember that the value of a test comes from the information it provides, not the number or frequency of tests performed. Our primary concern is to get the right information so that we can make the best decisions to improve system quality.

Make sure the team and management agree and understand the expected results of the automation solution so that everyone is on the same page!