Abstract: Test strategy describes the overall approach and objectives of test engineering. Describe what phase of testing is currently under way and what kind of testing (functional testing, performance testing, coverage testing, etc.) is taking place within each phase, as well as the testing manpower scheduling, etc.

This article is shared from huawei Cloud community “Discussion on agile Development test strategy”, author: Agile jianghu Taotaodao Sister Mei.

preface

With the advent of Agile and DevOps, the traditional model of software development has been changed. At the same time, testing is also faced with a challenge. In agile development, short cycle iteration delivery means shorter time, embracing change means frequent change, and the way user stories describe requirements means less documentation. Being on a full-featured team means fewer dedicated testers. Given this situation, how do you make testing agile and do it well? Today we are going to talk about how to do a good test strategy for agile development.

Agile development test strategy

A test strategy describes the overall approach and objectives of a test project. Describe what phase of testing is currently under way and what kind of testing (functional testing, performance testing, coverage testing, etc.) is taking place within each phase, as well as the testing manpower scheduling, etc.

We can plan a reasonable test strategy according to the test purpose, scope, start and end time, personnel arrangement, tools, namely 5W1H method.

  • Why: What is the purpose of the test?

  • What: The content and scope of the test, what to test, determine the focus of the test (RBT requirements based testing, etc.)

  • When: The start and end time of the test, considering the factors that affect the time

  • Where: location of relevant documents, location of defects, environmental geology

  • Who: Arrangement of testers

  • How: Which testing tools and methods to use for testing

Why

According to agile testing principles, the purpose of testing is to prevent defects and help teams build the best possible system. You can set an overall goal for testing based on business and project characteristics.

What

According to the four test quadrants, the test content is divided into classes from business and technical perspectives, as well as program and product perspectives, as shown in the figure below.

Figure 1 testing the four quadrants

According to the agile hierarchical planning principle, test testing also adopts different levels of testing, and you can refer to epic-feature-story-task to formulate strategies. The following can be used as a reference for formulating strategies. Businesses and products are mostly different and can be adjusted according to the characteristics of their own businesses and products.

While the agile development process is made up of iterations, Epic is made up of several iterations, usually integration testing and end-to-end testing; A Feature is usually implemented in several iterations, usually with Feature testing, functional testing, UAT, and scenario testing; Stories are usually completed within iterations, usually with functional testing, user Story testing; Tasks are tests within an iteration, typically unit tests, module tests, and code quality tests. The performance tests cover the Story, Feature and Epic levels.

When

In the traditional waterfall development model, testing is a phase, after the program is written into the test phase, as shown in the figure below.

Figure 2. Waterfall development pattern

In the Agile development model, testing is not just a phase, but an activity, with each Sprint having a test activity. Each iteration includes unit testing, code quality testing, user story testing, feature and capability acceptance testing; A sprint-level regression test, implemented as an automated test, is done starting with Sprint2. After several iterations have accumulated, end-to-end integration testing is performed before release. See the figure below.

Figure 3 Sprint test activities

Where

Although agile development uses the form of light documentation, it is also necessary to manage relevant documents. At the beginning of the test, the management and storage of relevant test deliverables should be agreed upon, including but not limited to test policies, test artifacts, defects, test data, virtual services, and automated scripts. This is typically managed in a project management tool and is associated with the development work items so that they can be traced and viewed later. In huawei Cloud DevCloud, for example, you can upload documents to Wiki and documentation, and then establish associations among work items.

Figure 4 Huawei Cloud DevCloud example

Who

In Agile development, testing activities work for the team, not just the testers. Among them, TDD, unit testing and code quality testing should be done by developers. Meanwhile, because interface testing involves internal logic problems such as data exchange, transfer and control management between interfaces, it is also recommended to be done by developers. Testers include testers within an iteration and technicians across iterations. The testers within the iteration are primarily responsible for the design and execution of the iteration tests, including exploratory tests and APIS, development and execution of UI test automation scripts, and automated regression smoke tests. Cross-iteration testers are more focused on coordinating tests and developing a test automation strategy.

At the same time, the tester is a member of the team who not only performs testing, but also participates in test planning, evaluation and scheduling, reviews, and any other team activities.

How

In order to work well with the agile development model of small steps and early delivery, testing requires the ability to test quickly and provide early feedback. In the time-constrained framework of Agile methods, the ability to automate tests is essential and can greatly relieve the stress of testing. According to Mike Cohn’s test pyramid, the ratio of automated tests is 7:2:1, i.e. 70% for unit tests, 20% for interface tests, and 10% for UI tests, thus achieving layered automation. On the basis of automation, manual exploratory testing should also be carried out.

Figure 5 Test pyramid

There are a lot of automation tools available, open source tools like Appium, Cucumber, Protractor at the UI layer, POSTMAN at the API layer, and DbFit at the database layer. Commercial tools such as IBM RFT, LeanFT in UI layer, SmartBear in API layer, etc.

In the selection of automation tools, we should start from the actual situation, from the cost budget, support platform, support language, measurable applications, technical requirements and other aspects to consider. Open source tools save cost, commercial tools cost is high; The choice of open source tools should also be combined with the code ability of team members. Open source also has its own technical difficulties. The subsequent support level of the tool should also be taken into account, and problems will inevitably be encountered along the way.

Test Policy example

A product typically consists of several releases, as shown in the figure below.

Figure 6 agile development

Taking a release cycle as an example, let’s look at the timeline of testing:

Figure 7 shows an example of a test policy

When developing a testing strategy, be careful to schedule proper testing rhythms and cycles, and the best testing is fully automated daily testing.

Afterword.

The 5W1H test strategy given above can be used as a reference. The most important thing to keep in mind is that the purpose of testing is to prevent defects and help the team build the best possible system to deliver valuable products to customers. Therefore, the test strategy of quality left shift should be regarded as one of the most important core concepts of project management throughout the delivery of the entire software life cycle. Quality should be moved to the front end of the whole life cycle through defect prevention, and major defects should be found as early as possible through the development of risk-based test strategy driving.

Click follow to learn about the fresh technologies of Huawei Cloud