First, concept introduction

In the black box use case design method, there is a well-known orthogonal analysis method, but few people know the “Pairwise” design concept.

Consider a common scenario in which a tight project can achieve higher quality in less time after the scheduled testing time has been “unreasonably” compressed.

For example, if you were to test word fonts, how many use cases would you compile?

The simple answer is 2 to the seventh =128 use cases, but which use cases do you choose to execute when the deadline is really tight?

Pairwise (www.pairwise.org/) is an idea that works… L. Thurstone(29 May1887 — 30 September 1955) was first proposed in 1927. He is a psychostatistician in the United States. Pairwise is based on mathematical statistics and optimization of the traditional orthogonal analysis method.

Pairwise is based on two assumptions:

(1) Each dimension is orthogonal, that is, each dimension has no intersection with each other.

(2) According to mathematical statistical analysis, 73% of defects (single factor 35%, double factor 38%) are caused by the interaction of single factor or two factors. Nineteen percent of the defects were due to the interaction of three factors.

Therefore, Pairwise is based on the most cost-effective set of use cases resulting from interactions covering all 2 factors.

For example, when factor A is A1 and factor B is b1, then A cannot be a1, B is b1 or B is b1 and A is A1 (why the reverse description, you can think about ).

Of course, in reality, there will be various constraints, so there will be IF… Then… Or in the same scenario, the combination of factors A, B, and C can be simplified, while factors D, E, F, and G need to be combined altogether (the term “order” in Pairwise). It is too troublesome to completely rely on manual output, each major contributor contributes their own open source tools, here pick a big company (Microsoft) to introduce the tool name PICT.

Tool download address: github.com/Microsoft/p…

The latest version supports multiple platforms (Linux, MAC, Windows) and requires self-compilation.

Two, PICT tool use

1) Prepare a text file abc.txt with the following contents:

Run Pict abc.txt to get the result after pairwise:

If you want the full combination, you can specify /o:3 in the command line argument, where 3 is the number of factors (the default value of /o is 2, which is the “order” described above) as follows:

2) Specify the combination of factors in the text,

Run Pict abc.txt directly to get the full combination of

3) Factor judgment structure:

If A is a1, B does not equal B2 and C does not equal c3.

Iii. Case practice

Take “spark wallet – account management” “query” function test as an example to introduce the use of PICT tool ideas and process in detail.

The query factors include Spark User ID, Account Opening Status, Account opening Channel, and Account Opening Time. The limits of each factor are as follows:

  • Spark user ID: unlimited input
  • Account opening status: Please select (default), Account opening success, account opening failure
  • Account opening channel: Please select (default), account opening success, account opening failure
  • Account opening time: Select the time range (limit of the front page: fill in one of the values, the other value will automatically fill in the same time, end date >= start date) (do not consider the background interface limit).

The range of values to be considered for each factor is:

Spark user ID: empty (default), non-digit, 120988476548, 120988476900

(Assume that 120988476548 can be queried, and 120988476900 can not be queried)

Account opening status: Please select (default), Account opening success, account opening failure

Account opening channel: Please select (default), PC, WAP, APP

Account opening time: empty (default), 20150101, 20160101, 20170101

End time: empty (default), 20150101, 20150301, 20160201, 20160301, 20171101

The three factors of “Spark user ID”, “account opening status” and “account opening channel” are easy to sort out, as shown in the picture, and can be directly filled into the file according to the above contents:

The “opening start time” and “opening end time” have dependencies, so special treatment is required. We “translate” the dependencies between them into a way that PICT can understand:

1) “Fill in one value and another value will automatically fill in the same time”

In other words, “account opening start time” and “account opening end time” can only be empty at the same time.

Translation:

IF [open account start time]= “empty” THEN [open account end time]= “empty “;

IF [open end time]= null THEN [open end time]= null;

(Of course, you can also write: IF [opening time] NOT IN {"20150101"."20160101"."20170101"} THEN [open end time]="Empty"; IF [open end time] NOT IN {"20150101"."20150301"."20160201"."20160301"."20171101"} THEN [opening time]="Empty";)Copy the code

2) “End Date >= Start Date”

Translation:

IF (open user start time) <> empty AND (open user end time) <> empty THEN (open user end time) <= (open user end time);

Have all the “translated” results ready, as shown below:

Let’s first look at the total number of use cases for the “full combination”. 481 answers, as shown in the figure:

Using the Pairwise idea, the number of use cases sorted out is only 30, as shown in the figure:

The use case details are as follows :(each line is each use case)

ID of spark user Open door state Open door channels Opening time Closing time of account opening
120988476900 Please select a Please select a empty empty
120988476548 Open door failure PC 20150101 20171101
The digital Open door failure wap empty empty
120988476900 Open door success app 20150101 20150301
empty Please select a pc 20160101 20160301
120988476548 Open door failure app 20160101 20160301
empty Open door success wap 20170101 20171101
empty Open door failure Please select a 20150101 20150301
The digital Open door success pc 20150101 20150101
empty Please select a wap 20150101 20150101
The digital Open door success Please select a 20160101 20171101
The digital Please select a app 20150101 20160301
120988476548 Open door success app empty empty
120988476548 Please select a app 20160101 20160201
120988476900 Open door failure wap 20160101 20160201
120988476548 Open door success wap 20150101 20160301
The digital Open door success Please select a 20150101 20160201
empty Open door success pc empty empty
120988476900 Please select a app 20170101 20171101
120988476900 Open door failure Please select a 20150101 20150101
empty Please select a app 20150101 20160201
120988476548 Open door failure app 20150101 20150101
120988476900 Open door success Please select a 20150101 20160301
120988476900 Please select a pc 20150101 20150301
120988476548 Open door failure Please select a 20170101 20171101
120988476548 Open door failure wap 20150101 20150301
empty Please select a pc 20150101 20160201
The digital Please select a Please select a 20150101 20150301
The digital Open door success pc 20170101 20171101

Four,

Pairwise is an effective test case generation technique, which reduces the number of test cases by combining all dimensions and values of test variables, avoiding enumerating all values of all dimensions and their combinations.

Back then, Cohen et al. used the Pairwise test technique to test the “Sort” command in Unix, and the test results showed that the coverage rate was more than 90%.

In an era where iterative development testing is so efficient, this testing method needs to be adopted and applied to testing.

Author: Song Jingang

Source: Creditease Institute of Technology