Unsupervised automated testing, which has certain applications in our company; It’s not my side of the story, it’s my side of the story.

Let me start with the idea.

Of all automated tests, the ones that are easiest to automate and have the highest input/output ratio are generally interface tests. For interface testing, there are two ways to generate test actions automatically:

1. Automatically generate the interface definition document or file; The advantage of this method is that you can know the input parameters (request parameters, request headers, request body), so it is very convenient to do parameter composition.

2 Record through online traffic and play back. You can also do some intelligent combination of input parameters here. Then we need to look at how the test validation is generated automatically.

Under the thinking of 1, interface definitions are generally difficult to describe exception scenarios and rarely describe the meaning of the response (there are response constructs). So basically you can only verify that the status code is equal to 200, and that’s pretty limited.

Under the thinking of 2, there are no above problems, so if we want to do in-depth, it must be based on “online traffic recording after playback”.

Recording playback of the topic is very big, in fact, a lot of programs, here will not expand to talk; Record playback for unsupervised automated testing applications.

First we can think of it as recorded traffic, with requests and responses, that can be processed automatically into action and validation of use cases. We might have to filter domain names, handle cookies, and override requests for duplicate utility; The 403,500 requests cannot be filtered here, because such exceptions are either bugs or some very precious exception scenario (effective exception testing).

The timing of the test, in fact, is before the launch; And because the online traffic recording and playback, is based on the real environment, so we need to provide a complete service environment and database. It’s hard to recreate at a large company, so it makes more sense:

Grayscale environment is used for traffic, and when this test traffic enters the database, shadow tables are used to avoid impact (or the database is read-only and cannot write, but this is a significant limitation).

By comparing the response difference of the same set of requests before and after version launch, we can judge the change of regression function in this launch.

Eventually, of course, you need someone to do some marking, to mark the cause of the test failure; After a few waves of run, the machine can make the generation and elimination of code more reasonable through human annotation; Here you can use the AI algorithm model, but I don’t understand it, so I don’t want to talk nonsense.

Then the whole set of ideas can be summarized as:

1 Record traffic and combine input parameters

2. Before going online, the traffic is played back to the grayscale environment, and the response difference with the previous version is judged to determine whether the test passes

3 Use traffic isolation and shadow tables to eliminate impacts

4. Train and generate the use case model of manual marking failure

So the limitations of this idea are:

There is no ability to test new functions and interfaces

2 For requests that modify data, the false positive rate can be high unless you frequently prepare the initial data (i.e. dump the shadow table frequently).

3 Rely on user requests of a very high magnitude to have complex requests and test cases

High requirements for gray scale and flow control

The following is my automation for the technology of some induction and summary, I hope to help the heart of the technology on the road to go all the way to black friends! With tutorial learning materials ~

These materials should be the most comprehensive and complete preparation warehouse for those who do [software testing]. This warehouse also accompanied me through the most difficult journey. I hope it can also help you! Everything should be done as early as possible, especially in the technology industry, we must improve our technical skills.

Follow my wechat official number: [Sad Latiao] free ~

My learning exchange group: 902061117 group of technical cattle to communicate and share ~

If my blog is helpful to you, if you like my blog content, please “like” “comment” “favorites” one key three even oh!