background

Recently, I participated in a project. The team wanted to automate testing, but the testers didn’t have programming ability, and the developers didn’t have energy to help. After exploring the existing automated testing tools, I decided to build a wheel by myself after failing to meet the team’s needs.

  • Postman

Postman is a very convenient automated testing tool that requires no coding capability. Since many test verification points in the project need to verify the correctness of data in the database, it is necessary to create some UNNECESSARY HTTP APIS for testing Postman. I’m afraid the wheel doesn’t meet our requirements.

  • Rest-assured

The second tool we explored is restful. Although it encapsulates a pretty neat DSL for API testing related operations, there is no problem with things like database validation because it is directly Java code. But writing reST-Assured automated tests requires a certain level of coding, which, unfortunately, is not a wheel we can use.

  • Cucumber

Cucumber allows you to write automated tests in natural language. Natural language calls code. If you package Cucumber well enough, you can write automated tests without writing code. There needs to be enough general-purpose packaging, so Pandaria is a DSL for automated testing of HTTP(S) APIS based on cucumber JVM encapsulation. Cucumber is very useful and you can use pandaria to access all the functionality of Cucumber JVM.

Pandaria

In the context of the test team’s lack of coding ability and the need to validate the database, we tried to encapsulate cucumber to enable testers who could not write code to write automated tests. We were surprised to see how effective this approach was, so we shared this tool in the hope that it could help other teams with similar needs.

Test the HTTP API

  Scenario: simple get
    * uri: /users/me
    * send: GET
    * status: 200
    * verify: '$.username'='jakim'
    * verify: '$.age'= 18Copy the code

Using Pandaria to write API automation tests, like the one above, just use abstracted keywords, describe the process of sending the request, and write your validation criteria.

Jakim = jakim; jakim = jakim; jakim = jakim; jakim = jakim; jakim = jakim

Validate database

* query:
"""
SELECT NAME, AGE FROM USERS
"""
* verify: '$[0].name'='jakim'
* verify: '$[0].age'= 18Copy the code

or

* query: select.sql
* verify: '$[0].name'='jakim'
* verify: '$[0].age'= 18Copy the code

select.sql

SELECT NAME, AGE FROM USERS
Copy the code

Testers can validate database content just as they would json by writing SQL. The above code uses the SQL statement SELECT NAME, AGE FROM USERS to query into the database and validates the NAME and AGE attributes on line 1 of the returned result.

To prepare test data, run the SQL file directly:

* execute sql: prepare_users.sql
* execute sql:
"""
insert into users(name) values('test');
"""
Copy the code

Waiting for the function

Automated tests often need to wait for something to complete, especially in asynchronous operations, where it is common to wait a certain amount of time, validate the results, and if not, retry a certain number of times until validation succeeds or fails beyond the maximum number of times. With Pandaria you can write:

* wait: 1000ms times: 3
* uri: /sequence
* send: GET
* response body:
"""3"""
Copy the code

The above code will send the request to /sequence and verify that the returned message body is equal to 3. If it is equal to 3, continue. If it fails, wait 1000ms and retry.

You can also wait for data in the database to meet a condition

* wait: 1000ms times: 3
* query: select.sql
* verify: '$[0].name'='jakim'
* verify: '$[0].age'= 18Copy the code

variable

Most of the time, the path of the uri of the restful API contains the ID of the self-growing database, which may be generated by an asynchronous operation. As a result, we cannot get the ID directly from the API return result, we can only find the ID from the database according to the condition of the test data, and use it in the subsequent API test.

* query:
"""
select id from users where name='test-user-name';
"""
* var: 'auto_generated_id'<-'$[0].id'

* uri: /users/${auto_generated_id}
* send: GET
* verify: '$.id'=${auto_generated_id}
* verify: '$.name'='test-user-name'
Copy the code

The code first looks up the self-growing ID from the database, then uses the <- operator to define the id in the result as a variable named auto_generated_id, and uses this variable in subsequent operations.

conclusion

Pandaria is still under development, and we’ve found that it’s already bringing tangible productivity gains to our team, so we took this opportunity to share it in the hope that it can help other teams like us.

  • The project address
  • More usage