Postman API automated testing

Postman’s most basic function is to replay requests, with a nice response formatting tool.

Advanced point usage can use Postman to generate scripts for various languages, but also can capture packets, authentication, file transfer.

This alone is not enough for a system to develop, or is too trivial, and you still need to frequently switch back and forth between development, test, and production. A single request is not enough either; you need to maintain requests from all of your system’s apis, each with a different QueryString and body.

Collection

Collection is used when all requests on the server side are organized by function or business module and markdown is used to add appropriate descriptions of all requests and examples. Here are some of Postman’s terms and suggestions for organization requests.

See Postman SDK Concepts and Creating Collections for details

  • A Collection corresponds to an Application, and each member of the group (server, Client, and QA) shares a Collection. You can add tests, documents to the entire Collection. For applications that do not organize requests in Postman at the beginning, you can set the Proxy to run through the application and capture all requests of the application.
  • Folder (ItemGroup)

    Corresponding to a module, or hierarchy of child routes. Such asrouter.use('/users')All requests are in one Folder, and you can nest folders with each other based on the route.
  • Request (Item) corresponds to a Request, and authentication information can be added. You can also set the proxy to capture packets. For details on CAPTURING HTTP Requests.
  • Example corresponds to a request with different parameters and responses, used for Mock Servers and documents.

Postman can generate documents and Mock Servers based on the structure of a Collection. However, they are all paid features, the free version has a limited number of times.

The document

Postman automatically generates documentation to facilitate team collaboration, eliminating major bugs such as manual documentation and late updates.

For GET requests, a description of the field can be added to Postman to generate documentation.

For POST and PUT requests, description generation documents can be added if the Content-Type is form-data or X-www-form-urlencoded. However, it’s much easier to pass JSON, so there’s a lot of application/ JSON, and YOU can’t add comments to JSON. If you need to document json, you can add a redundant field _{key}. Comment to indicate comments

{" id ": 128," _id. The comment ":" id ", "page" : 10, "_page. The comment" : "pages" pageSize ":" 15, "_pageSize. The comment" : "article number per page"}Copy the code

However, there are too many redundant fields, so a better solution would be to perform JSON validation of the request in the test, while also acting as part of the document. After all, JSON-schema is designed to describe data and make it more readable.

Speaking of requests above, the documentation for the response can be jSON-schema validation or descriptions of each field, with more test cases representing more detail.

Mock

When the API is not written on the Server side, the client can generate the Mock Server using Examples.

It is recommended that the client side make its own Mock, integrate it with the project, and incorporate it into version control, which is convenient and flexible. Json-server is highly recommended, simple and easy to use.

test

There needs to be a test case for every Request. Verify that the response is successful, that the response time is too long, or that the data type of the response JSON is correct.

Tests can be performed using Pm. expect for BDD testing, similar in style to CHAI and easy to get started with if you are familiar with CHAI.

Postman comes with some third-party libraries built in, either directly if you prefer Chai, or using pm. Expect, which uses the UNDERLYING CHAI implementation, consistent with the CHAI BDD API.

Postman also has some HTTP-related testing apis, such as Status Code, Header, and Body, and also provides snippets.

Pm. test('Status code is 200', () = > {PM. The response. To. Have. The status (200)}) / / response success chai. Expect PM. The test (' the status code is 200 ', () => {chai. Expect (pm.response).to.have. Property ('code', 200)}) () => { const jsonData = pm.response.json() chai.expect(jsonData.page).to.eql(100) })Copy the code

Json Schema

Json-schema can be used to describe JSON information to make it easier to read and verify the validity of JSON. Major languages have libraries that implement JSON-Schema.

Json-schema validation is recommended for all GET responses. It can be used as data validation, and can also be used as documents, using TV4 to validate JSON

pm.test("User info", () => { const jsonData = pm.response.json() const schema = { title: 'UserInfo', discription: 'User info ', type: 'object', Required: ['age', 'email', 'name'], Properties: {age: {description: 'age', type: 'number', mininum: 0,}, email: {description: 'mailbox ', type: 'string'}, name: {description:' name ', type: 'string' } } } pm.expect(tv4.validate(jsonData, schema)).to.eql(true) })Copy the code

Json validation can also be added to a request, but it is more complicated because Postman does not directly provide an API to get all the request parameters and needs to parse and calculate them himself

// GET data from application/json const json = json.stringify (pm.request.body.raw) const qs = GET data from GET Query String pm.request.url.query.toObject()Copy the code

If only Postman could automatically generate data based on the JSON-schema of the request parameters…

  • reference

    • json-schema.org
    • tv4 Documentaion
    • chai bdd – API
    • postman sandbox api reference

Test request parameters

A request takes several parameters, such as queryString (search) for GET and body for POST, and each parameter has a different response.

Suppose a Request returns a completely different JSON schema for different parameters, then two requests can be tested separately. If you return the same JSON schema with different values, you need to consider what parameters are passed and how many.

A classic scenario is to filter the list of criteria based on filter. For an example of a user list, the pseudocode looks like this

const url = '/api/users' const query = { name: 'san', age: 12, sex: 'MALE'} // Note that query data needs to be verified, Const SQL = 'select * from users where name = ${query.name} and age = ${query.age} and sex = ${query.sex}Copy the code

One idea is to test against the parameters of the request. An important snipet is to get a QueryString in Postman, which is a PropertyList of data, Defined in postman-collection-propertyList. The following

const name = pm.request.url.query.get('name') const age = pm.request.url.query.get('age') if (name) { pm.test('Items should match the name', () => { const jsonData = pm.response.json() expect(_.uniq(jsonData.rows.map(row => row.name))).to.eql([name]) }) } // There's a lot of redundant code, Postman snipets if (age) {pm.test('Items should match the age', () => { const jsonData = pm.response.json() expect(_.uniq(jsonData.rows.map(row => row.age))).to.eql([age]) }) }Copy the code

Of course, the above filter only contains the simplest scenario, which involves only the equality test. But there is inequality and inclusion.

const query = {
  name: 'san',
  age: 12,
  sex: 'MALE'
}
const sql = `select * from users where name like ${query.name} and age < ${query.age} and sex = ${query.sex}`Copy the code

Such request parameters rely on negotiation between the front and back ends, which is certainly not friendly for testing or an uninformed developer.

Of course, it’s also unfriendly to the back end, because you need to process every query you pass in and manually change it every time you add a filter field.

It is up to the front end to decide what data to filter, for example using a retrieval syntax similar to Mongo.

graphqlIs pretty cool and worth a try 

const query = {
  name: {
    $like: 'san' 
  },
  age: {
    $lt: 12 
  },
  sex: 'MALE'
}Copy the code

However, this also requires a high level of test development capability, requiring testers to parse parameters and test interfaces.

Testing multiple requests

When unit testing a function requires a large number of inputs and expected outputs, data can be used to simulate multiple inputs in Postman

Data is a variable that can only be used in Runner. It is necessary to establish relevant data files for each Folder and add version control

  • using csv and json files in the postman collection runner

Integration testing

Once a single API test passes, all requests need to be integrated for testing. Two problems arise

  1. How do I ensure API dependencies
  2. How is data passed between apis

The order of requests in the Collection is the order in which they originated. If you want to force a change in order, use setNextRuest().

There are three types of scoped data in Postman: data, Environment, and global. Use {{}} placeholders instead in the request.

Environment can be used to change hosts to avoid frequent manual switching between local, development, and production environments in urls. It can also be used to pass data.

A common scenario is that a project uses a token to store login information, and each request needs to carry the token. The token’s environment variables can be set in the login test code

const url = 'http://{{HOST}}/api/login' pm.test('There is a token', () => { const jsonData = pm.response.json() pm.expect(jsonData.token).to.a('string') pm.environment.set('token', jsonData.token) }) const urlNext = 'http://{{HOST}}/api/profile? token={{token}}'Copy the code

The test Collection

Once dependencies are secured, you can create a Runner on the Collection and introduce a data file to test all requests. Runner and data can also be used to test local folders.

Postman already supports new variables and tests for each Postman

All requests have some common tests, such as testing whether the interface responded successfully and testing the filter mentioned above

pm.test('Response is right', () => {
  // status code: 2XX
  pm.response.to.be.success
})

pm.test('Filter is matching', () => {
  // ...
})Copy the code

Continuous integration

When a Collection can be tested, you need to add version control to the tests, integrate them with the project, and keep test records so that bugs can be located on time. It can be integrated with Newman, postman’s official tool, but with a slight inconvenience, continuous integration can only save records, not restore them.

newman run https://api.getpostman.com/collections/{{collection_uid}}?apikey={{postman-api-key-here}} --environment https://api.getpostman.com/environments/{{environment_uid}}?apikey={{postman-api-key-here}}Copy the code

Compare UI automation testing

As I understand it, the purpose of UI automation testing is to test whether the process is smooth, such as logging in, registering, logging out, and taking screenshots if the use case fails. But constantly changing front-end requirements, coupled with the variety of front-end frameworks available today, mean that selectors are not particularly easy to get and the process is easy to change.

API automated tests are used to test whether the data is correct. And most of the problems are in the data problem, so API automated testing is more cost-effective.

conclusion

  1. How do I write test cases

    Postman low-level use
    [chai.js](http://chaijs.com/api/bdd/)BDD syntax as an assertion library, plus some special syntax.

  2. How to debug

    Click the menu bar View -> Show Devtools (Show Postman Console) to View the response and check the output, but without breaking points. For a single request from the system, you can use Proxy to listen for the request for debugging.

  3. How to use javascript third-party libraries to preprocess and postprocess requests

    For example, when sending a request, the server requires the time to be in the format of Timestmap (Unix), but the interface debugging is too readable. Can you use moment to convert the time? When you receive a response, you also need moment to parse the time for a better presentation. Or use lodash functions for data processing.

    You can write scripts in Tests and pre-request scripts to handle requests and responses. But you can’t format data, such as dates.


    It is recommended to use ISO strings to communicate dates to the front and back ends, which are easy to parse and readable.

  4. How do I manage request dependencies

    For example, the two apis need to have a dependency relationship, such as when a user is created (registered), to obtain his personal information. To obtain personal information, you need to rely on the create user API.

    You can manage dependencies using Environment Variables

  5. How do I set uniform request parameters

    For example, most interfaces require a unified token parameter.

    Is there anything we can do about it

  6. How does it integrate into a server-side project

    If subsequent releases of the system do not pass API tests, it is important to keep test records so that version control is aware of code changes during that time period. In git’s case, you need to run the tests after each commit and keep the results.

    You can use the NPM package Newman to integrate into your project

reference

  • using variables inside postman and collection runner
  • writing tests in postman
  • postman-echo
  • generate spoitify playlists using a postman collection