Document-driven interface testing tools

Quick start

Install the HTTE command line

npm i htte-cli -g
Copy the code

Write the test

Write the configuration config.yaml

modules:
- echoCopy the code

Write test modules/echo. Yaml

- describe: echo get req: url: https://postman-echo.com/get query: foo1: bar1 foo2: bar2 res: body: args: foo1: bar1 foo2: bar2 headers: ! @exist object url: https://postman-echo.com/get?foo1=bar1&foo2=bar2 - describe: echo post req: url: https://postman-echo.com/post method: post body: foo1: bar1 foo2: bar2 res: body: ! @object json: foo1: bar1 foo2: bar2Copy the code

Run the test

htte config.yaml
Copy the code

✔  echo get (1s)
✔  echo post (1s)

2 passed (2s)
Copy the code

The original

Why do interfaces need to be tested?

• Improve service quality and reduce bugs
• Locate bugs earlier, saving debugging and processing time
• Easier code changes and refactoring
• Tests are also documentation that helps familiarize you with service functionality and logic
• Service acceptance criteria

There are many projects that do not have interface testing because testing is difficult because:

• Writing tests doubles the work
• There is a learning cost to writing test code
• Data coupling between interfaces makes testing difficult to write
• Constructing request data and verifying response data can be tedious
• Test code is code, and not spending time optimizing iterations is corrupt

Is there a strategy to maximize the benefits of testing while minimizing its costs?

The answer is document-driven.

Documentation describes the tests and implements the documentation with tools.

This is the original intention of HTTE.

Document-driven benefits

It is easier to read

With an interface like this: the service address is http://localhost:3000/add, USES the POST and use json as a data exchange format, the request data format {a: number, b: number}, {c: return data format Number}, which implements the sum of a and b and returns c.

For this interface, test the idea: pass data to this interface {” A “:3,”b”:4} and expect it to return {“c”:7}.

This test is written as a document in the HTTE.

- the describe: two together the req: url: http://localhost:3000/add method: post headers: the content-type: application/json body: a: 3 b: 4 res: body: c: 7Copy the code

A complete list of requests and responses, with a description of what the test does, is written.

It is easier to read

- Describe: login name: fooLogin req: URL: /login method: post body: email: [email protected] password: '123456' res: body: token: ! @exist String-describe: change nickname req: URL: /user method: put Authorization:! $conat [Bearer, ' ', !$query fooLogin.res.body.token] body: nickname: bar res: body: msg: okCopy the code

Although you may not understand it yet! @exist, ! $concat, ! $query, but you should have a rough idea of the functionality of the two interfaces and the format of the request response data.

Because the test logic is documented, HTTE easily gains some of the benefits that other frameworks can only dream of:

Programming language independence

There is no need for the CARE backend to be implemented in that language, and no need to worry about switching from one language to another, let alone from one framework to another.

Low skill requirements, quick hands

High efficiency, fast development

Easy to write, easy to read, low skill requirements, of course, writing fast. Finally can freely enjoy the advantages of testing, but also the biggest test to avoid trouble.

Naturally suited to test-driven development

As a front-end interface instruction

What if you have a Swagger/Blueprint document but still can’t use the interface? Throw him the test file, full of examples.

As a back-end requirements document/development guidance document

HTTE advantages

HTTE has the following advantages in addition to all the advantages of document-driven testing.

Use the YAML language

Use plug-ins to generate request validation responses flexibly

Let’s start with why you need plug-ins for document-driven testing.

An interface has duplicate name detection, so when testing we need to generate a random string, how to document the random number? An interface returns an expiration date. We need to verify that the expiration date is 24 hours after the current time. How do we define the expiration date in the documentation?

One of the biggest obstacles to document-driven testing is the lack of flexibility and complexity of documentation to describe probabilities such as random strings and current times. Only functions can provide this flexibility. Plug-ins provide this flexibility by providing functions for documents.

The plug-in is presented as a YAML custom tag.

There’s a code that looks like this

req: body: ! $concat [a, b, c] res: body: ! @regexp \w{3}Copy the code

! $concat and! @regexp is the YAML tag, which is a user-defined data type. In HTTE, it’s just functions. So the above code looks like this to HTTE.

{ req: { body: function(ctx) { return (function(literal) { return literal.join(''); })(['a', 'b', 'c']) } } res: { body: function(ctx, actual) { (function(literal) { let re = new Regexp(literal); if (! Re. The test (actual)) {CTX. Throw (' does not match the regular ')}}) (' \ w {3})}}}Copy the code

In general, the document has the advantages of being easy to read and write, but it cannot bear complex logic and is not flexible enough. Functions/code provide this flexibility, but with too much complexity. HTTE’s use of the YAML format, which encapsulates functions in YAML tags, reconciles this contradiction, maximizing each other’s strengths and almost avoiding each other’s weaknesses. This is HTTE’s biggest innovation.

There are two main operations on data in interface testing, constructing a request and validating a response. So there are two types of plug-ins in HTTE.

• Constructor (resolver), used to construct data, tag prefix! $
• Differ, used to compare parity data, label prefix! @

The plug-in set:

• Builtin – Contains some basic commonly used plug-ins

Componentized, easy to expand

- describe: req: body: v: ! $randnum [3, 6] res: body: v: ! @compare op: gt value: 2Copy the code

{
  req: { // Literal Req
    body: {
      v: function(ctx) {
        return (function(literal) {
          let [min, max] = literal
          return Math.random() * (max - min) + min;
        })([3, 6])
      }
    }
  },
  res: { // Expect Res
    body: {
      v: function(ctx, actual) {
        (function(literal) {
          let { op, value } = literal
          if (op === 'gt') fn = (v1, v2) => v1 > v2;
          if (fn(actual, literal)) return;
          ctx.throw('test fail');
        })({op: 'gt', value: 2})
      }
    }
  }
}Copy the code

req: { // Resolved Req body: { v: 5; }}Copy the code

res: { // Decoded Res body: { v: 5; }}Copy the code

(function(ctx, actual) {
    (function(literal) {
      let { op, value } = literal
      if (op === 'gt') fn = (v1, v2) => v1 > v2;
      if (fn(actual, literal)) return;
      ctx.throw('test fail');
    })({op: 'gt', value: 2})
  }
})(ctx, 5)Copy the code

The interface protocol is extensible and currently supports HTTP/GRPC

The interface protocol is provided by the client extension.

• Htte – Suitable for TESTING HTTP interfaces
• GRPC – Suitable for GRPC interface testing

Report generator is extensible and currently supports CLI/HTML

Elegant solution to interface data coupling

There is coupling of data between interfaces. For example, you can log in and get the TOKEN before you have the permission to place an order or post it in moments.

So one interface test often needs to access another test’s data.

HTTE handles this problem through sessions + plug-ins.

I’m going to give you an example.

There is a login interface, and it looks like this.

-describe: Tom login name: tomLogin # <-- Register a name for the test, Why? req: body: email: [email protected] password: tom... res: body: token: ! @exist stringCopy the code

There is an interface to change the user name. It is the permission interface and must have an Authorization request header and a token returned from login to use it.

- describe: tom update username to tem req: headers: Authorization: ! $conat [Bearer, ' ', token?] How to become a TOKEN? body: username: temCopy the code

Your answer

Authorization: ! $conat [Bearer, ' ', !$query tomLogin.res.body.token]Copy the code

Can also through tomLogin. The req. Body. Email mailbox value, through tomLogin. The req. Body. The password for the password. Is it elegant?

How does this work?

After the Runner starts in HTTE, the session is initialized. After each unit test is executed, the execution results are recorded in the session, including request data, response data, elapsed time, test results, and so on. This data is read-only and is exposed to the plug-in function as CTX, so the plug-in can access the data from the tests it previously performed.

In the same test, data from the REQ can also be referenced in the RES.

- describe: res ref data in req req: body: ! $randstr res: body: ! @query req.bodyCopy the code

Use macros to reduce repetitive writing

- decribe: add api condition 1
  req:
    url: /add
    method: put
    type: json
    body: v1...
- decribe: add api condition 2
  req:
    url: /add
    method: put
    type: json
    body: v2...Copy the code

Req: url: /add method: put type: jsonCopy the code

Describe: add API with macro includes: add #Copy the code

Develop while debugging

This feature is implemented by combining command-line options. The two command line options are: — Bail stops execution if any tests fail; –continue Tests continue where they were last interrupted.

Combining these two options allows us to reset the interface that executes the problem numerous times until debugging passes.

configuration

modules:
- auth
- user/orderCopy the code

session: mysession.json
modules:
- auth
- user/order
clients:
- name: http
  pkg: htte-client-http
  options:
    baseUrl: http://example.com/api
    timeout: 1000
reporters:
- name: cli
  pkg: htte-reporter-cli
  options:
    slow: 1000
plugins:
- name: ''
  pkg: htte-plugin-builtin
defines:
  login:
    req:
      method: post
      url: /users/loginCopy the code

- op: replace
  path: /clients/0/options/baseUrl
  value: https://example.com/apiCopy the code

Test units/groups

The test unit is the basic unit of the HTTE.

• describe: Describe the purpose of the test
• name: Defines the test name for convenience! $query 和 ! @queryReference, can be omitted
• client: defines the use of client extensions, which can be omitted if the project has only one client
• includes: reference macro, can reference more than one
• metadata: meta label, dedicated data for HTTE engine
  • skip: Whether to skip this test
  • debug: True indicates details of the request and response data printed at the time of the report
  • stop: True: Terminates subsequent operations after the test
• req: Request to view the corresponding client extension document
• res: In response, refer to the corresponding client extension document

Sometimes a functional test requires multiple test units to complete. To express this composition/hierarchy, HTTE introduces the concept of groups.

• describe: Describes the purpose of the test
• defines: defines intra-group macros with the same syntax as global macros in the configuration
• units: Group element

- descirbe: group defines: token: req: headers: Authorization: ! $conat [Bearer, ' ', !$query u1.req.body.token] units: - describe: sub group units: - descirbe: unit name: u1 metadata: debug: true includes: login req: body: username: foo passwold: p123456 res: body: token: ! $exist - descibe: unit includes: [updateUser, token] req: body: username: barCopy the code

license