Practice of automatic test scheme for Linglong project

Share content and technology stack

This paper will share the practical experience of testing workflow based on the situation of Linglong project, as well as the unit test and integration test of tool method and interface for Node.js server application. Practical experience will give you:

1. Use Jest to build a development-experience friendly test workflow.
2. Write an efficient unit test case and integration test case.
3. Encapsulation technology is used to separate modules and simplify test code.
4. Use SuperTest to merge application and test processes.
5. Create efficient database memory services to implement isolated test suite running mechanisms.
6. Learn about the use of features like mocks, snapshots, and test coverage.
7. Understand TDD and BDD.
8. .

The basic stack of technologies covered in this article are (knowledge to know) :

1. TypeScript: A superset of the JavaScript language with a type system and new ES syntax support.
2. SuperTest: HTTP proxy and assertion tool.
3. MongoDB: NoSQL distributed file storage database.
4. Mongoose: MongoDB Object Relational Mapping Operation Library (ORM).
5. Koa: Basic Web application framework.
6. Jest: a rich JavaScript testing framework.
7. Lodash: JavaScript tool function library.

About the current which

Linglong is jd’s intelligent design platform, providing online design services, mainly including:

Picture design: online and offline design services, such as quick synthesis of advertising pictures, main pictures, public accounts with pictures, posters, leaflets, logistics sheets and so on.

Video design: fast synthesis of main picture video, Douyin short video, custom video and other design services.

Page design: quickly build activity page, marketing page, small games, small programs and other design services.

Practical tools: batch matting, size, color, watermarking and so on.

Based on industry-leading technology, to provide merchants, users with rich design capabilities, to achieve rapid output.

Design and test frame selection

Firstly, I will introduce the structure of Linglong project for the convenience of subsequent description and understanding. Linglong project adopts the mechanism of front and back end separation, and uses the React Family infrastructure at the front end, plus the Next. Js server rendering to provide better user experience and SEO ranking. Backend architecture is shown in the figure below, the process is probably the browser or third-party applications access project Nginx cluster, Nginx through load balancing cluster forwarded to current which the application server, application server via docking external or internal service, or read/write caching, database, logical processing after the return to the front right data over HTTP.

Comparison of major testing frameworks

Next, according to the requirements of the project, we compare the current mainstream node.js testing framework.

	Jest	Mocha	AVA	Jasmine
GitHub Stars	28.5 K.	18.7 K.	17.1 K.	14.6 K.
GitHub Used by	1.5 M	926K	46.6 K.	5.3 K.
Document friendly	good	good	good	good
Mock	support	The outer	The outer	The outer
Snapshot Function	support	The outer	support	The outer
Support the TypeScript	ts-jest	ts-mocha	ts-node	jasmine-ts
Verbose error output	support	support	support	The unknown
Support parallel and serial	support	The outer	support	The outer
Each test process is isolated	support	Does not support	support	The unknown

* Documentation friendly: The documentation structure is well organized, the API is well explained, and examples are abundant.

Analysis:

1. Mocha GitHub’s high usage is probably due to its early appearance (2011) and development led by TOP Node.js developer TJ (later switched to Go), so early projects chose Mocha as the testing framework. Jest and AVA are up-and-comers (2014), and the number of Stars is increasing. It is expected that new projects will be selected from these two frameworks.
2. Built-in support is likely to integrate better with the framework, be more conceptual, be maintained more frequently, and be easier to use than external functionality.
3. Jest mocks can implement method mocks, timer mocks, and module/file dependent mocks. Mock Modules are often used in actual test case writing to ensure that test cases respond quickly and are not volatile. Here’s how to use it and why.

To sum up, we chose Jest as the basic testing framework.

Practice from zero to one

Jest framework configuration

Next, we started from zero to one. First, we set up the test flow. Although Jest can be used out of the box, the project architecture is different, and most of the time, we need to do some basic configuration work according to the actual situation. The following is the simplified project directory structure extracted from linglong project, as follows.

├ ─ dist# TS compile the result directory├ ─ SRC# TS source directory│ ├ ─ app. Ts# App main file, similar to Express framework /app.js file│ └ ─ index. Ts# Application startup file, similar to Express framework /bin/ WWW file├ ─test                 # test file directory│ ├ ─ @ fixtures# Test fixed data│ ├ ─ @ helpers# Test tool method│ ├ ─ module1Set of test suites for Module 1│ │ └ ─test-suite.ts       Test suite, a class of test case collection│ └ ─ module2Test suite collection for Module 2├ ─ package. Json └ ─ yarn. The lockCopy the code

Here are two little dots:

1. In order to@The first directory is defined as a special file directory, which provides some test AIDS methods, configuration files, etc., while the other directories at the same level are the directories where test cases reside, divided by business modules or functions. In order to@The beginning can be clearly displayed at the top of the same directory, easy to develop location, coincidentally also convenient to write regular matches.
2. test-suite.tsIs the minimum test file unit in the project, which is called test suite. It represents a set of test cases of the same class, which can be multiple test cases of a general function or a series of unit test cases.

First, install the test framework.

yarn add --dev jest ts-jest @types/jest
Copy the code

Since the project is written in TypeScript, ts-jest @types/jest is also installed here. Then create the jest.config.js configuration file in the root directory and do the following minor configurations.

module.exports = {
  // preset: 'ts-jest',
  globals: {
    'ts-jest': {
      tsConfig: 'tsconfig.test.json',}},testEnvironment: 'node'.roots: ['<rootDir>/src/'.'<rootDir>/test/'].testMatch: ['<rootDir>/test/**/*.ts'].testPathIgnorePatterns: ['<rootDir>/test/@.+/'].moduleNameMapper: {
    '^ ~ / (. *)': '<rootDir>/src/$1',}}Copy the code

preset: If you want to set ts-jest to tsconfig.test.json, you need to mount ts-jest to globals. More configurations can be found in its official documentation, here.

TestEnvironment: set the testEnvironment by default. Node.js needs to be set to Node because the default value is jsdom for the browser environment.

Roots: Used to set the directory of the test listening. If the files in the matching directory are changed, the test case will be automatically run.

represents the project root directory, which is the same level as package.json. Here we listen for the SRC and test directories.

TestMatch: Glob mode sets the matched test files, which can also be the regular mode. Here, we match all files in the test directory, and the matched files are executed as test cases.

TestPathIgnorePatterns: Sets files that have already been matched but need to be ignored. Here we set directories that start with @ and all of their files not to be used as test cases.

ModuleNameMapper: Similar to TS Paths and Webpack Alias, moduleNameMapper is used to set directory aliases to reduce error rates when referring to files and improve development efficiency. Here we set the module name starting with ~ to point to the SRC directory.

The first unit test case

After setting up the test running environment, we can begin to write test cases. Let’s write an interface unit test case, for example, to test the correctness of the home page round-robin diagram interface. We will test cases in the test/homepage/carousel. Ts file, the code is as follows.

Import {forEach, isArray} from 'lodash 'import {JFSRegex, URLRegex } from '~/utils/regex' import request from 'request-promise' const baseUrl = 'http://ling-dev.jd.com/server/api' // declare a test case test(' The number of multicast graphs should return 5, And the data is correct ', Async () => {// send an HTTP request to an interface const res = await request.get(baseUrl + '/carousel/pictures') // Check return status code 200 Expect (res.statusCode).tobe (200) // The returned data is an array and length is 5 expect(isArray(res.body)).tobe (true) ForEach (res.body, href) expect(res.body.length).tobe (5) // picture => { expect(picture).toMatchObject({ url: expect.stringMatching(JFSRegex), href: expect.stringMatching(URLRegex), }) }) })Copy the code

After writing the test case, the first step is to start the application server:

The second step is to run the test. In the command line window, enter: NPX jest.

Of course, the best practice is to wrap the command in package.json, as follows:

{
  "scripts": {
    "test": "jest"."test:watch": "jest --watch",}}Copy the code

After that, you can use YARN test to run the test and run yarn test:watch to start the listening test service.

SuperTest enhancement

Although the basic test process has been developed, it is clear that every time we run the test, we need to start the application service. There are two processes that need to be started and we need to configure ling-dev.jd.com to point to 127.0.0.1:3800 in advance. So we introduced SuperTest, which can integrate the application service into the test service and start without specifying the host address of the HTTP request.

We wrap a common request method in a file called @helpers/ agency.ts as follows.

import http from 'http'
import supertest from 'supertest'
import app from '~/app'

export const request = supertest(http.createServer(app.callback()))
Copy the code

Explanation:

1. useapp.callback()Rather thanapp.listen()Because it can combine the sameappAs both HTTP and HTTPS or multiple addresses.app.callback()Return applies tohttp.createServer()Method to process the request.
2. After that,http.createServer()Create an unlistened HTTP object for SuperTest, which will also be called internallylisten(0)Such special ports allow the operating system to provide random ports available to start the application server.

So we can rewrite the above test case like this:

Import {forEach, isArray} from 'lodash 'import {JFSRegex, URLRegex} from '~/utils/regex' // import {request} from '.. /@helpers/agent' test ' Async () => {const res = await request.get('/ API /carousel/pictures') expect(res.status).tobe (200) // same check... })Copy the code

Because SuperTest has already wrapped the host address and automatically started the application service, we only need to write the specific interface when requesting the interface, such as/API /carousel/pictures, and also need to run the yarn test command to complete the test.

Database memory service

In the project architecture, you can see that the database is using MongoDB, and during the test, almost all the interfaces need to connect to the database. At this point, you can use environment variables to distinguish and create a test database for running test cases. The downside is that the test database needs to be flushed after the test suite is finished to avoid dirty data affecting the next test suite, especially when running concurrently, data isolation needs to be maintained.

A better option is to use MongoDB Memory Server, which will start a separate MongoDB instance (with a very low Memory footprint of about 7MB each) in which the test suite will run. If concurrency is 3, create 3 instances to run 3 test suites. This keeps the data isolated and in memory, which makes it very fast to run and destroys the instance automatically when the test suite is complete.

Next we will introduce MongoDB Memory Server into the actual test. The best way is to write it into the Jest environment configuration so that it is written once and runs automatically in each test suite. Therefore, replace the testEnvironment of the jest. Config. js configuration file with a user-defined environment

/test/@helpers/jest-env.js.

@helpers/jest-env.js

const NodeEnvironment = require('jest-environment-node')
const { MongoMemoryServer } = require('mongodb-memory-server')
const child_process = require('child_process')

// Inherit the Node environment
class CustomEnvironment extends NodeEnvironment {
  // Before the test suite starts, get the local development MongoDB Uri and inject the global object
  async setup() {
    const uri = await getMongoUri()
    this.global.testConfig = {
      mongo: { uri },
    }
    await super.setup()
  }
}

async function getMongoUri() {
  // Run the which mongod command to get the local MongoDB binary path
  const mongodPath = await new Promise((resolve, reject) = > {
    child_process.exec(
      'which mongod',
      { encoding: 'utf8' },
      (err, stdout, stderr) => {
        if (err || stderr) {
          return reject(
            new Error('Could not find system mongod, please make sure' which mongod' points to Mongod ')
          )
        }
        resolve(stdout.trim())
      }
    )
  })

  // Create a memory service instance using the local MongoDB binary
  const mongod = new MongoMemoryServer({
    binary: { systemBinary: mongodPath },
  })
  
  // Get the Uri address of the successfully created instance
  const uri = await mongod.getConnectionString()
  return uri
}

// Export the custom environment class
module.exports = CustomEnvironment
Copy the code

Mongoose can be connected like this:

await mongoose.connect((global as any).testConfig.mongo.uri, {
  useNewUrlParser: true.useUnifiedTopology: true,})Copy the code

Of course, in package.json, you need to disable MongoDB Memory Server to download binary packages, because local binary packages are already used.

"Config ": {"mongodbMemoryServer": {"version": "4.0", // disablePostinstall": "1", "md5Check": "1" } }Copy the code

Encapsulation and use of the login function

Most of the time the interface needs to be logged in to access it, so we need to pull out the whole logon function and encapsulate it into a common method, while initializing some test specific data.

To make the API easy to use, I want the login API to look like this:

import { login } from '.. /@helpers/login'

// Invoke the login method to create a user based on the passed role and return the Request object to which the user is logged in.
// Support multiple parameters, according to different parameters automatically initialized test data.
const request = await login({
  role: 'user',})// Use the logged-in request object to access the logged-in user interface,
// It should be in logon state and return the correct information about the currently logged user.
const res = await request.get('/api/user/info')
Copy the code

Development login method:

// @helpers/agent.ts 
// Add a new makeAgent method
export function makeAgent() {
  // Use supertest.agent to support cookie persistence
  return supertest.agent(http.createServer(app.callback()))
}
Copy the code

// @helpers/login.ts
import { assign, cloneDeep, pick } from 'lodash'
import { makeAgent } from './agent'

export async function login(userData: UserDataType) :Promise<RequestAgent> {
  userData = cloneDeep(userData)
  
  // If there is no user name, the user name is automatically created
  if(! userData.username) { userData.username = chance.word({length: 8})}// Create a nickname automatically if there is no nickname
  if(! userData.nickname) { userData.nickname = chance.word({length: 8})}// Get a request object that supports cookie persistence
  const request: any = makeAgent()

  // Send a login request. Here, a login interface is designed specifically for the test
  // Contains normal login functions, but also initializes test specific data according to different parameters
  const res = await request.post('/api/login-test').send(userData)

  // Assign the data returned by the login to the Request object
  assign(request, pick(res.body, ['user'.'otherValidKey... ']))

  // Return the Request object
  return request as RequestAgent
}
Copy the code

Use it in the actual use case as in the example above.

Use of simulation function

You can see from the project architecture that the project also calls more external services. For example, to create an interface for a folder, the internal code needs to call an external service to identify whether the folder name contains sensitive words, like this:

import { detectText } from '~/utils/detect'

// Call an external service to check if the folder name contains sensitive words
const { ok, sensitiveWords } = await detectText(folderName)
if(! ok) {throw new Error('Sensitive words detected:${sensitiveWords}`)}Copy the code

In actual testing, it is not necessary for all test cases to call external services at run time, which can slow down the response time and instability of test cases. We can set up a better mechanism, create a test suite specifically for verifying the correctness of the detectText tool method, while the other test suite runtime detectText method just returns OK, so that not only can ensure that the detectText method is verified, It also ensures fast response to other test suites.

Mocks were created for situations like this. We just need to create a __mocks__/detect.ts simulation file at the same level as the utils/detect.ts path of the detectText method.

export async function detectText(
  text: string
) :Promise<{ ok: boolean; sensitive: boolean; sensitiveWords? : string }> {// Delete all code and return OK
  return { ok: true.sensitive: false}}Copy the code

Then add the following code to the top of each test suite you want to emulate.

jest.mock('~/utils/detect.ts')
Copy the code

In the test suite that validates the detectText tool method, you just need jest. Unmock to recover the real method.

jest.unmock('~/utils/detect.ts')
Copy the code

Mock should be included in the setupFiles configuration, since most of the test suites that need to be simulated will automatically load the file before they run. This way, you don’t have to add the same code to every test suite, which can improve development efficiency.

// jest.config.js
setupFiles: ['<rootDir>/test/@helpers/jest-setup.ts']
Copy the code

// @helpers/jest-setup.ts
jest.mock('~/utils/detect.ts')
Copy the code

Simulation features also include method simulation, timer simulation, etc. You can consult their documentation for more examples.

Using the Snapshot Function

The Snapshot feature simplifies test cases by helping us test large objects.

Projects, for example, the template parsing interface, the interface will be PSD template file parsing, and then spit out a larger JSON data, if each check attribute of an object is properly may is not very ideal, so you can use the snapshot function, is to run the test case for the first time, the JSON data stored in the local file, It is called a snapshot file. The data returned in the second run is compared with the snapshot file. If the two snapshots match, the test succeeds; otherwise, the test fails.

The way to use it is simple:

// Request template parsing interface
const res = await request.post('/api/secret/parser')

// Asserts whether the snapshot matches
expect(res.body).toMatchSnapshot()
Copy the code

Updating snapshots is also agile, run the command jest –updateSnapshot or type U in listening mode to update.

Integration testing

The concept of integration test is to test all modules in series according to certain requirements or process relations on the basis of unit test. For example, although some modules can work independently, they cannot be guaranteed to work properly when connected together. Some problems that cannot be reflected locally are likely to be exposed globally.

Since the test framework Jest runs in parallel for each test suite, and the use cases within the suite run in serial, it is easy to write integration tests. Here is how to write integration tests using the folder usage flow example.

import { request } from '.. /@helpers/agent'
import { login } from '.. /@helpers/login'

const urlCreateFolder = '/api/secret/folder'      // POST
const urlFolderDetails = '/api/secret/folder'     // GET
const urlFetchFolders = '/api/secret/folders'     // GET
const urlDeleteFolder = '/api/secret/folder'      // DELETE
const urlRenameFolder = '/api/secret/folder/rename'     // PUT

const folders: ObjectAny[] = []
let globalReq: ObjectAny

test('No permission to create folder should return 403 error'.async() = > {const res = await request.post(urlCreateFolder).send({
    name: 'My Folder',
  })
  expect(res.status).toBe(403)
})

test('Make sure to create 3 folders'.async() = > {// Log in to users who have permission to create folders, such as designers
  globalReq = await login({ role: 'designer' })
  
  for (let i = 0; i < 3; i++) {
    const res = await globalReq.post(urlCreateFolder).send({
      name: 'My Folder' + i,
    })
    
    // Place the successfully created folder in the folders constant
    folders.push(res.body)
    
    expect(res.status).toBe(200)
    // More validation rules...
  }
})

test('Rename folder 2'.async() = > {const res = await globalReq.put(urlRenameFolder).send({
    id: folders[1].id,
    name: 'New folder name',
  })
  expect(res.status).toBe(200)
})

test('The name of the second folder should be [new folder name]'.async() = > {const res = await globalReq.get(urlFolderDetails).query({
    id: folders[1].id,
  })
  expect(res.status).toBe(200)
  expect(res.body.name).toBe('New folder name')
  // More validation rules...
})

test('Getting folder list should return 3 pieces of data'.async() = > {// Same as above, because the code is too much, first omit...
})

test('Delete last folder'.async() = > {// Same as above, because the code is too much, first omit...
})

test('Retrieving the folder list again should return 2 pieces of data'.async() = > {// Same as above, because the code is too much, first omit...
})
Copy the code

Test coverage

Test coverage is a measure of test completion that gives feedback on the quality of tests based on how well the files are being tested.

Run the jest –coverage command to generate the test coverage report. Open the generated coverage/ lcoV-report /index.html file, and you can see all the indicators in a list. Because Jest uses Istanbul internally to generate coverage reports, the metrics still refer to Istanbul.

Continuous integration

After writing so many test cases, or after the development of the function code, we hope every time code will be pushed to the hosting platform, such as GitLab, hosting platform can help us to run all test cases automatically, if the test failed email notification we repair, if the test through the combine development branch to the main branch?

The answer is absolutely necessary. This coincides with Continuous Integration, which, in layman’s terms, frequently merges code into trunk branches, with automated tests running before each merge to verify that the code is correct.

So we configure some automated test tasks to execute the install, compile, test, and so on in sequence, and the test command is to run the written test case. A GitLab configuration task (.gitlab-ci.yml) might look like this, for reference only.

The command executed before each job
before_script:
  - echo "`whoami` ($0 $SHELL)"
  - echo "`which node` (`node -v`)"
  - echo $CI_PROJECT_DIR

Define the test phase of the job and the command to execute
test:
  stage: test
  except:
    - test
  cache:
    paths:
      - node_modules/
  script:
    - yarn
    - yarn lint
    - yarn test

# Define the deploy phase to which the job belongs and the commands to be executed
deploy-stage:
  stage: deploy
  only:
    - test
  script:
    - cd /app 
    - make BRANCH=origin/${CI_COMMIT_REF_NAME} deploy-stage
Copy the code

Benefits of continuous integration:

1. Find errors quickly.
2. Prevent branches from straying too far from the main branch.
3. Allowing products to iterate quickly while still maintaining high quality.

TDD and BDD are introduced

TDD stands for Test-driven Development. It is a design methodology in agile development that emphasizes the transformation of requirements into concrete Test cases before developing code to make tests pass.

BDD stands for behavior-driven Development (BEHAVIor-driven Development), which is also an agile development design methodology. It does not emphasize the specific form, but emphasizes the argument that the Behavior is specified by the user story as what role, what function is desired, and what benefit is gained.

Both are very good development pattern, combined with the actual situation, our test is more like a BDD, but has not completely abandon the TDD, our suggestion is that if you feel to write tests first can help write code faster, then you have to write tests first, if feel to write code to write test, or better development and testing, have their own way, As a result, both coding functions and test cases need to be completed and passed. Finally, Code quality is further reviewed and controlled through Code Review.

The author calls it “learning from others and developing skills, focusing on oneself” : combining the actual situation of the project and being flexible, a set of pattern-driven development suitable for the development of the project can be formed.

conclusion

Automated testing provides a guaranteed mechanism to inspect the entire system, and regression testing can be performed frequently, effectively improving system stability. Of course, there are costs involved in writing and maintaining test cases, and the balance between input and output benefits needs to be considered.

---

Welcome to the bump Lab blog: AOtu.io

Or pay attention to the bump Laboratory public account (AOTULabs), push the article from time to time: