This article is collected in GitHub mountain moon blog: shfshanyue/blog, including my problems encountered in practical work, thinking about business and learning in the direction of the full stack

  • Front-end engineering series
  • Node advanced series

Back-end programmers with a sense of humor refer to themselves as CURD Boy. CURD, or adding, deleting, modifying, or reviewing a storage resource, is all data-oriented programming.

That’s great. Data-oriented programming often leads to a better understanding of the business, resulting in higher quality code and fewer bugs. Since it is data-oriented programming, it is more necessary to avoid the appearance of dirty data and strengthen data verification. Otherwise, why trust the front-end data validation, which goes directly to the user, for more user-friendly feedback at the UI level?

Data check layer

The back-end is divided into various levels due to its emphasis on business logic and various data to be processed. The back-end projects I have experienced are divided into Controller, Service, Model, Helper, Entity and other named layers, which are varied. But there must be a layer called Controller, which stands at the top of the back end and receives data directly from the client.

As the Controller layer is the top layer of data interaction between the server side and the client side, it inherits the principle of Fail Fast and shoulders the function of data filter, and directly calls back illegal data, just like Qin Qiong and Yuchi Gong.

Data validation is also a semi-documentation spin-off. You only need to look at the data validation layer to know which fields to pass and in what format.

The following are common data validations, and this article explains how to validate them:

  1. required/optional
  2. Basic data verification, such as number, string, timestamp, and value requirements
  3. Complex data verification, such as IP, mobile phone number, email and domain name
const body = {
  id,
  name,
  mobilePhone,
  email
}
Copy the code

Yamatsuk was working on a back-end project with no data verification layer, and if/else layers were filled with pain, refactoring every minute.

JSON Schema

JSON Schema is a son-based data verification format with a json-schema.org specification currently available in release 7.0. Specifications are implemented in various server programming languages, such as Go, Java, PHP, and of course great javascript, such as the tepid AJV.

The following is a Schema for verifying user information, which shows that the syntax is complex and tedious:

{
  "$schema": "http://json-schema.org/draft-04/schema#"."title": "User"."description": "User Information"."type": "object"."properties": {
    "id": {
      "description": "User ID"."type": "integer"
    },
    "name": {
      "description": "User name"."type": "string"
    },
    "email": {
      "description": "User email"."type": "string"."format": "email"."maxLength": 20
    },
    "mobilePhone": {
      "description": "User's Mobile phone Number"."type": "string"."pattern": "^ (? : (? : \ | + 00) 86)? 1[3-9]\d{9}$"."maxLength": 15}},"required": ["id"."name"]}Copy the code

For complex data type verification, the following formats are built-in in JSON Schema for easy verification

  • Dates and times
  • Email addresses
  • Hostnames
  • IP Addresses
  • Resource identifiers
  • URI template
  • JSON Pointer
  • Regular Expressions

For phone numbers that are not in the built-in Format, you can manually add the Format using ajv.addFormat

ajv.addFormat('mobilePhone'.(str) = > / ^ (? : (? : \ | + 00) 86)? 1[3-9]\d{9}$/.test(str));
Copy the code

Joi

Joi claims to be the most powerful JS verification library, with 16, 000 stars on Github. Compared to JSON Schema, its syntax is more concise and powerful.

The most powerful data validation library for JS

Less code is required to do the same validation, and more powerful validation can be done. The following is only an example, please go to the documentation for more examples.

const schema = Joi.object({
  id: Joi.number().required(),
  name: Joi.number().required(),
  email: Joi.string().email({ minDomainSegments: 2.tlds: { allow: ['com'.'net']}}),mobilePhone: Joi.string().pattern(/ ^ (? : (? : \ | + 00) 86)? 1[3-9]\d{9}$/),

  password: Joi.string().pattern(/ ^ [a zA - Z0-9] {30} 3 $/),
  // Same validation as password
  repeatPassword: Joi.ref('password'})),// The password and the duplicate password must be sent at the same time
  .with('password'.'repeat_password');
  // Please provide one email address and one mobile phone number
  .xor('email'.'mobilePhone')
Copy the code

Data verification and routing layer integration

Because the data is passed directly from the route, koAJS officially implements a Joi-Router based on JOI, which verifies the data to the routing layer and verifies the Query, body, and Params passed from the front end.

The Joi-router also parses and restricts the various content-Types transmitted by the front-end based on the co-body. For example, CSRF attacks can be prevented to a certain extent if the value is set to Application/JSON.

const router = require('koa-joi-router');
const public = router();

public.route({
  method: 'post'.path: '/signup'.validate: {
    header: joiObject,
    query: joiObject,
    params: joiObject,
    body: joiObject,
    maxBody: '64kb'.output: { '400-600': { body: joiObject } },
    type: 'json'.failure: 400.continueOnError: false
  },
  pre: async (ctx, next) => {
    await checkAuth(ctx);
    return next();
  },
  handler: async (ctx) => {
    await createUser(ctx.request.body);
    ctx.status = 201; }});Copy the code

Regular expressions and security regular expressions

During a performance check, Yamatsuki found that an API took too long in the data verification layer, which I did not expect. The root of the problem is an unsafe regular expression. What is an unsafe regular expression?

For example, the following regular expression that can run the CPU is a ticking time bomb, and the number of backtracks explodes exponentially.

Can refer to the article analysis of ReDos principle and practice

const safe = require('safe-regex')
const re = /(x+x+)+y/

// a regex that can run to death
re.test('xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx')

// Use safe-regex to determine whether the re is safe
safe(re)   // false
Copy the code

Data verification, which is mostly for string verification, is also full of various regular expressions, so it is very important to ensure the security of regular expressions. Safe-regex can discover which regular expressions are unsafe.

conclusion

  1. The Controller layer needs unified data verification, which can adopt JSON Schema (Node to implement AJV) and Joi
  2. JSON Schema has official specifications and implementations of various languages, but the syntax is cumbersome. Joi with more powerful verification functions can be used
  3. When performing string validation, be aware of performance problems caused by unsafe re’s