Peach and plum spring breeze a cup of wine, river’s lake night rain ten years lamp.

preface

When using Cloud development at CloudBase, it always comes to mind: how to separate back-end operations such as database calls from front-end services when using cloud development?

After reading the source code of “CloudBase CMS”, I came up with the idea of using NestJS + CloudBase +cloud function (no mistake, the original idea is cloud function).

What is NestJS?

The documentation describes it like this:

Nest is a framework for building efficient, scalable Node.js server-side applications. It uses progressive JavaScript, has built-in and full TypeScript support (but still allows developers to write code in pure JavaScript), and combines elements of OOP (object-oriented programming), FP (functional programming), and FRP (functional response programming). At the bottom, Nest uses powerful HTTP Server frameworks such as Express (the default) and Fastify. Nest provides a level of abstraction on top of these frameworks, while also exposing its APIS directly to developers. This makes it easy to use countless third-party modules for each platform.

What is Cloudbase?

Cloud Development (TCB) is a cloud native integrated development environment and tool platform provided by Tencent Cloud, providing developers with highly available, automatic and flexible expansion of back-end cloud services, including computing, storage, hosting and other serverless capabilities. Can be used for cloud integration development a variety of end applications (small procedures, the public, Web applications, because the client, etc.), to help developers build and management the back-end services and cloud resources, avoiding the tedious server builds in the process of application development in ops, developers can focus on the business logic implementation, development threshold is lower and more efficient.

Why is there cloud hosting?

Cloud Hosting (Tencent CloudBase Run) is a new generation of cloud native application Engine (App Engine 2.0) provided by Tencent CloudBase (TCB), which supports hosting containerized applications written in any language and framework. Together with other cloud development products (cloud functions, cloud databases, cloud storage, extended applications, HTTP access services, static web hosting, etc.), it provides users with a cloud native integrated development environment and tool platform, and provides developers with highly available, automatic, flexible and scalable back-end cloud services. It can be used for cloud integrated development of multiple applications (small programs, public accounts, Web applications, micro-service applications, Flutter clients, etc.) to avoid tedious server construction, operation and maintenance during application development, enabling developers to focus on the realization of business logic, with lower development threshold and higher efficiency.

Build the project based on NestJS

Because CloudBase provides the framework of NestJS, we can directly use the scaffolding tool of Cloudbase to build. By default, we have a cloud development environment and have configured the cloud development environment on Tencent cloud console.

Scaffold creation project

  • Scaffolding installation tool
npm i -g @cloudbase/cli
Copy the code
  • Test whether the installation is successful

After the cloudBase scaffolding is installed successfully, you can use CloudBase -V to check whether the scaffolding is installed successfully

tcb -v
Copy the code
  • Landing cloudbase
cloudbase login
Copy the code
  • Create a project locally
tcb new <appName> [template] 
#Such as 
tcb new nest_test nest-starter
Copy the code

The project file

file role
app.controller.ts Example of a basic controller with a single route.
app.controller.spec.ts Unit test sample for base controller
app.module.ts The root module of the application.
app.service.ts Basic services with a single method
main.ts Application entry file. It uses NestFactory to create Nest application instances.

Cloud Function construction

When the project was initially deployed as a cloud function, you had to configure the CloudBaserc.json file:

{
    "version": "2.0"."envId": "ID" environment."$schema": "https://framework-1258016615.tcloudbaseapp.com/schema/latest.json"."framework": {
      "name": "Function name"."plugins": {
        "node": {
          "use": "@cloudbase/framework-plugin-node"."inputs": {
            "name": "Function name"."path": "/ function path"."entry": "app.js".// Configure the build command for the function
            "buildCommand": "npm install --prefer-offline --no-audit --progress=false && npm run build"."functionOptions": {
              "timeout": 5.// Environment variables
              "envVariables": {
                "NODE_ENV": "production"."TCB_ENV_ID": "{{env.TCB_ENV_ID}}",}}}}}}}Copy the code

Json automatically recognizes environment variables in. Env. local or. Env, so we can dynamically import the configured environment variables in curly braces. Prohibit. Env. local or. Env from uploading to the repository.

# compiled output /dist /node_modules .env.local # Logs logs *.log npm-debug.log* yarn-debug.log* yarn-error.log* lerna-debug.log* # OS .DS_Store # Tests /coverage /.nyc_output # IDEs and editors /.idea .project .classpath .c9/ *.launch .settings/ *.sublime-workspace # IDE - VSCode .vscode/* ! .vscode/settings.json ! .vscode/tasks.json ! .vscode/launch.json ! .vscode/extensions.jsonCopy the code

Cloud functions are easy to upload, so once configured and uploaded, we can use them in our project in the form of app.callFunction().

Problems encountered by cloud functions

Ex – ceed_MAX_payload_size I need to provide a picture matting API, but I gave an error when I uploaded base64 pictures ex – ceed_MAX_payload_size

HTTP request body exceeds maximum size limit (text 100 KB, binary 20MB)

The first idea

The first idea is that we set the limited volume of the body, in the project to solve this problem, but found that after the set does not change the size of the HTTP request body directly, by tencent cloud console query log, found that each request and could not enter into the actual code, so we cannot to configure the size of the request body.

Through contact with cloudbase development students, we know that cloud functions are limited by packages and provide two solutions:

  1. Base64 pictures are uploaded through the client package of @Cloudbae/JS-SDK, and then passed to the back end for human body analysis through fileId. However, we do not plan to store the pictures uploaded by users, so this scheme is temporarily excluded

  2. Deploy in cloud hosting where the service is mounted so you can set the size of the request.

Therefore, I decided to deploy the service in the form of cloud hosting.

Difference between cloud functions and cloud hosting

The module Cloud function Cloud hosting
The request of concurrent Single instance single concurrency, multiple instances need to be pulled up for processing Single instance multiple concurrency
Language/Framework Development language and framework support are limited Compatibility with existing frameworks
Problem orientation Easy to locate Relatively flexible and dependent on customization
Resident run Does not support support
Log monitoring Based on the function Based on the service
Version of the gray Support gray scale by flow Supports the gray scale of traffic and URL parameters
Elastic expansion and contraction capacity support support
Foreign service Provide the default URL and SDK Provide the default URL
cross-platform Function rules are different and are difficult to deploy across platforms Cross-platform deployment
Private deployment Does not support Portable private/mixed deployment
To fit the difficulty simple medium
Billing way Billing by request, by number of requests, and GBS generated per call Based on the CPU and memory consumed by the container, outbound traffic generated by the service, and service construction duration

Cloud hosted Build

The definition of cloud hosting has been described above. I will not repeat it here. Those who are interested can check out the “Cloud Hosted Documentation”.

How to quickly build a cloud hosted Nodejs application?

Application containerization

In the project root directory, create a file called Dockerfile with the following contents:

# Use official Node.js 12 lightweight images.
# https://hub.docker.com/_/node
FROM node:12-slim

Define the working directory
WORKDIR /usr/src/app

Copy the dependency definition file to the working directory
COPY package*.json ./

Install dependencies in production form
RUN npm install --only=production

Copy the local code to the working directory
COPY . ./

# start service
CMD [ "node"."index.js" ]
Copy the code

So how do you build a cloud-hosted Nestjs project?

According to the above dockerfile file, we can make the relevant configuration:

# Use official Node.js 12 lightweight images
FROM node:12.15.0-alpine

# Set the time zone for Alpine Linux
RUN apk --update add tzdata \
  && cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \
  && echo "Asia/Shanghai" > /etc/timezone \
  && apk del 

RUN mkdir -p /usr/src/app

Define the working directory
WORKDIR /usr/src/app

Copy the dependency definition file to the working directory
COPY package.json /usr/src/app/package.json
Copy the dependency definition file to the working directory
COPY package*.json ./

Copy the local code to the working directory
COPY . ./
Install dependencies in production form
RUN npm install

# RUN npm run build

# port
EXPOSE 5000

# start service
CMD npm start
Copy the code

Add a.dockerignore file to exclude files from the container image:

.git
dist
node_modules
npm-debug.log
Copy the code

At this point, according to the document, we can upload the image file through docker local build.

So how do we deploy projects directly from the command line with one click?

Configure cloud hosting using CloudBaserc.json

Cloudbase provides the @CloudBase /framework-plugin-container plug-in for us to make cloud hosted upload files. Therefore, according to the “CloudBaserc. json document”, we can do the following configuration:

{
  "version": "2.0".ID / / environment
  "envId": "{{env.TCB_ENV_ID}}"."$schema": "https://framework-1258016615.tcloudbaseapp.com/schema/latest.json"."framework": {
    "name": "Name"."plugins": {
      "service": {
        "use": "@cloudbase/framework-plugin-container"."inputs": {
          "serviceName": "serviceName"."servicePath": "/serviceName-container"."localPath": "."."cpu": 0.25."mem": 0.5./ / the port number
          "containerPort": 5000."bumpVersion": true."dockerfilePath": "./Dockerfile"."buildDir": ". /"."uploadType": "package".// Environment variables
          "envVariables": {
            "NODE_ENV": "production"."TCB_ENV_ID": "{{env.TCB_ENV_ID}}"."SECRETID": "{{env.SECRETID}}"."SECRETKEY": "{{env.SECRETKEY}}"."BAI_DU_APPID": "{{env.BAI_DU_APPID}}"."BAI_DU_APP_KEY": "{{env.BAI_DU_APP_KEY}}"."BAI_DU_APP_SECRET_KEY": "{{env.BAI_DU_APP_SECRET_KEY}}"."COS_SECRET_ID": "{{env.COS_SECRET_ID}}"."COS_SECRET_KEY": "{{env.COS_SECRET_KEY}}"
          }
        }
      }
    }
  }
}

Copy the code

And that’s it?

When the build was uploaded using cloud hosting, we found that the deployment would always fail. After checking the build log on the console, we learned that the start command was wrong. Therefore, we reconfigured the NPM start command:

The HOSTING environment variable has been added to determine whether it is cloud hosted. In the project file, we have modified the previous method of determining cloud functions and local development:

The original way:

// Compatible with cloud functions and local development
if (process.env.NODE_ENV === 'development') {
  await app.listen(port);
} else {
  await app.init();
}
  
// Start development in development mode
if (process.env.NODE_ENV === 'development') {
  bootstrap().then(() = > {
    console.log(`App listen on http://localhost:${port}`);
  });
}
Copy the code

Now the way:

// Compatible with cloud functions and local development
if (isRunInServerModel()) {
  await app.listen(port);
} else {
  await app.init();
}
Copy the code

IsRunInServerModel () function:

// Run in server mode, that is, by listening on ports
export const isRunInServerModel = () = >
  process.env.NODE_ENV === 'development' ||
  process.env.HOSTING;
Copy the code

Configuration complete, we re-upload.

I think we’re done

Hey hey, upload successful! At the same time, query the console, we previously configured environment variables are all successful, then I can not try my interface ah!!

However, we occasionally have the problem of internal server error, but we can not find the specific error line number and content, which we can not tolerate, so we need to perfect the configuration of our project!!

Project configuration

After initializing the build project, we have a simplified version of the NestJS + CloudBase Demo service.

Wait!! That’s it?? That simple?

If we were just running a demo and trying out cloud development, it would be that simple!

However, we will definitely carry out more project configuration in business development to meet our business needs, such as filtering and processing HTTP exceptions, parameter verification, printing logs and so on.

Therefore, we need to do more project configuration for the project.

Request the body restriction

Nestjs still uses Express at the bottom, while NodeJS has a 100KB limit on body. If we pass base64 images, we are likely to fail

{
    code: 500,
    msg: "Service Error: PayloadTooLargeError: request entity too large"
}
Copy the code

Therefore, we need to configure the body size in our code:

Json () is commonly used, but typescript tells us bodyParser is deprecated.

So, we use express.json() to configure:

Verification of parameters

Nestjs provides a ValidationPipe for checking parameters.

  // Check parameters
  app.useGlobalPipes(new ValidationPipe());
Copy the code

First: we define a validation.pipe.ts file

import { ArgumentMetadata, Injectable, PipeTransform, BadRequestException } from '@nestjs/common';
import { validate } from 'class-validator';
import { plainToClass } from 'class-transformer';

@Injectable(a)export class ValidationPipe implements PipeTransform {
  // transform(value: any, metadata: ArgumentMetadata) {
  // return value;
  // }
  async transform(value: any, { metatype }: ArgumentMetadata) {
    console.log(`value:`, value, 'metatype: ', metatype);
    if(! metatype || !this.toValidate(metatype)) {
      // If no validation rule is passed in, data is returned without validation
      return value;
    }

    // Convert the object to Class for verification
    const object = plainToClass(metatype, value);
    const errors = await validate(object);
    if (errors.length > 0) {
      const msg = Object.values(errors[0].constraints)[0]; // Just take the first error message and return it
      // Logger.error(`Validation failed: ${msg}`);
      throw new BadRequestException(`Validation failed: ${msg}`);
    }
    return value;
  }

  private toValidate(metatype: any) :boolean {
    const types: any[] = [String.Boolean.Number.Array.Object];
    return !types.includes(metatype);
  }
}

Copy the code

Also, according to the xxx.to. ts file, we can set the input parameter form

/* * @Author: Lupoy * @Date: 2021-08-31 11:21:17 * @LastEditTime: 2021-09-08 11:06:55 * @LastEditors: Lupoy * @ Description: * @ FilePath DTO files: / SRC/modules/XXX XXX. DTO. Ts * /

import { IsNotEmpty, IsString } from 'class-validator';
import DefaultData from '@/modules/baidu/baidu.i18n';

export class xxxDTO {
  @IsNotEmpty({ message: DefaultData.IS_NOT_EMPTY })
  @IsString({ message: DefaultData.IS_STRING })
  readonly base64: string;
}

export class xxxDTO {
  @IsNotEmpty({ message: DefaultData.IS_NOT_EMPTY_IN_KOU_TU })
  readonly base64: string;
  @IsNotEmpty({ message: DefaultData.IS_NOT_EMPTY_IN_KOU_TU })
  readonly beautify: number;
  @IsNotEmpty({ message: DefaultData.IS_NOT_EMPTY_IN_KOU_TU })
  readonly builder: number;
}

Copy the code

It can now be used in controller:

/* * @Author: elizhai * @Date: 2021-08-26 14:51:33 * @LastEditTime: 2021-09-08 11:08:18 */
import { Body, Controller, Post } from '@nestjs/common';
import { xxxService } from '@/modules/xxx/xxx.service';
import { xxxDTO, xxxDTO } from '@/modules/xxx/baidu.dto';

@Controller('x')
export class xxxController {
  constructor(private readonly xxxService: xxxService) {}

  @Post('bg/xxx')
  removeBg(@Body() body: xxxDTO) {
    return this.xxxService.removeBg(body.base64);
  }
  // Human body analysis
  @Post('xxx')
  xxx(@Body() body: xxxDTO) {
    const { base64, beautify, builder } = body;
    return this.xxxService.koutu({ data: base64, beautify, builder }); }}Copy the code

When passed parameters do not conform to the specification we defined, an error is intercepted.

Setting up the Log System

It is recommended to learn from “Nest.js from Zero to one series (iv) : Building a Logging System using Middleware, Interceptors and Filters” by “Bradpie”.

Other configuration

Nestjs provides a number of apis to set, such as app.setGlobalPrefix(‘ global route prefix ‘), app.disable(‘ X-powered-by ‘), and so on……

These can be customized according to the requirements of the project.

How to switch between cloud functions and cloud hosting mode freely?

How to freely switch between cloud function and cloud hosting two upload modes?

We define the scripts folder in the root directory, where we store cloud functions and cloud hosting json files, and then run the script command to write them to the cloudBaserc. json folder in the root directory.

/* * @Author: elizhai * @Date: 2021-09-08 10:42:50 * @LastEditTime: 2021-09-08 10:57:54 * @LastEditors: Please set LastEditors * @description: node script, perform cloud function upload or cloud hosting upload * @filepath: /scripts/index.js */

const fs = require('fs');
const path = require('path');

const fileName =
  process.env.FILE_ENV && process.env.FILE_ENV === 'hosting'
    ? 'cloudbaserc'
    : 'cloudbaserc-fx';

fs.readFile(path.join(__dirname, `. /${fileName}.json`), 'utf8'.function(err, data,) {
  if (err) throw err;

  fs.writeFile(path.join(__dirname, '.. /cloudbaserc.json'), data, 'utf8'.err= > {
    if (err) throw err;
    console.log('done');
  });
});

Copy the code

conclusion

At this point, we have successfully deployed the service and configured our own project, and the basic project based on CloudBase + cloud function or cloud hosting + NestJS is completed.

In the past, due to the need to configure the server, there was a huge gap between the front end and the back end. However, cloudBase provides cloud functions and cloud hosting so that the front end can also do the back end project in some aspects, which can make a front end quickly become a “pseudo-full stack”.

All rivers run into the sea, not because the sea is big, because the sea is low posture