preface

The last part introduces the project process design, database construction, JWT login and other modules

The basic platform construction section introduces branch management design and webSocket basic module

The next chapter introduces the basic modules of process management and test

The basic part mainly introduces the infrastructure design of the project management process and some basic Node development. This part starts to build the system series, and briefly describes the cooperation between Jenkins and the project management system

Jenkins

What is the Jenkins

Jenkins is an open source continuous integration (CI) tool that provides a user-friendly interface. Originated in Hudson (Hudson is commercial), Jenkins is mainly used for continuous, automated building/testing of software projects and monitoring the execution of external tasks. Jenkins is written in the Java language and can run in popular servlet containers such as Tomcat or on its own. It is often used in combination with version management tools (SCM) and build tools. Common version control tools include SVN and GIT, and build tools include Maven, Ant, and Gradle.

In this project, Jenkins is used as the primary build tool

Jenkins mounting (Trampling pit set)

Usually I don’t cover the installation process in my blog, but this one caused me to tread all day, so I’ll list it.

Windows environment

Download the Jenkins installation package directly, then download a JAVA SDK, and click to install a shuttle.

If you have a 64-bit system, don’t go to the sys32 directory to find the password, even though the web page shows the path to sys32

But C:\Windows\SysWOW64\config\ SystemProfile \AppData\Local\Jenkins. Jenkins is the correct path

Windows version is not recommended, and then make a virtual machine, double the experience effect

Mac environment

Brew direct download is recommended, easier to manage

Brew install too slow solution: Click here

The plug-in installation speed is slow

  1. Enter the http://ip:10086/pluginManager/advanced plugin management page, will address from updates. Jenkins. IO/update – cent… For mirror.xmission.com/jenkins/upd…

  2. Modify the Jenkins/updates/default. Json configuration

Replace updates.jenkins-ci.org/download mirrors.tuna.tsinghua.edu.cn/jenkins

Replace www.google.com with www.baidu.com

Over direct http://ip:10086/restart restart

Docer installation is not recommended, anti-aircraft guns play mosquito things less dry

Pipeline script

First, a brief introduction

Connection: Node, Agent, and slave are all used to refer to the server managed by Jenkins Master that performs Jenkins Jobs.

Difference: Agents are used in expressive pipelines, and can be not just Nodes, but docker containers, etc. Node (this is not that JS node) is used in scripted pipelines.

Directly on the node pipeline script, strong masturbation (simple to run first, late need to optimize the place is still a lot of)

Configuration parameters:

The name of the describe
PROJECT_NAME The name of the project
PROJECT_VERSION Engineering Version No.
PROJECT_GIT_PATH The project address
BRANCH_NAME Engineering branch
BUILD_PATH The build directory
CACHE Whether the cache 
node {
    stage('Pre Git') {
        echo "${params.PROJECT_NAME},${params.PROJECT_VSERSION},${params.PROJECT_GIT_PATH}"
        dir("D:/jenkins/build") {
            if(fileExists("${params.PROJECT_NAME}")) {
                echo " git exit"
                dir("D:/jenkins/build/${params.PROJECT_NAME}") {
                    powershell " git fetch --all && git reset --hard origin/${params.BRANCH_NAME} && git pull"
                    powershell " git checkout ${params.BRANCH_NAME}"}}else {
                echo " git is not exit"
                powershell " git clone ${params.PROJECT_GIT_PATH}"}}}stage('Pre Env') {
        echo "check node_modules,${params.CACHE}"
        dir("D:/jenkins/build/${params.PROJECT_NAME}") { 
            if(! fileExists("node_modules")) {
                powershell "cnpm i"
            }
            if(! params.CACHE) { echo"CACHE --- ${params.CACHE}"
                powershell "rimraf node_modules"
                powershell "cnpm i"}}}stage('build') {
        echo "check node_modules"
        dir("D:/jenkins/build/${params.PROJECT_NAME}") { 
            bat "npm run build"}}stage('test') {
        echo "test case"
    }
    stage('deploy') {
        echo "deploy project"
        if(! fileExists("D:/jenkins/deploy/${params.PROJECT_NAME}")) { 
            powershell " mkdir D:/jenkins/deploy/${params.PROJECT_NAME}"
        }
        if(! fileExists("D:/jenkins/deploy/${params.PROJECT_NAME}/${params.PROJECT_VERSION}")) { 
            powershell " mkdir D:/jenkins/deploy/${params.PROJECT_NAME}/${params.PROJECT_VERSION}"
        }
        powershell "cp D:/jenkins/build/${params.PROJECT_NAME}/${params.BUILD_PATH}/* D:/jenkins/deploy/${params.PROJECT_NAME}/${params.PROJECT_VERSION} -Recurse"}}Copy the code

The above script creates five stages, breaking the build process into five steps:

  1. Pull item: determine whether there is a local project, pull does not directly clone
  2. Install project dependencies: Check whether project dependencies are installed. If they are not installed and the cache is forcibly cleared, install them first
  3. Project Build: Run project build where scripts can be pulled out later
  4. Project test: reserved for later use
  5. Project publishing: directly cp or SSH upload to your publishing place (Nginx environment is set up locally, so copy to the corresponding directory)

Of course, the above scripts only complete simple construction tasks, encounter complex systems will directly kneel, simply list the probability of problems

  1. Multi-endpoint builds, such as a project that needs to directly build multi-endpoint artifacts
  2. Differentiate between development, test, pre-release, online and other environments
  3. Multi-command builds, complex projects may need to execute multiple commands to complete the build artifacts
  4. Build output catalog, publish catalog, etc
  5. And so on…………………………………………

Can be based on parameter transfer, multi-script and so on to solve the above problems, according to the specific business design

Because I am a Windows system, so the file processing is relatively poor, the running speed is slow and the script is difficult to write, if you want to use it, I strongly recommend Linux.

The figure above is built for 5 times in total. Due to the addition of cache judgment, the corresponding dependencies will be installed during the first build, which takes a lot of time.

However, in general, long-term iterative projects, dependencies do not change very frequently, so you just need to determine whether dependencies have been installed, and then skip dependency installation and go directly to the construction process to save the time to pour a glass of water.

Depending on your options, it’s ok to reinstall all dependencies with each build. Being happy is the most important thing!

Build product demo

Please note that the link above has a version number, which needs to be modified together with the scaffolding, which will be explained in detail in the Scaffolding section

Nginx

Nginx is a lightweight Web/reverse proxy server and email (IMAP/POP3) proxy server, distributed under a BSD-like protocol, and can run on UNIX, GNU/Linux, BSD, Mac OS X, Solaris, And Microsoft Windows and other operating systems.

In the actual operation, can support 20,000 to 40,000 concurrent, cost-effective. If you have no money, you can build it yourself. If you have money, you can go to COS, CDN and buy cloud services. PS: It’s nice to be rich!

Server {# here is my own configuration service port listen 10010; server_name resouce; root D:/jenkins/deploy; Autoindex on; Location / {add_header access-control-allow-origin *; } add_header Cache-Control "no-cache,must-revalidate"; # cache or not}Copy the code

Based on the above configuration, you can simply configure a static server. Drop the front-end project in and access the corresponding port directly.

Copy nginx.config directly to nginx.config and restart it

DevOps Jenkins Coding

Encapsulate the base Jenkins Api

The Jenkins library is selected for the project extension, note that you need to install the @types/ Jenkins dependency if you are using TS mode.

import * as jenkins from "jenkins";

/** * Jenkins connects *@param type* /
const getJenkins = function (
  type: "h5" | "node" | "nodeProduct" | "android" | "java"
) {
  const jenkinsConfig = {
    h5: {
      baseUrl: "http://devOps:118844ffb045d994acf8bb353e8d7b34f0@localhost:9001".crumbIssuer: true,},node: {
      baseUrl:
        "http://devOps:118844ffb045d994acf8bb353e8d7b34f0@localhost:9001".crumbIssuer: true,}};return jenkins(jenkinsConfig[type]);
};
/ * * *@description: Trigger Jenkins pipeline */
const buildJenkins = async ({ type, job, params }) => {
  const jenkinsCallback: any = await new Promise((resolve) = > {
    getJenkins(type).job.build(
      { name: job, parameters: params },
      (err: any, data: any) = > {
        if (err) {
          console.log("err: ", err);
          throw err;
        }
        resolve({ queueId: data }); }); });return { data: jenkinsCallback };
};
/ * * *@description: Obtain information about the current node */
const getQueuedInfo = async ({ type, queueId }) => {
  const jenkinsCallback: any = await new Promise((resolve) = > {
    getJenkins(type).queue.item(queueId, (err: any, data: any) = > {
      if (err) {
        console.log("err---->", err);
        throw err;
      }
      resolve(data);
    });
  });
  return { data: jenkinsCallback };
};
/ * * *@description: Get current build information */
const getJenkinsInfo = async ({ type, job, buildNumber }) => {
  console.log(type, job, buildNumber);
  const jenkinsCallback: any = await new Promise((resolve) = > {
    getJenkins(type).build.get(job, buildNumber, (err: any, data: any) = > {
      console.log("data: ", data);
      console.log("err: ", err);
      if (err) {
        console.log("err---->", err);
        throw err;
      }
      resolve(data);
    });
  });
  const { statusCode } = jenkinsCallback;
  if(jenkinsCallback && statusCode ! = =404) {
    return { data: jenkinsCallback };
  } else {
    return { data: jenkinsCallback }; }};/ * * *@description: Get Jenkins console.log information */
const getJenkinsConsole = async ({ type, job, buildId }) => {
  const jenkinsCallback: any = await new Promise((resolve) = > {
    getJenkins(type).build.log(job, buildId, (err: any, data: any) = > {
      if (err) {
        return console.log("err---->", err);
      }
      resolve(data);
    });
  });
  return { data: jenkinsCallback };
};

export default {
  buildJenkins,
  getQueuedInfo,
  getJenkinsInfo,
  getJenkinsConsole,
};
Copy the code

The above is the basic encapsulation of Jenkins, which simply encapsulates some methods we need to use. The specific customization can be designed by ourselves based on the business.

For business construction at each end, you can select multiple Jenkins projects or separate different jobs. It is not recommended to directly transfer one job to black, because the script will be very complicated.

A real jock to the black, here’s to you for being a man

// Controller
import { Post, Prefix, Get } from "egg-shell-decorators";
import BaseController from "./base";
@Prefix("build")
export default class BuildController extends BaseController {
  / * * *@description: Create a build task */
  @Post("/creatJob")
  public async getUserToken({ request: { body: { params }, }, }) {
    const { ctx, app } = this;
    const { access_token: accessToken } = this.user;
    const {
      projectId,
      branchName,
      projectVersion,
      buildPath,
      type,
      cache,
    } = params;
    const project = await ctx.service.project.getProject({ projectId });
    let projectGitPath = project.projectUrl.replace(
      "http://".`https://oauth2:${accessToken}@ `
    );
    const callBack = await ctx.service.build.buildProject({
      type,
      projectName: project.projectGitName,
      projectVersion,
      projectGitPath: `${projectGitPath}.git`,
      branchName,
      buildPath,
      cache,
    });
    this.success(callBack); }}// Service 
import { Service } from "egg";
export default class Build extends Service {
  / * * *@description: Build project */
  public async buildProject({
    type = "h5",
    projectName,
    projectVersion,
    projectGitPath,
    branchName,
    buildPath,
    cache,
  }) {
    const { ctx } = this;
    const callBack = await ctx.helper.api.jenkins.index.buildJenkins({
      type,
      job: "fe-base-h5".params: {
        PROJECT_NAME: projectName,
        PROJECT_VERSION: projectVersion,
        PROJECT_GIT_PATH: projectGitPath,
        BRANCH_NAME: branchName,
        BUILD_PATH: buildPath,
        CACHE: cache,
      },
    });
    returncallBack; }}Copy the code

The above is the business code, generally when obtaining the return value, save the record queueId, by calling Jenkins API to obtain the release time and log

As shown above, Jenkins is integrated with the project management system to facilitate user operation.

Building information push

The front-end polling

The Jenkins Api can be queried directly with the returned queueId to retrieve the information

Advantages: violent, simple, the fastest development speed, relatively fast

Disadvantages: users will not be able to perceive when they leave the page, data drop will be interrupted, and extremely consume performance, multiple users in the operation of the same project, unable to timely notify in place

Background polling + socket

The Node background directly polls Jenkins Api through queueId, and pushes it to the front end for display through Websocket

Advantages: violence, development speed, moderate difficulty, even if the user leaves the page, data can still fall library, can be pushed to multiple users at the same time

Disadvantages: Node background performance consumption increases, need to cooperate with the front and background development, a large number of useless messages need to fall into the library, and the Node can not perceive

webhook + socket

Node opens webhook interface, Jenkins pipeline pushes message to Node background at each stage, and then pushes message to front-end display through socket

Advantages: Saves resources to the greatest extent, and can customize the effective data and node perception, with the highest timeliness

Disadvantages: The front-end, node, and script need to be developed together, resulting in high cost

Students can choose the way of low cost and high profit to cooperate with the development in the actual development process

The best way is not necessarily your best choice, the most important cost performance

advice

For those of you who read this blog from the first post, if the team lacks proper project management or wants to practice Node, you can try it out. I have posted most of the key code directly on my blog (most of it can be copied).

The construction section is officially opened, and blog posts of the construction section will be released gradually in the later stage, which may be more inclined to operation and maintenance development. If the front-end students have difficulty, they can consult the back-end or operation and maintenance students.

Don’t take away everyone’s jobs on the front end is not a good front end, manual dog head

Full series of blog entries

The backend module

  1. DevOps – Gitlab Api usage (completed, click to jump)
  2. The conversation – build enterprise foundation platform Basic platform building last | foundation platform structures, medium-length | foundation platform set up next
  3. DevOps – Gitlab CI pipeline construction
  4. Devops-jenkins pipelined construction
  5. Devops-docker use
  6. DevOps – Release task flow design
  7. DevOps – Code review points
  8. Devops-node monitors service quality

The front-end module

  1. DevOps – H5 base scaffolding
  2. Devops-react project development

The above series may be adjusted later based on the actual development progress of the DevOps project

The end of the

This project is from zero development, the subsequent series of blog will be launched according to the actual development progress (really f * * king tired), after the completion of the project, will open part of the source code for your reference.

Why open part of the source code, because some business needs to fit the actual project targeted development, open out of the public module I write seriously

In order to write a series of blog, actually really masturbate a complete system (not generally tired), feel good students trouble three even (praise, attention, forward).

If you have any questions about this article, please leave a comment