This post was first posted on my CSDN blog Tank_in_the_street

Before we talk about CI/CD, let’s talk about what it means.

CI is short for Continuous Integration. CD is short for Continuous Deployment.

Continuous integration means that multiple developers can frequently merge lines of code in the process of developing different functions without affecting each other’s work.

Continuous deployment means the automated build, test, and deployment of code to an online environment based on a tool or platform to deliver high-quality products. Continuous deployment, to some extent, represents the rate of iteration of a development team.

That sounds a bit esoteric, but how do you put it together? Let’s start with the traditional deployment method.

The traditional small program deployment and testing process requires developers to push the code into GitLab, and then test the code of the designated branch of the small program, and configure the corresponding parameters such as appID, before the test can be carried out. The whole process is inefficient and requires testers to be familiar with wechat developer tools. Manual parameter configuration is also prone to error, resulting in a long test of the wrong version.

The new CI/CD addresses these problems, from automatic merging of code branches to foolproof automated deployment, in order to improve efficiency and reduce errors. This is also what I have been doing recently. However, since I just got in touch with the company’s business and was not very familiar with it, I stopped doing CI for the time being and instead focused on the code pre-detection and CD work before CI.

I. Code pre-submission detection

Before CI, we must first unify the code format of each member. The esLint thing is also only used as a validation tool. Members do not follow rules and can still submit code to GitLab. If some code specification is different, such as declaring variables unused, not adding semicolons, these are minor things. If you name multiple variables repeatedly, or even push up the offending code, the entire code repository will be contaminated by this record.

How to do? At this point it’s husky’s job to force the members to comply. Husky is a package that captures git hooks at different stages, and by configuring some methods for those hooks, it does what I want.

This plugin alone won’t do. It will only capture the hooks at different stages of Git, and it’s not clear which files need to be processed. This is where lint-staged plug-ins will have to be used. Lint-staged file filters are file filters designed to filter files from git caches. Desired files can be obtained by configuring file suffixes to be filtered. The huskyrc file is configured as follows:

{ 
    "hooks": { 
        "pre-commit": "lint-staged" 
    } 
}
Copy the code

Even with these two add-ons, there’s still something missing. Yes, there is a plug-in that automatically formats the code that is missing. Prettier is the plug-in used here. Prettier is a multilingual code formatting tool, Prettier is more powerful than ESLint in that it can format entire code. Lint-staged use looks like this:

{ 
    "src/**/*.{js,jsx,ts,tsx}": ["prettier --write", "eslint"] 
}
Copy the code

There is esLint after the statement to prevent code from being submitted without formatting when Prettier is wrong.

In conjunction with these tools, code pre-commit detection is sufficient.

Second, the CI

For CI, there are a lot of things to consider, such as what the current team’s git workflow looks like. Since the git workflow of my technical team is Gitflow, continuous integration is not possible for master, but for the Release branch.

However, because Gitflow is a decade-old workflow idea, it is somewhat inadequate for the continuous integration that is now required for continuous delivery. I’m going to avoid that pit here and not work too much on CI for now, but more on CDS instead.

As for the fact that Gitflow is not A good fit for continuous integration, look at an article written by the founder of the Gitflow workflow: A successful Git Shoot model He personally recommended GitHubFlow.

Three, CD

This is where I’ve been focusing my research recently, thinking about the current landing of continuous deployment and what to do in the future. Continuing to deploy this thing frees up a lot of their workforce for testers to focus on doing their job.

The CD construction work is like this. The tool is Jenkins. At the beginning, create a new task in Jenkins and fill in the corresponding name.

​

Then fill in the project description. The most important thing is to fill in the code address to pull and configure the token.

​

To configure the code address, you only need to enter the address in the Repository URL. In the Credentials column, you need to enter your SSH in Gitlab and check git SSH on Baidu. I don’t need to write more details here.

Then on General, add the Parameter and select Git Parameter. In Git Parameter, to be consistent with $branch, name is branch and select the default branch.

​

The miniprogram is divided into upload and preview. The integration of these two functions into Jenkins requires the plug-in miniprogram-ci provided by the official miniprogram. The official documentation of the miniprogram has been very clear about the use of this plug-in, interested in this document: MiniProgram-CI overview

We just need to install this plug-in on the project, configure a run file of their own, and then.

// const CI = require('miniprogram- CI '); const { appid, qrcodeOutputDest, robot, branch, type, desc, projectPath, privateKeyPath, version, setting } = require('./constants'); Let descInfo = ` branches: ${branch} \ r \ n version: ${version} \ r \ n description: ${desc} `; (async () => { const project = new ci.Project({ appid: appid, type: 'miniProgram', projectPath: projectPath, privateKeyPath: privateKeyPath, ignores: ['node_modules/**/*','build/*','package.json','package-lock.json','gulpfile.js','**/*.less'], }); if (type === 'preview') { const previewResult = await ci.preview({ project, desc: descInfo, setting, robot, qrcodeFormat: 'image', qrcodeOutputDest: qrcodeOutputDest, onProgressUpdate: console.log, }); console.log(previewResult); } else if (type === 'upload') { const uploadResult = await ci.upload({ project, version: version, desc: descInfo, robot, setting, onProgressUpdate: console.log, }); console.log(uploadResult); }}) ();Copy the code

Because of the distinction between upload and preview types, you need to add an additional text parameter to Jenkin to specify the method to be selected. This parameter is injected into the Node environment, and pubType is captured in process.argv.

​

In addition, small programs are distributed in different packages according to the current team situation. The current situation of my team is to distinguish between official packages and test packages, so I have to add another configuration in the text parameter to configure what type of package needs to be uploaded and previewed:

​

Later parameters like build bots, appids, etc., are used in a similar way, which I won’t go into.

Finally, the last step is to configure the shell command for the build. There is no node_module after the project code is pulled down, so we need to install node_module ourselves, and then we need to write shell command.

In the column of “build”, add the construction step of “execute shell”, first of all, install dependencies for the project. Since Jenkin usually runs on the internal network of the company, the internal network cannot be scaled over the wall to a large extent, so it is better to configure the NPM agent in a proper manner:

node -v
npm -v
npm install --registry=https://registry.npm.taobao.org
Copy the code

Remember the environment configuration parameter (env) above, we usually run the program locally by running NPM run dev, which is configured in package.json, and in production, we need to configure the production environment on package.json.

Of course, after we run it, we also need to execute the miniProgram-ci run file that we configured, pass in the configuration parameters that we wrote in the text parameters, and we’re almost done.

node -v
npm -v
npm install --registry=https://registry.npm.taobao.org
echo $env

npm run $env

node CI/index.js type=$pubtype branch=$branch desc=$desc version=$version robot=$robot
Copy the code

If you need to upload or preview a branch, fill in the following configuration:

​

Since the qrcodeOutputDest parameter in the miniProgram-CI configuration file specifies the generation path of the preview TWO-DIMENSIONAL code image, you only need to find the image according to the configured path, download it to the local, and compile it with the two-dimensional code of the developer tool to see the packaged project in the developer tool.

​

Four, encountered pit

1. socket hang up

Socket hang up is a problem that’s killing me. Sometimes you’ll find that your build is not working and you don’t even know what went wrong.

The solution to this problem is to proceed from its original meaning. When the applet performs upload and preview, it calls an interface called Servicewechat.com, with a route that changes depending on whether you’re uploading or previewing, and a long list of parameters. These are the parameters that you pass in when you execute the ci.preview or ci.upload methods.

This domain name is a connection with the applet side, upload the code to the applet background side. If you are running on a local computer, you will rarely see this error. But if you are in the Jenkin, because the Jenkin is in the Intranet environment, there is a restriction on the Internet. Take the Jenkin service of our company as an example, the bandwidth configured for the whole Jenkin service is only 5M, and it is allocated to five machines for use. The default timeout time is one minute. Due to network problems and large project code, sometimes the code cannot complete push within one minute, which leads to the emergence of socket hang up.

The simplest and most crude way to solve this problem is to increase the bandwidth or start with the project code and do some code size optimization. You can even ask the operations staff to make the timeout longer, but this is a bit of a band-fix, because bandwidth is so large and there are five machines using it, and a longer timeout will only waste bandwidth and affect other services that are being deployed.

Finally, of course, I still ask the operation and maintenance students to improve the bandwidth. As for the project code, we need to negotiate well.

Of course, there are other solutions on the Internet, which can be realized by modifying the designated DNS. This still depends on the cooperation of operation and maintenance students. For details, please see: The implementation of small program automatic deployment scheme in Good Doctor

2. syscall name

This is usually caused by manually canceling the Jenkin deployment process during build time when the dependency is being installed, because node_moDUE already generates a dependency cache file when the dependency is installed, and this error will be triggered when you click deploy again.

To resolve this error, determine which dependency generated a cache file with the same name. Delete it and execute this command in shell:

npm cache clean --force
Copy the code

Then reinstall the dependencies.

It is easy to prevent this problem by not executing the NPM I command after the node_moDUE file is generated during the first shell build. Unless the package.json file changes.

Or you can delete node_module each time the shell executes a build and then reinstall it without this problem.

3. Preview images cannot be displayed automatically

The preview image will eventually be generated to the address you configured in qrcodeOutputDest, so you will generally have to open the file manually from the Jenkin project.

How do I show this preview image? Many of the solutions on the web are to install the jenkin plug-in and then modify the global configuration to display the image link directly in the build history, for example:

​

That’s a good idea. You can do that. But if on a corporate project, unless the entire Jenkin is newly deployed, suddenly changing global configuration for your project, when there is no clear risk, you are ready to run away.

Unfortunately, if the Jenkin task of building a project through the whole set of procedures mentioned above, the picture cannot be displayed without plug-in. It has to be rebuilt and a new project construction task is created by pipeline, which is what I will do later.

Five, late things to supplement

Let’s review how we just generated a build task.

First of all, we realized our CD workflow by selecting new task in Jenkins, and then clicking the box to fill in variables and shell commands step by step.

​

But this also has a disadvantage, first of all, the first disadvantage is just said, can not preview the TWO-DIMENSIONAL code directly into the build history display, have to let the test through the generation of address to find two-dimensional code pictures. If another student clicks build in the project, the previous QR code picture will be automatically recycled, and the test students may not even see it.

Another problem is not being flexible enough. All I see so far is a semi-automatic CD workflow, and my goal is a fully automatic CD workflow? What is fully automatic?

First, in the above deployment, there were so many parameters that you had to fill them out before hitting Build. In fact, since there is no sense of branch code changes in the GitLab repository, you have to manually configure each time before hitting Build deploy. With a free-style software project, the logic of each step is not well executed.

So what to do? Pipeline (pipelining) can once and for all solve the difficult bottleneck encountered above.

Pipeline is a set of workflow frameworks running on Jenkin. Unlike previous configurations, Pipeline can be configured more flexibly through Groovy scripts and can perform more fine-grained operations on GitLab triggers. The following configuration method is roughly as follows:

First, select line tasks in Jenkins.

​

Then click the advanced option in the Build trigger, copy the WebHook, and select the branch you want to listen on. I have chosen to match branches in the canonical form:

​

Click Generate to save the Secret Token. Then open GitLab, paste in the configured address and token, and click on the new tag to trigger the builder.

​

Then you can try to tag the target branch in GitLab. If you can see the build on Jenkins, it will prove successful.

Then fill in the pipeline code, pipeline code is a Groovy script, you can make more detailed operations on the build process, because it has not yet to practice, here is a piece of pseudo-code.

Before writing pseudo-code, another word, pipeline is divided into declarative and scripted, the difference between the two from the code is like this:

// Pipeline {agent None stages {stage('Example Build') {agent {docker 'Maven: 3.8.1-adoptopenjdK-11'} steps { echo 'Hello, Maven' sh 'mvn --version' } } stage('Example Test') { agent { docker 'openjdk:8-jre' } steps { echo 'Hello, JDK' sh 'Java -version'}}}} // Scripting node {stage('Example') {if (env.branch_name == 'master') {echo 'I only execute  on the master branch' } else { echo 'I execute elsewhere' } } }Copy the code

According to the official website:

Scripted Pipeline, like Declarative Pipeline, is built on top of the underlying Pipeline sub-system. Unlike Declarative, Scripted Pipeline is effectively a general-purpose DSL [2] built with Groovy. Most functionality provided by the Groovy language is made available to users of Scripted Pipeline, which means it can be a very expressive and flexible tool with which one can author continuous delivery pipelines.

Simply put, scripting is more flexible because you can control finer nodes. But it depends on the actual situation of the project code. I’ll just use the declarative form here.

For details on pipeline Syntax, see the document: Pipeline Syntax

Other options

Jenkins Pipeline is a good solution to our problems. Some students may wonder if there are other solutions. There are certainly some, such as tekton for CI/CD workflow, document: Tekton, and gitlab-CI can be used to achieve similar effects.

Reference documentation

1. What is a GitFlow workflow?

2. Jenkins cooperated with GitLab to realize automatic merging of branches and automatic Tag creation

3. What is CI/CD

4. Mouth to mouth, hand to hand, 10 minutes to teach you how to use Jenkins +miniprogram-ci to automatically generate wechat miniprogram preview qr code

5. Jenkins Pipeline Series 1 — How to configure extended shared libraries

6. Pipeline Syntax