One, foreword

The contents of this article include:

  • Understand CI/CD and its necessity
  • Gitlab-runner installation and registration
  • Gitlab-ci configuration description
  • SSH Encryption-free login
  • Rsync deployment file
  • Multi-environment publishing and rollback
  • High frequency problems and solve

Reading this article requires you to:

  • Have a basic knowledge of SSH, SSH operation guide
  • Gitlab CI/CD conceptual foundation
  • Take about 10 minutes

CI/CD popular science

2.1 CI with CD

Continuous Integration refers to the process by which developers submit code at feature branches (frequently), immediately perform build and unit tests, and integrate the code into the trunk once it passes the test criteria. The emphasis is on branch code submission, build, and unit testing, and the output of this process is a unit test report.

instructions
unit test

Continuous Delivery is the process of deploying built code to a “class production environment” on the basis of Continuous integration, and then manually deploying it to a production environment after completing QA testing. Emphasize code deployment, which produces test reports.

instructions

Continuous Deployment is the next step in Continuous interaction, emphasizing automation of the process of deploying production code while handling operations such as notification of going online.

instructions

2.2 Necessity of CI/CD

In a nutshell, I think it should be: let machines do what machines do. A development team without CI/CD might look something like this: no way to manage multi-person code collaboration (Git Repository is part of CI), a series of shells need to be handled manually, code distribution needs to be logged into the server, etc. Instead, with CI/CD, these tasks are handed over to the machine, freeing up fragmented time for more meaningful things (like fishing for relaxation).

2.3 What would the ideal CI/CD development flow look like?

I think the ideal CI/CD development flow should consist of three phases: Build, deploy, and notify. Focus on code construction and unit testing in build phase, code deployment in test/ Gray/PROd environment in deploy phase, and online notification in notify phase, as shown in the figure below.

The following completes the 0-1 deployment around the Build and deploy phases. Author’s system environment: Ubuntu 18.04.1 LTS

Iii. Install and register gitlab-Runner

3.1 installation runner

sudo apt-get install gitlab-runner
Copy the code

Install gitlab-runner

3.2 registered runner

sudo gitlab-runner register
Copy the code

After that, qA-type operation is performed. Input information as prompted. Please refer to the official website for operation.

①. Gitlab host and token are found on your Gitlab project in Settings >> CI/CD >> Runners

②. Runner select docker, image input node:8.11.2-stretch

Once registered, we can see our registered runner under Settings >> CI/CD >> Runners

4. Gitlab-runner configuration

Create a.gitlab.yml file in the project root directory and add the following contents:

unit_test
compile
deploy_test

  • Cache: Cache sets the cache file. The node_module dependency package is cached to improve the efficiency of job construction. It is defined globally and takes effect on all jobs.
  • Stage: Sets the Build and deploy phases
  • Artifacts: Download a file. Define the dist folder produced by Compile to be cached to gitlab server for download (GitLab Web page download) or shared among jobs of the same stage;
  • only: Defines job trigger conditions. You can specify branch names,tagsThese conditions are the relation of or, meet one of the trigger);
  • when: Defines the trigger time of the job. The value can be:on_success,always,manualAnd so on;
  • Dependencies: Defines the jobs that the current job depends on.
  • Environment: defines the environment to which the current job belongs, which is useful for rollback operations.
  • Deploy_test this jobbefore_scriptThere is a long section about configuring SSH password-free login, which will be detailed later.

5. SSH password-free login

5.1 Why do I Need to Log In Without Password?

  • The dist produced by the build is transferred to the target server (test machine/generator), either over the HTTP network protocol or over SSH (or some other file transfer protocol?).
  • HTTP based on the need to write files to receive interface, here directly use rsync based on SSH file transfer, simple, safe!
  • A series of scripts defined in runner are executed in a Docker container, which cannot be manually intervened, so the login server must be made secret free.

First, configure the process of non-secret login server on the local machine (the machine where runner is registered) :

5.2. Generate a pair of public and private keys

Using RSA as asymmetric encryption:

ssh-keygen -t rsa -C "$(whoami)@$(hostname)-$(date -I)"
Copy the code

Note: Enter all the way, remember to enter passphrase directly, so that is no passphrase. If you have to add a password, sorry, hopeless!

5.3. Define SSH Config

In the ~/.ssh/config file, write the following:

Host any_name
  Port your_port
  HostName server_ip
  User user
  IdentityFile ~/.ssh/id_rsa
Copy the code

SSH config file is defined for quick access, just like you configure host, without hostname, you can only access IP. After the configuration, you can log in to the server using SSH any_name. Of course, you will be asked to enter your login password to the server.

5.4. No-secret login

The essence of secret free login is to store the local public key in the authorized_keys file of the target server (this file server does not exist and can be created directly).

ssh-copy-id -i ~/.ssh/id_rsa.pub username@ip
Copy the code

In particular: if your PORT is not the default 22 PORT, add the PORT number -p PORT

5.5. Verify login

ssh any_name
Copy the code

No surprises, you should be able to log directly into the server. So, let’s go back to the configuration of GitLab

Define SSH configuration information on gitLab

Gitlab: Settings >> CI/CD >> Environment variables

  • SSH_PRIVATE_KEYRun the following command to copy the private key: ~/. SSH /id_rsa
  • TEST_CONFIG: copy the information defined in SSH config: ~/.ssh/config
  • TEST_KNOWN_HOST: This variable is defined to allow SSH to authenticate the server (otherwise it would be considered an untrusted environment by SSH). The value of the variable is generated using the following command:
ssh-keyscan -p PORT IP
Copy the code

Use rsync to transfer files

rsync -rve ssh dist/ user@hostname:project_path/dist
Copy the code

Note: Hostname is the Host value you defined in SSH config. Rsync Operation Guide

Define environment

The advantage of setting the environment is that the release environment can be managed, especially for online release, so that bugs can be rolled back in time. To view environments under the current project, click the Preview button to the right of the environment to see how the environment will be released

Environment: test_env

Nine, step on the pit notes

In the whole process of building, many problems are related to SSH login server, select a few frequently occurring problems to explain:

9.1. Host key verification failed

When we log in to the server using SSH for the first time, SSH requires that the remote server be authenticated before the connection is allowed. There are two ways to solve this problem:

  • Set identity exemption: Add a section to SSH ConfigStrictHostKeyChecking noSo,.gitlab.ymlAbout can be removed from the configurationknown_hostsSet;
  • Authentication through the public key fingerprint of the remote server:
ssh-keyscan -p PORT IP
Copy the code

Save the output of the script in the ~/. SSH /known_hosts file (as mentioned in the section of @gitlab defining SSH configuration information). In this way, SSH obtains the public key fingerprint of the target server from this file for authentication before login.

Permission denied, please try again

SSH Password-free login is not configured. For details about how to configure SSH password-free login, see @Password-free Login

9.3. rsync: Failed to exec a: No such file or directory

This problem doesn’t usually occur, but it’s a real problem. I upload the dist directory generated by the build to the server using rsync in the deploy job and throw an error indicating that the dist directory does not exist.

I compile the job and list the files/folders in the root directory (gitlab-runner output) :

release-[timestamp]
release-[timestamp]
.gitlab.yml
artifacts
dist
release-[timestamp] 

cp dist public
Copy the code

In the compile job, copy a dist directory after the build, and give the public directory to artifacts cache.

Refer to the article

  • Use the SSH config file
  • Use sSH-keyscan to obtain the SSH public key fingerprint of cluster machines
  • The Product Managers’ Guide to Continuous Delivery and DevOps