The author:Clfeng

Learners’

CICD stands for Continuous Integration, Continuous Delivery, and Continuous Deployment. The automatic execution of a series of scripts during the development process to reduce the probability of introducing bugs and minimize human intervention in the process of new code from development to deployment. I believe that we often come into contact with it at work, but maybe due to the nature of the position, although we use it every day, we don’t know much about it. When something goes wrong with the CI/CD assembly line, we can only ask for help. To get out of this dilemma, let’s take a closer look at CI/CD through a series of articles.

This chapter will take us step by step from the initial project creation to CI/CD environment installation to a simple pipeline to let us experience how to build your OWN CI/CD. At the end, we will explain the basic concepts and give us a clear picture of the first assembly line we wrote.

Create the GitLab project

Environmental installation

Note: In the enterprise, most of the runners use The Linux environment as CI/CD, so the following environment is installed using Linux;

Operating system information:

Linux Version 3.10.0-1160.el7.x86_64 ([email protected]) (GCC version 4.8.5 20150623 (Red Hat 4.8.5-44)  (GCC) ) #1 SMP Mon Oct 19 16:18:59 UTC 2020Copy the code

Note: The author’s Linux environment is the centos:7 system installed on the local macOS through a VIRTUAL machine

Readers do not need to buy a server, use the local host to install Linux system through virtual machine or use the local macOS or Window as runner, but Windows is not recommended, after all, there will be a lot of symbols to pay attention to when writing CI configuration files

Install the docker

The following runner will use docker as the executor, so let’s first install docker on the system.

The installation can be carried out in accordance with the website tutorial docker docs.docker.com/engine/inst…

Basic installation process:

1. Remove the old docker

sudo yum remove docker \
                docker-client \
                docker-client-latest \
                docker-common \
                docker-latest \
                docker-latest-logrotate \
                docker-logrotate \
                docker-engine
Copy the code

2. The Docker installation starts

sudo yum install -y yum-utils
sudo yum-config-manager \
    --add-repo \
    https://download.docker.com/linux/centos/docker-ce.repo

#Install the Docker engine
sudo yum install docker-ce docker-ce-cli containerd.io
Copy the code

3. Start the docker

sudo systemctl start docker
Copy the code

4. The installation is successful

Install the runner

Docs.gitlab.com/runner/inst…

1. Execute the following script to install gitlab-Runner

#1. Add the official GitLab warehouse
curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.rpm.sh" | sudo bash

#2. Install gitlab - runner
sudo yum install gitlab-runner
Copy the code

2. The installation is successful

Registered runner

Enter the CI/CD page of the project repository to view the corresponding URL and token

performgitlab-runner registerCommand and follow the prompts to register runner

Once registered, check for available runners

At this point, we are ready to learn the basic environment for CI/CD

And then we try to write a simple pipeline

Note: it is not necessary to understand the contents of.gitlab-ci.yml for the time being

The first assembly line

Add the pipeline configuration file

Add the.gitlab-ci.yml file to the root directory of the project and push it to the remote branch with the following contents:

# .gitlab-ci.yml
stages:
  - build
  - test
  - deploy

build_job:
  stage: build # specify stage to which the job belongs
  image: centos:7 # specify the docker image used to execute the job
  tags: 
    - clf-centos-runner # specify the runner that executes the job.
  script: Script run during job execution
    - echo "build project"

test_job:
  stage: test
  image: centos:7
  tags: 
    - clf-centos-runner
  script:
    - echo "test project"

deploy_job:
  stage: deploy
  image: centos:7
  tags: 
    - clf-centos-runner
  script:
    - echo "deploy project"
Copy the code

View pipeline execution

Understand basic concepts

In the first Pipeline section, we added.gitlab-ci.yml and wrote a bunch of configurations. Then submit the file and push it to GitLab and you can see the pipeline running. So what do we mean by the configuration we wrote earlier in the.gitlab-ci.yml file? Pipeline configuration and how to write?

To write a basic pipelining configuration, we need to learn the following:

  • Yaml syntax
  • What is the CI/CD
  • Key concepts of CI/CD
    • pipeline
    • stage
    • job

Yaml syntax

CI/CD pipeline configuration files are written using YAML syntax, so you need to understand the syntax first. Ruan Yifeng’s article is recommended for learning:

www.ruanyifeng.com/blog/2016/0…

What is the CI/CD

CI/CD can be split into CI and CD, where CI is continuous integration and CD is continuous delivery and continuous deployment. CI/CD is a way to frequently deliver applications to customers by introducing automation during the application development phase.

  • Continuous Integration (CI)
  • Continuous Delivery (CD)
  • Continuous Deployment (CD)

Of course, this introduction may be a little simple, so excerpted below more detailed introduction, interested can have a look

Continuous integration

The goal of modern application development is to have multiple developers working on different features of the same application simultaneously. However, if an enterprise schedules the merging of all branch source code in one day (called “merge day”), it can end up being cumbersome, time-consuming, and manual. This is because when a single developer working independently makes changes to an application, it is possible to conflict with changes made at the same time by other developers. If each developer customizes his or her own local integrated development environment (IDE), rather than having the team agree on a cloud-based IDE, the problem is exacerbated.

Continuous integration (CI) helps developers merge code changes into a shared branch or “trunk” more frequently (sometimes even daily). Once the changes made by developers to the application are merged, the system verifies the changes by automatically building the application and running different levels of automated testing (usually unit and integration tests) to ensure that the changes have not broken the application. This means that the test covers everything from classes and functions to the different modules that make up the entire application. If automated tests find conflicts between new code and existing code, CI can make it easier to quickly fix those errors.

Continuous delivery

Continuous delivery automatically publishes validated code to the repository after completing the automated process of building and unit and integration tests in CI. To achieve an efficient continuous delivery process, it is important to ensure that CI is built into the development pipeline. The goal of continuous delivery is to have a code base ready to deploy to production.

In continuous delivery, each phase, from the consolidation of code changes to the delivery of production-ready builds, involves test automation and code release automation. At the end of the process, the operations team can quickly and easily deploy the application into production.

Continuous deployment

The final stage for a mature CI/CD pipeline is continuous deployment. As an extension of continuous delivery, which automatically releases production-ready builds to code repositories, continuous deployment can automatically release applications to production. Since there is no manual gating during the pipeline phase prior to production, continuous deployment relies heavily on well-designed test automation.

In practice, continuous deployment means that a developer’s changes to an application can take effect within minutes of being written (assuming it passes automated testing). This makes it easier to continuously receive and integrate user feedback. In sum, all of these CI/CD related steps help reduce the deployment risk of the application, making it easier to release changes to the application in small pieces rather than all at once. However, because automated tests need to be written to accommodate various testing and release phases in the CI/CD pipeline, the upfront investment is still significant.

Personal summary:

Continuous integration: High-frequency integration of code into the main trunk, triggering single and integration tests before integration to verify code changes and ensure that the changes do not break the application

Continuous delivery: Code is aggregated into the code repository. The goal is to have a code base ready to deploy to production

Continuous deployment: At the end of the process, the operations team can quickly and easily deploy the application into production

Key concepts of CI/CD

From the official website:

Pipelines are the top-level component of continuous integration, delivery, and deployment.

  • Jobs, which define what to do. For example, jobs that compile or test code.
  • Stages, which define when to run the jobs. For example, stages that run tests after stages that compile the code.

In general, pipelines are executed automatically and require no intervention once created. However, there are also times when you can manually interact with a pipeline.

The translation:

Pipelines are the top component for continuous integration, delivery and deployment.

  • The Job,What do you do. For example, a job to compile or test code.
  • Defined Stages,whenRun the job. For example, the phase of running tests after the phase of compiling code.

In general, pipes are executed automatically and no intervention is required once they are created. However, sometimes you can interact with pipes manually.

understand

Pipelines are the top component of CI/CD; when a pipeline is created, it automatically executes a series of tasks without human intervention. There are several subcomponents, job and stages, underneath the pipeline component. “Stages” defines how many stages (steps) there are in a pipeline and how these stages (steps) are implemented in a proper order. Job refers to specific tasks in each link [multiple tasks can be performed in one stage (link)]. When defining a task, we should clearly describe the circumstances under which the task will be performed and what it will do.

Let’s take a look at configuration in The First Pipeline:

stages:
  - build
  - test
  - deploy

build_job:
  stage: build # specify stage to which the job belongs
  image: centos:7 # specify the docker image used to execute the job
  tags: 
    - clf-centos-runner # specify the runner that executes the job.
  script: Script run during job execution
    - echo "build project"

test_job:
  stage: test
  image: centos:7
  tags: 
    - clf-centos-runner
  script:
    - echo "test project"

deploy_job:
  stage: deploy
  image: centos:7
  tags: 
    - clf-centos-runner
  script:
    - echo "deploy project"
Copy the code

The above configuration is a complete pipeline configuration. How are pipeline stages and jobs reflected in this configuration?

First of all, when defining a pipeline, we need to consider how many stages this pipeline has and what is the sequence of the stages? Take the first pipeline as an example:

There are three phases: Build, test, and deploy. Together, these three phases describe what needs to happen after the code is loaded into the library: code construction, testing, and deployment.

Reflected in the configuration is:

stages:
  - build
  - test
  - deploy
Copy the code

The order is the order of each item in the array: build => test => deploy.

After defining each phase, we need to further define jobs for each phase, taking the Build phase as an example:

stages:
  - build

build_job: # This is the name of job
  stage: build # specify stage to which the job belongs
  image: centos:7 # specify the docker image used to execute the job
  tags: 
    - clf-centos-runner # specify the runner that executes the job.
  script: What does job do
    - echo "build project"
Copy the code

As mentioned earlier, we need to define a job for each phase. So how does job correspond to the corresponding stage? Corresponding by the stage keyword:

stages:
  - build

build_job: # This is the name of job
  stage: build # specify stage to which the job belongs
Copy the code

The value build of stage corresponds to one of the values build defined in the stages array. This indicates that build_job is a task in the build phase.

We also said that job defines what to do, so what does the job build_job do? Look at the script keyword:

stages:
  - build

build_job: # This is the name of job
  stage: build # specify stage to which the job belongs
  script: What does job do
    - echo "build project"
Copy the code

Emmm, simply type “Build project”.

Several other configurations appear in the previous configuration, and we’ll also take a look at their respective roles

Tags: This field specifies the runner (machine) used to execute the job. We specify the runner previously registered with the name clF-centos-runner

Image: Docker is used by the executor of our job. This field specifies the image used by the docker

At this point, I believe we have a general understanding of the configuration of the First Pipeline; As a whole:

Build => test => deploy; We defined a task for each phase. Each task specifies the machine used to execute the task (Runner) and the mirrored centos used by the executor :7. Currently, the execution task of each job is simply to output the corresponding text.

Hold on, there’s something missing. Why do we see the pipeline as soon as we write the.gitlab-ci.yml file is committed and pushed? How did Gitlab know to perform the assembly line at this time? Gitlab-ci. yml file will trigger pipeline? Then I have written it now and will not modify the.gitlab-CI file. Can the assembly line still be triggered?

This involves the pipeline trigger mechanism, which is generally triggered whenever code is pushed. When the code is pushed to the remote GibLab repository, GibLab looks for the pipeline configuration file.gitlab-ci.yml, which triggers the pipeline if it is present (of course we may have configured the pipeline in the configuration so that it is not created). Besides code push, there are other ways to trigger pipelining. For example, scheduled tasks, manual triggering, and invoking GibLab apis are triggered.

conclusion

We have already created a test project, installed the environment, written our first pipeline and understood the basic concepts of CI/CD. I believe that you have a relatively complete understanding of CI/CD. Later chapters will delve into the core concepts of CI/CD.

Refer to the link

Docs.gitlab.com/ee/ci/index…

www.redhat.com/zh/topics/d…

www.ruanyifeng.com/blog/2016/0…