Gitlab CI/CD is introduced

Gitlab CI/CD is a tool for continuous integration (CI), continuous delivery (CD), similar tools include Jenkins, Travis CI, GoCD, etc.

Continuous Integration, or Continuous Integration, is the process of automatically detecting (code Lint), building, and unit testing after source code changes (Git push). The goal of Continuous Integration is to quickly ensure that new code submitted by developers is good (less buggy). And suitable for further use in your code base.

Continuous Delivery, or Continuous Delivery, usually refers to the entire process chain (pipeline) that automatically monitors source code changes and runs them through build, test, package, and related operations to produce a deployable version (either APK package or web site deployment) with virtually no human intervention. It includes continuous integration, continuous testing (ensuring code quality), and continuous deployment (automatically releasing a version for users to use).

Gitlab CI/CD is relatively simple, just rely on a “.gitlab-ci.yml”, upload the file along with the code, Gitlab will automatically perform the corresponding task, so as to achieve CI/CD.

Gitlab installation

To use Gitlab’s CI/CD capabilities, there are several general options:

  • First, build a set of Gitlab by yourself and deploy your own Runner (i.e., the machine that actually executes the code). Docker deployment is recommended
  • Second, use the company/official Gitlab, and use the built-in Runner, namely shared Runner, as shown in the following figure.

Link: juejin. Im/post / 684490… www.xuxusheng.com/post/ using the Dock… Docs.gitlab.com/runner/inst…

Gitlab – ci writing

The.gitlab-ci.yml follows the syntax of the YAML file, which keeps track of the instructions you want to execute for specification checking (e.g. PEP8), auto-packaging (e.g. Android auto-packaging), auto-deployment, etc.

For beginners, if you don’t know if your.gitlab-ci.yml has errors, you can check with Gitlab’s own CI Lint.

This article uses the version GitLab Community Edition 12.5.6-EE

The official documentation

Docs.gitlab.com/ee/ci/yaml/…

image

This keyword specifies the docker image used by a job. For example, image: python:latest Uses the latest Python image.

Image download policy:

  • Never: When using this policy, Gitlab Runner is prohibited from pulling down images from Docker Hub or elsewhere, and can only use the images that are pulled down manually by Gitlab Runner
  • If-not-present: When this policy is used, the Runner checks whether there is a mirror on the local, uses the mirror if there is, and then drops down if there is not. If this strategy is combined with periodic deletion of images, a better result can be achieved.
  • Always: This is the default strategy used by Gitlab-ci, which is to pull down the image again every time. As a result, it takes a lot of time

Link:docs.gitlab.com/runner/exec…

services

This keyword points to other Docker images that will be bound to the image specified by the image keyword. For example, you can bind a mysql service to store fake data needed for unit tests.

default_job:
  image: Python: 3.6

  services:
    - mysql:5.7

  before_script:
    - apt-get install mysql-client
Copy the code

Before_script and after_script

The before_script keyword defines a set of commands to be executed before each task starts, and the after_script is the opposite. For example, you can prepare SSH connections in before_script, as shown below.

Note: before_script can be used for all tasks or a single task.

Gitlab-ci Implements SSH connections

2. Write the gitlab-ci file as follows:

image: Python: 3.6

before_script:
  # which ssh-agent is used to find whether ssh-agent exists,
  - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
  # 启动ssh-agent
  - eval $(ssh-agent -s)
  Set environment variable SSH_PRIVATE_KEY and add the private key to the Agent Store
  - echo "$SSH_PRIVATE_KEY" | tr -d '\ r | SSH - add - # modify permissions - mkdir -p ~ /. SSH && chmod 700 ~ /. # SSH over SSH ask -' [[- f /. Dockerenv]] && echo - e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/. SSH /config' *\n\tStrictHostKeyChecking no\n\n"ssh-test:
  script:
    - git clone [email protected]
    Remote to another machine and execute the command
    - ssh [email protected] "cd /root && mkdir your_dir"
    If not default port is used
    - ssh -p 5555 [email protected] "cd /root && mkdir your_dir"
Copy the code

Link: docs.gitlab.com/ee/ci/ssh_k… Zacksleo. Making. IO / 2017/04/14 /…

stages

As mentioned earlier, we can define a set of tasks (jobs) to perform the instructions we want, but to specify the order of tasks, we need to use the stages keyword.

The stages keyword has two features: 1. If two tasks have the same stage name, the two tasks are run in parallel 2. Tasks associated with the next stage will continue to run after the previous stage succeeds, and will not run if the previous stage fails

image: Python: 3.6

stages:
  - build
  - test
  - deploy
# build-job1 and build-job2 are executed in parallel
build-job1:
  stage: build
  script:
    - echo $PWD

build-job2:
  stage: build
  script:
    - echo $PWD

This task will be run after build-job1 and build-job2 have been successfully executed
test-job:
  stage: test
  script:
    - echo $PWD

# Notice that all defined stages are needed
deploy-job:
  stage: deploy
  script:
    - echo $PWD

Copy the code

The renderings are as follows:

Note: If you want to control whether a stage is executed at the beginning or at the end, you can use the.pre and.post keywords

only / except

These are two fields that I love and hate, not one.

To better illustrate, we need to introduce the concept of Pipelines, which are the rows of records we see in Gitlab’S UI interface:

As shown in the figure above, there are three Pipelines. Inside each Pipeline is the execution of the tasks defined by us. It is as simple as that.

As mentioned earlier, the stages keyword controls the order in which tasks are executed, while the only/except keyword controls the triggering conditions of tasks.

The only/except keyword contains a number of sub-keywords (or policies if you can think of them as such). Execution of Pipelines is triggered only when the defined policy is fulfilled; except is the opposite.

The default policy for the only keyword is [‘branches’, ‘tags’], when you have submitted or tagged a branch, it will trigger,

In addition, the only/except keyword can be divided into basic features and advanced features, and contain different sub-keywords. It is recommended to read the official document carefully.

Only/except (basic) :

  • Branches: Triggered when your Git Refs correspond to a branch
  • Tags: Triggered when your Git Refs correspond to a tag
  • Intermediate: When you usegit pushTriggered when
  • Web: When you use the Web interfaceRun PipelineTriggered when
  • Merge_requests: Fired when you create or update a merge_requests
  • .

only/except(advanced):

  • Refs: Pay attention to exclusivity or relationships
  • variables
  • changes
  • kubernetes

Here are some examples that may help you

1. Pipelines are fired only when the submitted branch is master

image: Python: 3.6

# The first way
job1:
  only:
    - master
  script:
    - echo $PWD

# Second way
job2:
  only:
    refs:
      - master
  script:
    - echo $PWD

# Third way
job3:
  only:
    variables:
      - $CI_COMMIT_REF_NAME = = "master"
  script:
    - echo $PWD
Copy the code

2. Open Pipelines only when Tag is played (Not normal submission)

image: Python: 3.6

# The first way
job1:
  only:
    - tags
  script:
    - echo $PWD
  
# Second way
job2:
  only:
    refs:
      - tags
  script:
    - echo $PWD
Copy the code

3. Open Pipelines when the submitted branch is master or tags are played

image: Python: 3.6

This example is used to show that the relationship between the only keyword is "or"
job1:
  only:
    - master
    - tags
  script:
    - echo $PWD
Copy the code

4. When the submitted branch is master and tags are played to trigger Pipelines, no way can be found at present, it should be implemented with rules keyword

5. Trigger when using Run Pipelines in the Web interface (personally I think this way is very useful)

image: Python: 3.6

This example is used to show that the relationship between the only keyword is "or"
job1:
  only:
    - web
  script:
    - echo $PWD
Copy the code

Link: git refs: git-scm.com/book/zh/v2/… Regular matching: stackoverflow.com/questions/5…

tags

This keyword specifies which Runner (which machine) to use to perform our task, distinguishing it from the tags of the only keyword above.

image: Python: 3.6

This example is used to show that the relationship between the only keyword is "or"
job1:
  only:
    - dev
  tags:
    - machine1
  script:
    - echo $PWD
Copy the code

when

As mentioned earlier, the “stages” keyword can control the execution order of each task, and the latter stage will wait for the previous stage to successfully execute before executing. What if we want to achieve the effect that the latter stage can still execute if the previous stage fails?

This is where the when keyword comes into play. It has five values:

  • On_success: Only performed if all the work of the previous stages was successful, which is the default.
  • On_failure: the task is executed after any job in the current stages fails
  • Always: Perform regardless of the previous stages jobs state
  • “Manual” : Manual execution
  • Delayed: Delayed execution
# Official example
stages:
  - build
  - cleanup_build
  - test
  - deploy
  - cleanup

build_job:
  stage: build
  script:
    - make build

# If the build_job task fails, the task execution will be triggered
cleanup_build_job:
  stage: cleanup_build
  script:
    - cleanup build when failed
  when: on_failure

test_job:
  stage: test
  script:
    - make test

deploy_job:
  stage: deploy
  script:
    - make deploy
  when: manual

# Always execute
cleanup_job:
  stage: cleanup
  script:
    - cleanup after jobs
  when: always
Copy the code

cache

This keyword specifies the folder or file to cache in order to speed up execution.

It is important to note that cache cannot cache files outside the working path. Your user is yangan, for example, is the helloworld project, the default working directory is/build/yangan/helloworld, if you want to cache/under the root of a file, will appear “can’t find file” error.

In addition, cache does not guarantee a hit every time, and if you cache a large number of files, sometimes the opposite can happen, resulting in slower speeds.

image: Python: 3.6

cache:
  # Relative path
  paths:
    - cache_dir/


job1:
  script: echo $PWD
  For a single task
  cache:
    - binaries/

Copy the code

Relative paths: stackoverflow.com/questions/5…

artifacts

Similar to the cache keyword, you can also cache files or folders. The difference is that these files can be downloaded from the Gitlab UI and are generally used to store the ApK generated by Android packaging.


image: Python: 3.6

job1:
  artifacts:
    paths:
      - binaries/
    # Set expiration time
    expire_in: 1 week

Copy the code

variable

Gitlab-ci has several variables, which are predefined environment variables, such as CI_COMMIT_SHA, CI_COMMIT_MESSAGE; The variables you set through the Web interface Settings/ ci-cd, and the variables you define in.gitlab-ci.yml.

Through the pre-defined environment variables, we can get the branch name, the submitted message, so as to determine whether the task needs to be executed; The Web Settings/ CI-cd variables are ideal for storing inconvenient data such as public and private keys, accounts and passwords. Only Owner and Maintainer can view.

The following figure defines an SSH_PRIVATE_KEY to store the private key:

Link: key words: the variables docs.gitlab.com/ee/ci/varia… Gitlab predefined variables: docs.gitlab.com/ce/ci/varia… The environment variable: docs.gitlab.com/ce/ci/varia…

Problems encountered

1.! = This symbol is only supported in versions 11.11 and later

2. Through the API to delete Pipelines: docs.gitlab.com/ee/api/pipe… Gitlab.com/gitlab-org/…

reference

What is CI/CD?

Recommend some of the top open source CI/CD tools

Gitlab-ci Configuration (2)

Debug a running task

Use of tag tags in Git