preface

Due to historical reasons, the project was deployed on Ali cloud server, so the company’s internal assembly line was not used for integration and deployment. The assembly line is integrated with GitLab’s OWN CI tool, so I would like to take this opportunity to share GitLab CI/CD with you and teach you how to build an assembly line of a project.

The background of CI/CD

Before we introduce the CI/CD, let’s take a brief look at the background

What is the CI/CD

www.redhat.com/zh/topics/d…

CI (Continuous Integration)

Continuous integration, or continuous integration, is the frequent (multiple times a day) integration of code into the trunk in order to ensure product quality and rapid iteration. It usually requires automated testing to ensure the stability of the integrated code;

CD (Continuous Delivery/Deployment)

Continuous delivery/deployment, seen as the next step in continuous integration, refers to the frequent delivery of new versions of software to quality teams or user testing. If the test passes, the code can be deployed to production.

Why CI/CD?

In the past, a team of developers might work in isolation for long periods of time, merging their changes into the main branch only after their work was completed. This makes it difficult and time consuming to incorporate code changes, and can cause errors to accumulate for a long time without being corrected. These factors make it more difficult to deliver updates to customers quickly.

With CI/CD, we can get the following benefits:

It liberates repetitive labor. Automated deployment frees up repetitive work such as integration, testing, and deployment, and machine integration is significantly more frequent than manual integration.

Fix problems faster. Continuous integration captures changes earlier, gets into testing earlier, finds problems earlier, and the cost of solving problems drops dramatically.

Faster deliverables. Finding errors earlier reduces the effort required to resolve them. The integration server can notify the developer to fix any errors found during the build process. The integration server can roll back to the previous version if it finds an error during deployment, and the server always has an available version.

Reduce manual errors. When it comes to repetitive movements, humans tend to make mistakes, whereas machines make almost no mistakes.

Reduced waiting time. Shorten the time from development, integration, testing, deployment of each link, thus shortening the middle can appear waiting time. Continuous integration means continuous development, integration, testing, and deployment.

Higher product quality. The integration server often provides functions such as code quality inspection, indicating irregularities or errors, and setting up emails and SMS warnings.

GitLab CI/CD

Gitlab-ci is a CI tool provided by GitLab. It can trigger CI processes by specifying actions such as push/merge code, tagging, etc. You can also specify different build scripts to trigger for different scenarios (scripts can be viewed as a step in the pipeline or a single task)

It is used by configuring a.gitlab-ci.yml file in the project root directory to enable its functionality. Let’s take a look at the.gitlab-ci.yml file

Configuration File Introduction

.gitlab-ci.yml uses YAML syntax, which can be interpreted as a JSON-like format, with some syntax differences. Such as:
 tabitha:
    name: Tabitha Bitumen
    job: Developer
     skills:
      - lisp
      - fortran
      - erlang
Copy the code
Corresponding to JSON:
Tabitha: {name: Tabitha Bitumen, Job: Developer, skills: [Lisp, FORTRAN, Erlang],}}Copy the code
The indent corresponds to the key in the JSON object. The indent corresponds to an item in the array

Before formally introducing the.gitlab-ci.yml configuration file, we want to mention a few related concepts in Gitlab CI

Job

A Job can be understood as a single task in a CI process.

Job is a top-level element (equivalent to a root element in the YML configuration) that can be named with any number of names, but must contain at least a script clause specifying the script to be executed by the current task, such as:

job1:
  script:  execute-script-for-job1 
job2:
  stage: build
  script:
    - scripts/build.sh
  only:
    - master
# job n...
Copy the code

Stages

Stages is used to define the Stages of a CI, as follows

stages:
  - build
  - test
  - deploy
Copy the code

At the same time, each stage can be associated with several jobs, that is, a stage can execute multiple jobs in parallel. Use the stage keyword to associate each job with the corresponding stage:

stages:
  - build
  - test
  - deploy
  
build_job:
  stage: build
  script:
    - scripts/build.sh

test_job:
  stage: test
  script:
    - scripts/test.sh

deploy_job:
  stage: deploy
  script:
    - scripts/deploy.sh
Copy the code

Pipeline

Pipeline is the top component of continuous integration, delivery, and deployment. It can be understood as a complete task flow of Pipeline.

A Pipeline can contain stages, and each Stage can specify jobs to be executed so that we can string together the entire build process. These Stages and their corresponding jobs can be seen in GitLab pipelines as follows:

If a task in the Pipeline succeeds, it goes to the Job of the next Stage. Otherwise, if it fails midway, the pipeline execution will be interrupted by default.

Commonly used configuration

Next, we will focus on the common keywords in the.gitlab-ci.yml configuration:

stage

We can use stage to define which stage a job should run in:

stages:
  - build
  - test
  - deploy
Copy the code

script

Used to specify the script commands to be executed by the runer. multiple commands can be specified

job:
  script: 
      - echo  start job! 
      - scripts/deploy.sh
Copy the code

before_script & after_script

Defines what should be executed before/after the job executes the script:

job:
  before_script:
    - echo  Execute this command before the `script` section completes. 
  script:
    - echo  An example script section. 
  after_script:
    - echo  Execute this command after the `script` section completes. 
Copy the code

allow_failure

Used to configure whether the pipeline should continue running when the current job fails:

job2:
  stage: test
  script:
    - execute_script_2
  allow_failure: true # or false (default)
Copy the code

cache

Specify a list of cached files to be shared between different jobs.

Cache :key: Each cache can be assigned a unique identifier key. If this is not set, the default key is default.

Cache: Paths: Specifies the file or directory to cache

rspec:
  script:
    - echo  This job uses a cache. 
  cache:
    key: binaries-cache
    paths:
      - binaries/*.apk
      - .config
Copy the code

only / except

Only is used to define when a job is executed; except is used to define when a job is not executed.

They have four keywords that can be used together:

Ref: matches a regular that matches a branch name or a branch name;

Variables: variables expressions;

Changes: Indicates whether the file in the corresponding path is modified.

Ref: - tags - /^feat-.*$/ - merge_requests variables: - $RELEASE == staging - $STAGING changes: - Dockerfile - docker/scripts/* - dockerfiles/**/* - more_scripts/*.{rb,py,sh} - **/*.jsonCopy the code

retry

Set the number of retries when job execution fails:

Job: script: rspec retry: Max: 2 When: # Retry in the following cases - runner_system_failure - stuck_or_timeout_failureCopy the code

variables

Can be used to define some variables during execution

variables:
  DEPLOY_SITE:  https://example.com/ 

deploy_job:
  stage: deploy
  script:
    - deploy-script --url $DEPLOY_SITE --path  / 

deploy_review_job:
  stage: deploy
  variables:
    REVIEW_PATH:  /review 
  script:
    - deploy-review-script --url $DEPLOY_SITE --path $REVIEW_PATH
Copy the code

when

Conditions for configuring job running:

on_success(Default) : Only all jobs at the previous stage are successful or configuredallow_failure: true;

manual: Runs the job only when it is manually triggered.

always: Runs regardless of the state of the previous stage job.

on_failure: runs only if at least one job of the previous stage fails.

delayed: Delay job execution.

never: Never execute job;

cleanup_build_job:
  stage: cleanup_build
  script:
    - cleanup build when failed
  when: on_failure
Copy the code

Gitlab-runner install & register

After configuring our YML file, we also need to configure Gitlab-Runner to execute the corresponding script.

Installation:

# download # Replace ${arch} with any of the supported architectures arm64 # A full list of architectures can be found here https://GitLab-runner-downloads.s3.amazonaws.com/latest/index.html curl -LJO https://GitLab-runner-downloads.s3.amazonaws.com/latest/rpm/GitLab-runner_${arch}. RPM install RPM -i # GitLab-runner_<arch>.rpmCopy the code

Registration:

The URL & token required for registration is available at GitLab -> Settings -> CI/CD -> Runners

sudo GitLab-runner register

# Enter the GitLab instance URL (for example, https://GitLab.com/):
${url}
# Enter the registration token:
${token}
# Enter a description for the runner:
${description}
# Enter tags for the runner (comma-separated):
${tags}
# Registering runner... succeeded 
# Enter an executor: shell, virtualbox, docker+machine, docker-ssh+machine, docker, docker-ssh, parallels, custom, ssh, kubernetes:
${executor}
# Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!
Copy the code

Create an assembly line

Preview the Pipeline

With the above preparations, we configured.gitlab-ci.yml file, wrote the corresponding script, and configured Gitlab-Runner, then we could start and experience the CI pipeline. GitLab CI Pipeline can be triggered and stages and jobs can be executed 🎉 when the code is submitted (it can also be triggered when the only keyword is set to tag and Mr)

The following figure can be seen in GitLab -> CI/CD -> Pipeline:

Configure a pipeline

The planning is as follows:

Code compilation: provide a build.sh script for compiling code;

Automated tests: the scripts keyword executes the instructions for testing, thus running pre-written automated test scripts;

Manual card points: utilize the above mentionedwhen:manualManual trigger, cooperateallow_failure: false, can achieve the point effect;

Project deployment: also use a script to send our previous build products to the target machine and directory for deployment; At the same time useonly:masterSpecify jobs that perform this step only if they are submitted to the Master branch.

In summary, we can get the following approximate.gitlab-ci.yml configuration (the script corresponding to job can be written according to the actual situation and needs) :

stages:
  - scm
  - test
  - manual-point
  - deploy

scm:
  stage: scm
  script:
    - scripts/build.sh

test:
  stage: test
  script:
    - scripts/test.sh

manual-point:
  stage: manual-point
  script:
    - echo  I am manual job 
  only:
    - master
  when: manual
  allow_failure: false

deploy:
  stage: deploy
  script:
    - scripts/deploy.sh
  only:
    - master
Copy the code

After triggering the pipeline, we can see that after SCM compilation and test automation test steps, the pipeline reaches the manual-point point, and the pipeline is locked for execution, which requires Manual click confirm to continue execution.

Click Execute Job

Timing task

For some scenarios we want to execute Pipeline on a regular basis rather than by event triggering, go to GitLab -> Setting -> Schedule

The interval is set using cron syntax, which is the syntax for setting scheduled tasks in Unix and UniX-like systems.

It uses 5 placeholders to represent the day, month and week of the minute, hour, month and month respectively. As follows:

# ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ minute (0-59) # │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ hour (0-23) # │ │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ day of the month (1-31) # │ │ │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ the month (# 1-12) │ │ │ │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ day of the week (0 to 6) (Sunday to Saturday. # │ │ │ │ │ # * * * * * <command to execute> * * * * * # execute 30 10 * * * # daily 10:30 30 10 * * SAT,SUN # Every Saturday and Sunday 10:30 08 1-20 * * # Every month 1-20 8:00Copy the code

After setting, you can see the scheduled task list:

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

GitLab CI builds an assembly line of its own

preface

The background of CI/CD

What is the CI/CD

Why CI/CD?

GitLab CI/CD

Configuration File Introduction

Job

Stages

Pipeline

Commonly used configuration

stage

script

before_script & after_script

allow_failure

cache

only / except

retry

variables

when

tags

Gitlab-runner install & register

Create an assembly line

Preview the Pipeline

Configure a pipeline

Timing task

GitLab CI builds an assembly line of its own

preface

The background of CI/CD

What is the CI/CD

Why CI/CD?

GitLab CI/CD

Configuration File Introduction

Job

Stages

Pipeline

Commonly used configuration

stage

script

before_script & after_script

allow_failure

cache

only / except

retry

variables

when

tags

Gitlab-runner install & register

Create an assembly line

Preview the Pipeline

Configure a pipeline

Timing task

Related Posts

Front-end Nginx stuff

Not knowing the front end of physics is not a good game developer

Vue long connection, using both SockjS-client and STOMpJS