What is Gitlab CI?

Modern continuous software development has resulted in developers having to constantly build, test, and deploy code changes. These repetitive processes are tedious, but important to ensure that code is constantly updated and iterated. There is a need for a non-manual way to automate these repetitive tasks.

This idea of auto-completion is called CI/CD. In this case, Gitlab CI/CD is a service released by Gitlab to help developers finish repetitive tasks.

For example, byte’s internal tool E*** CI is improved on the basis of Gitlab CI and Codebase CI, adding more detailed classification of usage scenarios and more friendly support for Monorepo.

Two, how to use?

The use of Gitlab CI mainly requires two steps

Step 1: Configure the Runner

We can simply understand Gitlab Runner as the implementer of. Gitlab-ci. yml file content,.gitlab-ci.yml tells Gitlab Runner what to do.

Gitlab Runner is not a configuration item, it needs to be specially deployed, such as deploying a Runner image with Docker to a container that can connect to the Intranet. You can also use the company’s shared Runners.

Use a public runner, or create one yourself

  1. Using shared runner:

Make sure that the shared Runner option is enabled to use the public Runner directly without further action. (below)

  1. Configure runner yourself:

Deploy Gitlab runner official document: Run Gitlab runner in a container | Gitlab

Associated runner to gitlab official document: Registering runners | gitlab

A. install docker:

$ brew install --cask docker
Copy the code

B. Start Gitlab Runner Container:

Create a Docker Container to perform the Gitlab Runner image

  docker run -d --name gitlab-runner --restart always \
   -v /Users/Shared/gitlab-runner/config:/etc/gitlab-runner \
   -v /var/run/docker.sock:/var/run/docker.sock \
   gitlab/gitlab-runner:latest
Copy the code

Note: The Shared folder on macOS should be set to /Users/Shared/ instead of/SRV

C. Register runner to Gitlab:

Using only Docker to create a GitLab Runner cannot associate the gitLab Repository we are currently using with the runner created in the previous step. At this point, it is necessary to correlate them and register the runner in our GitLab Repository

Replace the — URL and –registration-token of the directive below with the URL and –registration-token obtained in the figure above

— Tag-list is the tag of the current runner. It is used when configuring pipeline jobs. You can configure different tag names

docker run --rm -v /Users/Shared/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner register \ --non-interactive \ --executor "docker" \ --docker-image alpine:latest \ --url "https://gitlab.com/" \ --registration-token "wC-reg7qupGRb4lw3q9y" \ --description "gitlab -- cicd-runner" \ --tag-list "yehanli,liyehan" \ --run-untagged="true" \ --locked="false" \ --access-level="not_protected"Copy the code

So far, Gitlab Runner and configuration have been successful, open setting/CICD page, you can see the following prompt:

We can also see the runner running in docker (as follows):

This content cannot be displayed outside of the flying book document at this time

Step 2: Create.gitlab-ci.ymlfile

Create a file in the root directory of the Mono repo named.gitlab-ci.yml and push it to the Master branch.

git add .gitlab-ci.yml
git commit -m "Add .gitlab-ci.yml"
git push origin master
Copy the code

Note:

  • In lower GitLab versions (e.g. 11.4), if not under the master main branch.gitlab-ci.yml– Gitlab Runner will not have a TAB in the left navigation bar.

  • < span style = “box-sizing: border-box; word-break: inherit! Important; word-break: inherit! Important

How to compile.gitlab-ci.yml file?

First, let’s familiarize ourselves with the common way yamL is written and see how it differs from JSON.

For details on how to write Yaml Syntax, see => Yaml Syntax – Ansible Documentation

YAML syntax:

Array notation:

{
    "array": ["red", "blue", "yellow"]
}
Copy the code

Convert to YAML

array:
  - red
  - blue
  - yellow
Copy the code

Object writing:

{" student1 ": {" name" : "xiao Ming"}, "array" : [" red ", "blue", "yellow"], "student2:" {" name ":" xiao Ming "}}Copy the code

Convert to YAML

Student1: name: student2: name: red - blue - yellow student2: name: red - blueCopy the code
  • Unlike JSON, which is unannotated, yamL can be used#annotate

Assembly line link combing:

According to the configured.gitlab-ci.yml file, the flow of the pipeline can be divided into the following steps:

  1. Configure before individual pipelined tasks are executed
  1. Define a single pipeline job and configure it accordingly

Structure:

For the form of a single pipeline task, please refer to the following example. The specific field definitions can be ignored for the time being and will be explained in detail later:

My_job: only: -master tags: -yehanli script: -echo 'job ========= complete 'stage: build retry: 2Copy the code

Keywords: Gitlab-CI

There are many keywords in Gitlab CI, but only a small number are actually used. Common keywords are explained in detail in this section.

If you need any special configuration can be the reference document keywords = > Keyword reference for the. Gitlab – ci. Yml file | gitlab

Let’s first introduce keywords and then look at a complete configuration demo for details.

script

The script that needs to be run. It is configured as an array.

pipeline_job: ...... Script: -cd < folder directory path > -npm install -npm build......Copy the code

before_script

Scripts that need to be executed before all pipeline tasks are executed, or scripts that are executed before scripts in a single pipeline task are executed

The variables variables

In GITlab-CI, variables can be roughly divided into three categories:

  1. Gitlab gives us predefined variables, such as CI_COMMIT_BRANCH.

    1. Predefined variables reference | GitLab
  1. Setting => Gitlab CI/CD => variables

  1. In gitlab – ci. Yml defined in the variable (as sample) variables gitlab ci/CD | gitlab
variables:
  TEST_VAR: "All jobs can use this variable's value"

job1:
  variables:
    TEST_VAR_JOB: "Only job1 can use this variable's value"
  script:
    - echo "$TEST_VAR"
  ......
Copy the code

image

CI pipeline environment docker image (any related code environment image is ok), such as byte some internal tool E *** image, which contains the NVM manager.

  • Node, Python, Java, etc…

stages

Gitlab CI allows us to customize pipeline stage configuration. We can divide our pipeline into stages and do different things on different stages. (Examples will follow later)

Stages are executed sequentially. If the tasks of the previous stage are not completed, subsequent stages are not triggered.

If a task in a stage fails, the task in the next stage will not be executed. If the current stage defines multiple tasks, if one task fails, the other task will continue to execute.

The following are custom stages:

stages:
  - first_stage
  - second_stage
  - third_stage
  - fourth_stage
  - fifth_stage
Copy the code

If there is no custom stages, the default stages is. Pre => build => test => deploy =>.post

(Custom stages will override all default stages except. Pre and. Post)

Stages: -. Pre # Run before all stages - build-test-deploy -. Post # run after all stagesCopy the code

Note: If multiple jobs are defined in a build, they are executed in parallel, and in other stages they are executed in sequence.

stage

Stage is a subitem of Stages, which can be configured when you customize a single flow line task

cache:

Caches files, directories, etc that are shared between multiple pipeline tasks.

Cache: key: cache-node-modules # Write the path to the file that needs to be cached, in this case node_modules paths: -node_modulesCopy the code

retry:

Maximum number of jobs that can be re-executed. It has to be a positive integer, 0 minus 2, 2 being the maximum

  • When can be set to execute in the case of a specific cause of failure
job:
  script: rspec
  retry:
    max: 2
    when: runner_system_failure
Copy the code

only & except

  • Only: Sets the execution time of pipeline tasks
  • Except: Sets the time when pipeline tasks are not executed

Configurable options (several commonly used) :

< branch name > Triggered when a particular branch is changed
pushes Triggered when Git push applies to all branches
merge_requests Triggered when MR is created. Applies to all branches
web Trigger when run Pipeline is clicked manually in Gitlab Runner/ Pipeline

Configure the branch name to specify the branch that triggers the task (as follows),

Myjob: only: variables: $CI_COMMIT_REF_NAME == "master" - $CI_COMMIT_REF_NAME == "master"Copy the code

rules:if

This field can be configured in a single pipeline job or Workflow field.

Under the rules keyword, you can configure if statements, and if so, you can perform some custom configurations (shown in the Pipeline task Demo2 and Workflow configuration).

rules:
  - if: $CI_COMMIT_REF_NAME =~ /feature/
Copy the code

Note: Only & except and rules:if are used to determine the execution time of a single job. Only one job can be configured. Otherwise, an error is reported.

workflow

Rules need to be used in conjunction with rules to control the behavior of the entire pipeline, such as whether or not the entire pipeline executes. The pipeline must fire at least one if statement in its rules configuration.

Configure in the outer layer of each pipeline task

variables:
  IS_FEATURE: "false"
workflow:
  rules:
    - if: $CI_COMMIT_REF_NAME =~ /feature/
      variables:
        IS_FEATURE: "true"
    - when: always # Run the pipeline in other cases
Copy the code

when

This keyword needs to work with stage. If one stage fails, what should we expect the next stage to do?

(When to run a job?)

  • on_success(default): The task of the current stage is executed only after all tasks of the previous stage are successfully executed, or tasks of the previous stage fail are configuredallow_failure: true
  • on_failure: Perform the current task only if the previous pipeline task has failed at least once.
  • always: The current task will be triggered regardless of the state of the pipeline task in the previous stage.
  • never: Does not run the current task
  • manual: The pipeline is triggered manually, click Play, as shown below

  • delayed: Delays execution for a period of time
timed rollout 10%: stage: deploy script: echo 'Rolling out 10% ... 'When: delayed # Run this task 30 minutes after the previous stage execution # The time unit can be seconds, minutes, day, week start_in: 30 minutesCopy the code

tags

This is to specify which runner Gitlab will use when executing the script

Matters needing attention:

Error message XXX config conatins Unknown keys is displayed for keywords such as Workflow and rules.

This is because the company’s version of Gitlab is older and Gitlab does not support these configurations until 12.5.

e.g:

The merge_requests configuration in only was not supported until 11.6, so some companies still don’t support it in Gitlab 🙁

Modular writing

As the number of pipelining tasks increases, writing all of them in a.gitlab-ci.yml file becomes bloated

The solution is to split multiple tasks into different modules

To import different YML files in the.gitlab.yml file, write as follows

include:
  - '/yml/job_1_install.yml'
  - '/yml/job_2_build.yml'
  - '/yml/job_3_deploy.yml'
Copy the code

4. Demo Configuration + Running examples

Let’s run an example to see how file configuration works

Pipeline Demo1(Basic usage) :

#.gitlab-ci.yml # image is the image of the node environment, and can be changed to another project environment if you are using another environment. - first_stage-second_stage-third_stage-fourth_stage-fifth_stage # Custom task 1 Job_1_push: only: - Enlish # set Shared runner tags: -Yehanli # Script that is required to be executed in the current task - echo 'job1 ========= complete '# set stage of the current task stage: first_stage # User-defined task 2 Job_2_push: only: - Enlish tags: -Yehanli script: -echo 'job2 ========= Completes' stage: third_stage # User-defined task 3 Job_3_push: only: - Enlish tags: -yehanli script: -echo 'job3 ========= complete 'stage: fourth_stage # Set the current task to manually triggered when: manual # Define the task 4 job_4_push: only: - Enlish script tags: -yehanli script: -echo 'job4 ========= complete 'stage: fourth_stage When: always # Custom task 5 job_5_web: - Web tags: -yehanli script: -echo 'job5 ========= complete 'stage: fifth_stageCopy the code

Job3 set for manual execution requires us to manually click the pipeline for execution.

The output results of the first 4 Jobs are shown below (click each Jobs in the figure above to see the output below).

As expected, all jobs did what they were set to do.

As shown in the figure, we did not define the second_stage task, so the default is skipped and handled as success

Job5 can be triggered in the following ways:

  1. Pipeline interface click Run Pipeline

  1. Select Branch, then click Run Pipeline

The following is the output

Pipelining Demo2(detailed example) :

#.gitlab-ci.yml image: node:latest # # Run the following script before running all tasks: before_script -echo '====== executing =========' variables: is_experiencing_complex: "false" IS_ON_MYBRANCH: "false" workflow: rules: - if: $CI_PIPELINE_SOURCE == "merge_request_event" variables: IS_EXPERIENCING_MERGING: "true" - if: $CI_COMMIT_REF_NAME =~ /myBranch/ variables: IS_ON_MYBRANCH: "true" - if: $CI_COMMIT_REF_NAME =~ /.*/ cache: key: Cache-node-modules # Write the path to the file that needs to be cached. - first_stage - second_stage - third_stage - fourth_stage - fifth_stage include: - '/yml/job_1_install.yml' - '/yml/job_2_build.yml' - '/yml/job_3_deploy.yml' - '/yml/job_4_mybranch.yml' - '/yml/job_5_beforeMR.yml'Copy the code
# '/ yML /job_1_install.yml' # install stage Job_1_install: only: -master tags: -yehanli before_script: - echo '= = = = = = = = = = before job1 is script execution = = = = = = = = = =' - NPM install script: - echo 'job1 is = = = = = = = = = complete' stage: First_stage # Maximum 3 retry times: 2Copy the code
# '/yml/job_2_build. Yml '# build stage job_2_build: only: -master tags: -yehanli script: -npm run build - echo 'job2 ========= Finish 'stage: second_stage retry: 2Copy the code
Yml '# deploy phase job_3_deploy: image: docker only: -master tags: -yehanli script: - docker build -t reactimages . - if [ $(docker ps -aq --filter name=react-container) ]; then docker rm -f react-container; Fi-docker run -d -p 800:80 --name react-container reactimages -echo 'job3 ========= complete 'stage: third_stage when: alwaysCopy the code
# '/yml/job_4_mybranch.yml' job_4_mybranch: only: - myBranch tags: - yehanli script: - echo 'is it on myBranch? '$IS_ON_MYBRANCH - echo 'job4 ========= complete' stage: fourth_stage Retry: 2Copy the code
# '/yml/job_5_beforeMR.yml' job_5_afterMR: rules: - if: '$CI_COMMIT_BRANCH == "master"' tags: - yehanli script: - echo 'Is experiencing merging? '$IS_EXPERIENCING_MERGING - echo' = = = = = = = = = = job_5_afterMR complete = = = = = = = = = = = 'stage: fifth_stage retry: 2 job_5_beforeMR: rules: - if: '$CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"' tags: - yehanli script: - echo 'Is experiencing before-merge? '$IS_EXPERIENCING_MERGING - echo' = = = = = = = = = = job_5_beforeMR complete = = = = = = = = = = = 'stage: fifth_stage retry: 2Copy the code
# pull official image, Update to builder FROM node:latest as Builder # set working directory to /app WORKDIR /app # Follow package.json to install dependencies on COPY package.json Package-lock. json./ RUN NPM install --registry=https://bnpm.byted.org # # nginx is a lightweight Web server FROM nginx:latest COPY -- FROM = Builder /app/build /usr/share/nginx/htmlCopy the code

❤️ Thank you

That is all the content of this sharing. I hope it will help you

Don’t forget to share, like and bookmark your favorite things.

Welcome to the public account ELab team harvest dachang good article ~

We are from the front end department of Bytedance, responsible for the front end development of all bytedance education products.

We focus on product quality improvement, development efficiency, creativity and cutting-edge technology and other aspects of precipitation and dissemination of professional knowledge and cases, to contribute experience value to the industry. Including but not limited to performance monitoring, component library, multi-terminal technology, Serverless, visual construction, audio and video, artificial intelligence, product design and marketing, etc.

Bytedance correction/social recruitment internal push code: HHQDYFS

Post links: job.toutiao.com/s/LWYmeaJ