This is the third day of my participation in Gwen Challenge

One, foreword

.gitlab-ci. yML has a lot of parameters, if the first contact must be very dizzy, here is a common configuration to interpret again, add a specific display of some concepts, so as to better understand.

.gitlab-ci.yML has the following capabilities

  • 1, run the shell script, through the script file
  • 2. Reference other template files and configuration files via the include directive
  • Define caches and dependencies via cache directives
  • 4, define the tasks to be performed and the order of execution, pipeline, jobs
  • 5. The location of application deployment can be customized
  • 6. How to start CI, API and UI

2. Basic concepts

1, Pipelines,

When.gitlab-ci.yml is configured, ci can be triggered.

In Gitlab -> CICD you can see the Pipeline started as shown in the figure,

Passed for success, failed is displayed for failure,

Each pipeline is composed of a Job, which is executed in Gitlab-runner

2, the Jobs

Click the pipeline in the figure above, you can see the figure below, which includes two Jobs. The execution sequence of these Jobs can be configured in. Gitlab-ci. yml, or parallel execution can be configured.

3, the Variables

(1).gitlab-ci.ymlThe variables defined in Jobs have scopes.

(2) Official predefined variables, for reference

(3) Defined in Gitlab->CICD->Variables

The above variables can be used directly in. Gitlab-ci.yml

4, Runners

Runner to use the application of Pipeline execution, need to be installed and registered in advance, specific details can refer to my other article Gitlab CI/CD execution process and Gitlab Runner installation and registration

Sample files

Here is an example configuration file, followed by a detailed explanation of what these keywords mean

include:
  - remote: "http://www.baidu.com/data/prod.yml"

variables:
  - buildImage: "http://hub.com/nginx"

workflow:
  rules:
    - if: '$CI-PIPELINE_SOURCE' == "push"'
      when: never
    - when: always

stages:
    - build
    - nextStep
    - testStep1

build-job:
  stage: build
  script:
    - echo "Hello, $GITLAB_USER_LOGIN!"

test-job1:
  stage: nextStep
  script:
    - echo "This job tests something"

test-job2:
  stage: nextStep
  script:
    - echo "This job tests something, but takes more time than test-job1."
    - echo "After the echo commands complete, it runs the sleep command for 20 seconds"

deploy-prod:
  stage: testStep1
  script:
    - echo "This job deploys something from the $CI_COMMIT_BRANCH branch."
Copy the code

This file contains four Jobs that can be executed. The stages of test-job1 and test-joB2 are named nextStep, and the two Jobs are executed in parallel.

Global keywords

.gitlab-ci.yml can be divided into several levels. First, the outermost variables include:

The keyword instructions value note
stages Phase, type array Respectively are user-defined jobs Specify the execution sequence of each task. If the task name is the same, the task is executed at the same time
include The YML or YAML configuration file for the application Key: can be local, remote, file, or template Local is a local file, remote is another accessible remote address, filter is the file path of another project, and template is an official template
variables variable Predefined or custom The priority is different according to the position of the variable. The same variable will be overwritten according to the priority
workflow workflow rules Rules is used to define when CI/CD is triggered, similar to rules and only in Jobs

1, the variable

There are a number of different scenarios for variables, including

  • predefined
  • Custom declarations (global variables or declarations in job, scripts)
  • In Gitlab->setting->CICD add, generally used to save account secret

Variable use code example, show the use of variables, print information

before_script:
  - echo "this is .gitlab-ci.yml $CI_PIPELINE_SOURCE $CI_COMMIT_TAG, $CI_COMMIT_BRANCH"
  - echo "VARIABLE CI_COMMIT_SHA IS $CI_COMMIT_SHA!"
  - echo "VARIABLE CI_COMMIT_TAG IS $CI_COMMIT_TAG!"
  - echo "VARIABLE CI_PROJECT_DIR IS $CI_PROJECT_DIR!"
Copy the code

Considerations for variable use:

  • Variables have priority and are overwritten if they are the same
  • Variables are defined in many places, including global, JOBS, scripts, Gitlab project or group variable pages (pages are often used to configure secure keys), and can be used directly in configuration files
  • Each iteration of Gitlab will add different variables. When using some variables that are not commonly used, pay attention to whether the current version of Gitlab supports them
  • Variables are used in scripts:$CI_BUILDS_DIR
  • Predefined variable

2. Jobs and Stages

  • Like the example configuration above, it is configuredbuild-job,test-job1,test-job2,deploy-prodFour tasks to perform.
  • Stages defines the order of execution, where stage isnextStepTasks are executed in parallel
  • At least one task must be configured.
  • Each Job is part of the pipeline.
  • These jobs will be performed in Giblab Runner
  • The Job execution process is displayed on the console, and the status of each stage is updated in real time.

Specific execution demonstration

Stages is used to determine the order in which jobs are performed

3, the include

The common attributes of the include command include local, remote, file, and template. It is used to introduce additional YML configuration files and perform in-depth merging.

  • Include can appear more than once in a file
  • No matter where the include is in the file, it’s going to be introduced first and merged, right
  • Include files that are introduced are overwritten if the variables are the same and are not executed more than once.
  • Include includes loacl and remote can be omitted.
  • Include can use defined variables.
  • Local indicates the file to import the current project.
  • The repository for the remote YML file must be public or an error will be reported, which means that the YML file can be stored in a separate repository and retrieved by reference
  • Project represents a YML file that references a project accessible to the current user
  • Template means you can use the YML file defined by Gitlab, the official public template

Include: include: include: include: include: include: include: include: include: include: include: include: include: include: include: include: include: include:

include: "http://www.baidu.com/file/prod.yml"
include: ".test1.yml"
include:
  - local: ".test1.yml"
  - remote: "http://www.baidu.com/file/prod.yml"
include: 
  - "http://www.baidu.com/file/prod1.yml"
  - ".test.yml"
  - template: "Auto-DevOps.gitlab-ci.yml"
include:
  - project: "console/gitlab-ci-templates"
    ref: "master"
    file: ".gitlab-ci-console-cli.yml"
Copy the code

4, workflow

Workflow determines when a pipeline is triggered or disabled

The main parameters are:

  • If, combined with predefined variables, determines that the condition is true, triggers pipeline or disables
  • If the condition is met, variables can be defined directly
workflow:
  rules: 
    - if: '$CI_COMMIT_SOURCE = "schedule"'
      when: never
    - if: '$CI_COMMIT_SOURCE = "push"'
      when: never
    - when: always
    - if: $CI_COMMIT_BRANCH = "feature"
      variables:
        IS_A_FEATURE: "true" 
Copy the code

As indicated above, if the condition is met, look at the value of when, never means that pipeline is forbidden in this case, and if no write is met, pipeline is triggered by default. Finally, when: always will be triggered except for the above conditions and the value of when is never.

Finally, $CI_COMMIT_BRANCH = feature defines the variable IS_A_FEATURE when the branch name is feature

The usage is similar to: rules and only in Jobs

Five, the key words of Jobs

  • Per project.gitlab-ci.ymlHave at least one Job
  • Each project implements different functions by defining configuration fields, such as unit testing, static checking, building mirrors, and so on
  • In one mission, exceptscriptsEverything else is optional
  • Many of these variables can be declared directly globally, overwriting tasks with identical names
The keyword instructions note
image Gitlab-runner performs the required Docker image
services The execution phase synchronizes the dependent Docker service
script The statement executed in docker Before_script, after_script, script Declares an array of scripts, executed one by one
stage It is used to declare which task should be executed in the stages array. If two tasks are the same, they should be executed synchronously If not defined, stages is executed in. Pre build test deploy. Post order
extends Define common configurations that are referenced repeatedly through the extends field Array introduction, the same level of configuration, the last overrides the previous, can be used with include
rules Similar to when, the execution time is defined at the task level, and CI is triggered when the judgment condition is true There are many fields that can be judged, and you can combine them according to your needs
only, except Similar to rules, only and except are used to define the timing of CI You are advised to use rules
tags Defines the runner that executes the Job. The value is the tag that defines the runner. If the Job is not written, the Job cannot be executed
allow_failure Allows the current task to fail without interrupting the execution of other tasks You can also define which error codes are encountered and proceed
when Define when to execute the job. The default value on_SUCCESS indicates that all previous jobs are executed successfully Other common attributes are: on_failure, always, manual, of, never
cache Define cache files or cache folders
environment Define the environment variables of the Job
artifacts After the task is executed, cache the corresponding resources and download them to the local PC
coverage Set code test coverage
retry Example Set the number of Job retries after Job execution fails
timeout Timeout period of Job execution
parallel The number of concurrent tasks ranges from 2 to 50
trigger When performing tasks, trigger CI processes for other projects
variables Variables of the Jobs dimension

Note: Variables that cannot be used as job are

image services stages types before_script after_script variables cache include

Because these variables have a specific meaning, they are no longer used as the name of the Job.

Six, Jobs keyword interpretation

Docker, Services

Docker The image to be used by the current Job. If the current Job is not declared, a globally defined docker will be used.

Docker properties

  • Name Indicates the name. If there is only one name, you can omit it
  • Command executed after entrypoint docker is pulled

Services attributes

  • Alias Service alias
  • Name Indicates the name. If there is only one name, you can omit it
  • Entrypoint Command to execute after the container is started by pulling the image
  • Command Indicates the command to be executed
Image: name: Ruby :2.6 EntryPoint: ["/bin/bash"] services: -name: my-postgres:11.7 Alias: db-postgres entryPoint: ["/usr/local/bin/db-postgres"] command: ["start"]Copy the code

2, the script

Script is a mandatory field of Job. It is used to write shell scripts. Multiple commands can be executed

Before_script and after_script scripts executed before and after the Job starts and ends can be written in global variables, as shown in the following figure:

job:
  before_script:
    - echo "Execute this script instead of the global before_script."
  script:
    - echo "This script executes after the job's `before_script`"
Copy the code

3, the environment

testStep1:
  tags:
    - uaek-c1
  stage: nextStep
  image: nginx
  script:
    - echo "this is third testStep"
  environment:
    name: dev
    url: http://www.baidu.com
Copy the code

Add the Environment field and you will see the following changes on the Gitlab page

1, CI/CD, enter a task page, it will be displayedenvironment.

2,OperationsSo it says Job pressenvironmentTo categorize, click dev to open the following page

3. You can see the View Deployment button, and clicking on it will take you to the URL set under the Environment field. In the example above, it will take you to http://www.baidu.com, which will facilitate regression after successful release

4, extends

Extends can reference a defined common configuration and introduce multiple at the same time, which will be merged

If the variables at the same level are the same, the ones at the latter level will be overwritten by the ones at the front.

.only-important:
  variables:
    URL: "http://my-url.internal"
    IMPORTANT_VAR: "the details"
  only:
    - main
    - stable
  tags:
    - production
  script:
    - echo "Hello world!"

.in-docker:
  variables:
    URL: "http://docker-url.internal"
  tags:
    - docker
  image: alpine

rspec:
  variables:
    GITLAB: "is-awesome"
  extends:
    - .only-important
    - .in-docker
  script:
    - rake rspec
Copy the code

The configuration files above are merged

rspec:
  variables:
    URL: "http://docker-url.internal"
    IMPORTANT_VAR: "the details"
    GITLAB: "is-awesome"
  only:
    - main
    - stable
  tags:
    - docker
  image: alpine
  script:
    - rake rspec
Copy the code

5, rules

The attributes of rules are

  • The if condition is true
  • When the defaulton_successAnd other valuesdelayed.start_in
  • Allow_failure Job Failed to execute, whether to continue pipeline
docker build:
  script: docker build -t my-image:$CI_COMMIT_REF_SLUG .
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      when: delayed
      start_in: '3 hours'
      allow_failure: true
Copy the code
The trigger condition instructions
branches When the branch is pushed
API Trigger the pipeline through the API
external When using CI third-party services
external_pull_request_event Pull requests from external repositories
merge_request_event When a Git request is merged, the pipeline fires
pipeline Use the trigger field in the project to trigger the project for CI execution
push Branch push or tag push
schedule Configure scheduled pipeline execution on the Gitlab -> Schedule page
trigger Trigger by Token
web On the Pipeline page on Gitlab, click Run Pipline
| webide Trigger CICD using WebIDE

Rules, like when, can be flexibly configured to trigger rules and can be used to replace only

job:
  script: echo "Hello, Rules!"
  rules:
    - if: '$CI_PIPELINE_SOURCE == "schedule"'
      when: manual
      allow_failure: true
    - if: '$CI_PIPELINE_SOURCE == "push"'
Copy the code

6, only, never dull

CI is triggered only when the condition is met. Except, it is not triggered when the condition is met

  • Variables uses these two parameters to define variables that can only be applied to the current Job
  • Refs is followed by the corresponding branch name, which can be matched using the re
  • Changes listens for file changes to trigger a CI
deploy:
  script: cap staging deploy
  only:
    variables:
      - $RELEASE == "staging"
    refs:
      - main
      - /^issue-.*$/
      - merge_requests
    changes:
      - Dockerfile
Copy the code
job2:
  script: echo 'test'
  only:
  - branches
  - tags
Copy the code

7, tags,

job:
  tags:
    - uaek-c1
Copy the code

The value of tags is to create the tag of Gitlab Runner, and this value is configured to indicate that the current Job will be executed in the runner of the tag.

Cache, artifacts

Cache Cache files or folders to speed up construction

Artifacts configuration files or folders, which can be downloaded on the Gitlab UI after the task is successfully executed, as shown in the figure

  tags:
    - uaek-c1
  stage: nextStep
  parallel: 2
  image: nginx
  artifacts:
    paths:
      - package.json
  script:
    - echo "this is first nextStep"
Copy the code