Odd technical guidelines

This paper introduces how the front end team of Chinese prickly ash realizes automatic construction and release based on GitLab CI/CD.

This article is from the contribution of Prickly ash technology

After the company set up an internal GitLab platform, the front end activity projects were migrated from SVN to GitLab. This article describes how to implement an automated build and release based on GitLab CI/CD.

In the process of migrating from SVN to GitLab and accessing GitLab CI/CD, I would like to express my special thanks to the support of the publishing system and server side students.

01

Current build and release processes

In this section, we first describe the benefits of using GitLab CI/CD and then describe the build and release processes before and after using GitLab CI/CD.

1

The team returns

  1. The average release time was reduced from 5 minutes to 1.5 minutes. From 5 minutes of manual work all the way to 1.5 minutes of branch code merging, automated build and release;

  2. The front-end build is placed in CI/CD, which solves the problem that local builds can lead to inconsistent packaging of online code.

2

Build, release before using GitLab CI/CD

2.1 process

  1. After the development is completed, build locally and submit the packed HTML file after build.

  2. Open the publishing system and publish the HTML file.

2.2 pain points

  1. The operation process of releasing code is long and time-consuming. After the completion of development, the release of code, need to go through a lot of manual operation, coupled with the release system is slow, each release takes about 5 minutes;

  2. Everyone’s local environment is inconsistent, and there is a risk of inconsistent distribution by people with different code online.

2.3 Time consuming points requiring manual operation

A total of at least eight manual operations are required

Local build:

1. Run the build command and wait until the build is complete.

2. After the build is complete, submit the packed HTML file.

Release code:

1. Open the publishing system

2. Select the release project and environment

3. Open the release page

4. Select the release file

5. Enter the release information

6. Click Ok to publish

3

Build and release using GitLab CI/CD

Publishing code in one step: just merge the development branch to the corresponding branch of the publishing environment, and after submitting the branch, GitLab CI/CD will automatically build and publish.

02

What is GitLab CI/CD

In this section, we will briefly introduce GitLab CI/CD and then introduce how to build a GitLab CI/CD from scratch.

1

A brief introduction to GitLab CI/CD

As shown in the figure below, after the code is submitted to GitLab, the pipeline will be triggered for automatic construction and release after the specified conditions are met.

A pipeline can be understood as a build task and can contain multiple processes, such as downloading dependencies, running tests, compiling, and deploying. When a pipeline fires, is divided into several processes, and what each process does is defined in the project’s.gitlab-ci.yml file.

For our active project, pipeline mainly includes three processes: dependency download, build code, and release to the server. The follow-up practice will be introduced in detail.

2

GitLab CI/CD overall process

The specific pipeline process and operation of GitLab CI/CD are stated in.gitlab-ci.yml file. After pipeline is triggered, GitLab Runner will run according to.gitlab-ci.yml file. It will return to the GitLab system after running.

2.1. Gitlab – ci. Yml file

The.gitlab-ci.yml file is a declarative file that defines the stages of the Gitlab CI/CD process and what each stage does.

As for what to do and how to do, we mainly use the command line and script operations, which will be described in detail in the practice section later.

If there’s some logic involved, we’ll use scripts, and our project will use Shell scripts, Python scripts.

2.2 GitLab Runner

GitLab Runner is the CI execution environment, responsible for executing the Gitlab-ci.yML file and returning the results to the GitLab system. Runner can have various forms, such as Docker, virtual machine or shell. The way is selected when registering Runner.

3

Build a GitLab CI/CD from scratch

To understand the process, set up a project on GitLab and run through the CI/CD process.

3.1 Create a GitLab project

  1. Go to GitLab: https://gitlab.com/

  2. Create your own project

3.2 configuration Runner

GitLab provides some shared runners, so we don’t have to deal with runners.

3.3 Create the.gitlab-ci.yml file

  1. Pull items to local

  2. Create a.gitlab-ci.yml file in the project root directory

  3. Submit the.gitlab-ci.yml file

  4. In the CI/CD of the project, you can see the CI/CD in action

.gitlab-ci.yml file example

image: node# define stagesstages:  - build  - test# define jobThe build phase:  stage: build  script:    - echo "build stage"# define jobPublish to test environment:  stage: test  script:    - echo "test stage"Copy the code

03

New projects are connected to GitLab CI/CD process

The team needs to access GitLab CI/CD for a new project, first apply for the GitLab project, then ask the GitLab system maintainer to help configure Runner, write.gitlab-ci.yml file, and trigger the construction to see the SITUATION of CI/CD.

1. New GitLab project

2. Configure GitLab Runner

3.. gitlab – ci. Yml file

At present, two new project routes have been connected to GitLab CI/CD, and the access condition is good. The operation process is relatively smooth according to the documents.

04

GitLab CI/CD practice

In the practice part, GitLab Runner and.gitlab-ci.yml files are mainly introduced here. The main processes and problems encountered and solutions are included in the introduction process of.gitlab-ci.yml files.

1

GitLab Runner

GitLab Runner is generally managed by GitLab system maintainer. After configuration, similar projects can be shared without modification. There is no specific introduction here, but mainly introduces the attention points in the process of use. For specific use, please refer to the GitLab Runner document.

(https://docs.gitlab.com.cn/runner/)

1.1 GitLab Runner use process

1. Download GitLab Runner

2. Register with GitLab Runner

3. Use GitLab Runner

1.2 GitLab Runner pay attention

In the process of using Runner, we encountered some problems. The problems and solutions are briefly introduced below without specific introduction.

1.2.1 After configuring Runner, push code starts pipeline, but it is always in Pending state

The error message is: This job is stuck, because you don’t have any active runners that can run This job

Runners can run untagged jobs from Settings→CI/CD→Runners Settings by default.

1.2.2 Types of GitLab Runner

There are three types of Runners. Shared Runners can be used in all projects in the whole system. After Group Runners are registered, different code bases under the same project are Shared. Using Specific Runners Consider whether Shared Runners and Group Runners need to be turned off.

    1. Shared Runners

    2. Specific Runners

    3. Group Runners

1.2.3 Use Docker in GitLab CI

When deploying to Ali Cloud, you need to use Docker in GitLab CI/CD for mirroring and publishing. See Building Docker Images with GitLab CI/CD

(https://docs.gitlab.com/ee/ci/docker/using_docker_build.html)

1.2.4 Access Runner host directory in GitLab CI/CD

The Runner executor we used was Dokcer, and in Dokcer volumes we configured the directories we needed to access.

2

. Gitlab – ci. Yml file

The activity project.gitlab-ci.yml file is as follows. The following is mainly through the.gitlab-ci.yml file of the activity project to introduce our practice process, detailed usage of.gitlab-ci.yml. Refer to the GitLab CI/CD Pipeline Configuration Reference document

(https://docs.gitlab.com/ee/ci/yaml/README.html)

2.1. Introduction to gitlab-ci. yML file

Image is the Docker base image that performs CI/CD dependencies. The image contains Node, Yarn, and Dalp (internal rsync tool).

Stages defines our pipeline processes as follows:

1. Download the dependency phase pre_build

2. Build phase

3. Deploy during the release phase

Stage States the current stage. It is used in stages

Variables are used to define variables

Before_script Operations performed before script

Script The operation that the current stage needs to perform

Changes specifies the stage trigger condition

Refs specifies the branches triggered by the stage

Image: registry.huajiao.com/gitlab-ci/node-yarn:v1.4variables:# $CI_PROJECT_PATH: the project ID that uniquely distinguishes this project from other projects# $CI_PROJECT_DIR: local project path# $PROCESS_PATH: temporary file directory (including logs and some temporary files)  NODE_MODULES_PATH: /runner-cache/frontend/$CI_PROJECT_PATH/$CI_BUILD_REF_NAME/node_modulesstages:  - pre_build  - build  - deployDownload dependencies:  before_script:If no node_modules file is available, create a node_modules file    - /bin/bash ./ci/mkdir.sh $NODE_MODULES_PATHSoftlink node_modules to the host    - ln -s $NODE_MODULES_PATH .    - cd webpack@next  stage: pre_build  script:    - echo "yarn install"    - yarn install  --network-timeout 60000  only:    changes:      - webpack@next/package.json    refs:      - test      - test-99      - test-128      - master      - ci      - feature/ci-testConstruction:  stage: build  variables:    CI_COMMIT_BEFORE_SHA_PATH: /mnt/gv0/gitlab-runner-cache/$CI_PROJECT_PATH    CI_COMMIT_BEFORE_SHA_FILE_NAME: $CI_BUILD_REF_NAME.sh    CI_COMMIT_BEFORE_SHA_FILE: /mnt/gv0/gitlab-runner-cache/$CI_PROJECT_PATH/$CI_BUILD_REF_NAME.sh  before_script:# save the files of this CI CI_COMMIT_SHA    - /bin/bash ./ci/mkfile.sh $CI_COMMIT_BEFORE_SHA_PATH $CI_COMMIT_BEFORE_SHA_FILE_NAMESoftlink node_modules to the host    - ln -s $NODE_MODULES_PATH .    - rm -rf php/share/*    - cd webpack@next  script:# cache the last CI    - source $CI_COMMIT_BEFORE_SHA_FILE    - echo "CI_COMMIT_BEFORE_SHA=$CI_COMMIT_SHA" > $CI_COMMIT_BEFORE_SHA_FILE- python3 .. / ci/build. Py # compiler- /bin/bash .. /ci/commit  only:    changes:      - www_src/**/*    refs:      - test      - test-99      - test-128      - master      - ciTest release:  stage: deploy  variables:PROCESS_PATH: / MNT /gv0/gitlab-runner-cache/deploy/process/$CI_JOB_ID #  script:- mkdir $PROCESS_PATH # Create a temporary path for publishing configuration files and result logs    - dplt $CI_PROJECT_DIR/.deploy_test.yml $CI_PROJECT_PATH $CI_PROJECT_DIR/php/ $PROCESS_PATH# DPLT publishes YML configuration- the echo "release is completed, the error log view" http://new.admin.wolffy.qihoo.net/log?path= $PROCESS_PATH    - echo `ls $PROCESS_PATH/*.log`  only:    changes:      - php/**/*    refs:      - testCopy the code

2.2 Pre_Build Stage

When the package.json file changes, the pre_build stage is triggered and yarn install is executed. The downloaded node_modules is stored on the host machine, and dependencies are retrieved via the soft chain.

2.3 Build Stage

The construction stage is divided into three parts

1. The DIff file changes

2. The front end build

3. Result after commit build

2.3.1 Diff file changes

At each CI, the current CI commit SHA (CI_COMMIT_SHA variable) is stored in the file as CI_COMMIT_BEFORE_SHA variable. At diff, git diff changes the CURRENT CI from the last COMMIT SHA.

2.3.2 front-end build

According to the change of Git diff, determine the project to be packaged this time.

2.3.3 commit HTML file generated after packaging

When submitting code in GitLab CI/CD, commit packaged HTML files using Git credential stores. Git credential store details can be found in the credential store documentation

(https://git-scm.com/book/zh/v2/Git-%E5%B7%A5%E5%85%B7-%E5%87%AD%E8%AF%81%E5%AD%98%E5%82%A8)

2.4 Deploy Stage

In the release phase, the packaged HTML files are deployed using the internal Rsync tool DPLT. DPLT allows you to configure the cluster and machine list.

05

Current problems and future directions

1. For a continuous integration, although automated build and release is implemented, critical testing is missing.

2. With the help of GitLab CI/CD, we have achieved the consistency of the online environment, but the local development environment is still inconsistent with the online environment. There may be no problems in the local environment, but problems in the online environment.

3. After switching from SVN to Git development, the branch management strategy, branch naming specification, Commit message specification, CodeReview, etc., need to be gradually standardized in the team.

06

Main References

What is continuous integration?

(http://www.ruanyifeng.com/blog/2015/09/continuous-integration.html)

What is CI/CD?

(https://www.redhat.com/zh/topics/devops/what-is-ci-cd)

GitLab Docs(https://docs.gitlab.com/)

Introduction to CI/CD with GitLab

(https://docs.gitlab.com/ee/ci/introduction/)

Continuous integration with GitLab CI

(https://scarletsky.github.io/2016/07/29/use-gitlab-ci-for-continuous-integration/)

How to realize continuous integration and continuous deployment of front-end engineering?

(https://www.zhihu.com/question/60194439)

Practice of FRONT-END engineering CI/CD based on GitLab CI

(https://github.com/giscafer/front-end-manual/issues/27)

World of you when not

Just be your shoulders

There is no


360 official technical official account

Technology of dry goods | | hand information activities

Empty,