Gitlab CI — Front-end automation construction and optimization

Now there are more and more front-end tricks, today I’m going to talk about nothing new, automated build deployment, if you are a Javaer or operations students, for Jenkins and other automated deployment is certainly familiar. But as mentioned, as a front end, especially for business-driven development, you don’t get much exposure to these things. I also recently plan to front-end automation deployment on the container, and then learn to look at the next learning process and summary to share with you, very shallow, Daniel direct upper right corner (upper left corner) can be closed.

The purpose of my article is to summarize the detours I have gone through to everyone, so as to avoid you stepping on pits, not asking for your praise and spraying less. Because I have experienced a lot of “millions of articles, practice is doomed” scenarios too many, we are not responsible for the reader, so my articles will generally have code and Demo examples, not the best but at least if you have encountered the same problem, certainly can solve ~

Pre –

• Gitlab
• Gitlab Runner
• Docker

The above belongs to the pre-content, not to mention Gitlab. Generally, the company uses Gitlab warehouse internally. Gitlab Runner, which typically requires an extra installation and runs pre-written scripts to automate build deployments (described below); Finally, the project is packaged as an image to be deployed on the container, so Docker is also required.

Because it is the internal process of the company rather than playing alone, it is ok to let the operation and maintenance to deal with it for you. In this era, it should be the default in the company. Personal projects should most likely be posted on Github. You can choose Github Actions. There are many tutorials online.

Create a new project

For an example, create-React-app automates the build deployment and generates Gitlab Pages. First, we create a new project to push to the remote repository:

.gitlab-ci.yml — Initialization

When using Gitlab CI for automatic build deployment, we need to create a.gitlab-ci.yml configuration file, which contains the script of the specific steps of our automatic build. Here is a simple initialization, which will be described in detail below:

# Define stages
stages:
  - build
  - deploy

# Define job-build projects
job1 - build Phase:
  You need to install dependencies before you start
  stage: build
  script:
    - echo "finish build stage"

# define job
job2 - Publishing to an online environment:
  stage: deploy
  script:
    - echo "finish deploy stage"
Copy the code

Above configuration file generally means that the project, an automated build roughly divided into two stages, respectively is the build and deploy phase, and each stage will perform some corresponding task, because here is initialized demonstration, is simply the two lines of text output, meaning you don’t need to know, and the field will be described in detail below.

Push the file up and see the effect:

< span style = “box-sizing: border-box! Important; word-break: inherit! Important

As you can see from the figure above, the automated build of the project is complete, so easy, you might ask? It’s as simple as that, and even though I’ve simplified the build to two lines of output, the overall flow remains the same, just more complexity and scripting. Next, we will use this example to deepen our understanding of Gitlab CI step by step.

Analysis of YAML concepts in Gitlab

As mentioned earlier, all the build and deployment content is written in the.gitlab-ci.yml script, so the content of this file and the corresponding meaning of each keyword is the first thing to know.

The official documents are here. I am just a porter for the screening of official documents. You can also translate youdao Dictionary by yourself.

The keyword	Reqired	describe
image	no	Use Docker images
service	no	Use the Docker service
stages	no	Define the Construction phase
types	no	An alias for stages (deprecated, it is recommended to use only stages)
before_script	no	eachScript Script executed before a task is executed
after_script	no	eachScript Script that is executed after a task is executed
variables	no	Define variables that can be used during the construction phase
cache	no	Define cache files that can be used in later stages

job

Job is a top element in.gitlab-ci.yml, which is a must for any automated deployment. In my opinion, it can be divided into tasks, and each task contains at least one script indicating what to do in the task. Multiple jobs can be defined and each job is independent of each other. The job will be executed by Gitlab Runner in its own environment.

# define job1 is
job1:
  script: 
    - echo "i'm job1"
# define job2
job2:
  script: 
    - echo "i'm job2"		
Copy the code

The simplest example above defines two jobs, each containing a script that outputs a string.

${string}: string + semicolon (); ${string}: string + semicolon ();

image && services

Since Gitlab Runner is also built using Docker, we can also use the corresponding image for base building when building our code. The corresponding fields are image and services. Here is a simple example:

# Base image
image: Ruby: 2.1
# use the mirror service - postgres
services:
  - postgres
Copy the code

Next, let’s look at our own project. We are a create-react-app built project, so we must rely on node images. The service is optional because it is not required.

# Rely on mirror node
image: Node: 10.16.0
# Define stages
stages:
  - build
  - deploy

# Define job-build projects
job1 - build Phase:
  You need to install dependencies before you start
  stage: build
  script:
    - echo "finish build stage"

# define job
job2 - Publishing to an online environment:
  stage: deploy
  script:
    - echo "finish deploy stage"
Copy the code

Above, we added mirror dependencies and made the version number v10.16.0.

stages

For example, our project is divided into two phases. The first phase is build — compile and package, and the second phase is deploy — publish and live. In fact, it corresponds to two jobs, and then more specific description of each stage is given below.

The stages of the definition of the “stages” field are in the same order in the building of pipelines, and the rule is as follows:

• The former phase is completed and the latter phase is executed

That is, deploy will not be executed until build is successful.

• All phases are successful; the whole building process is marked as pass
• One of the phases fails, subsequent pipelines will not be executed, the entire build is marked as failure.

We can check the specific execution process in Charge, as shown in the figure below

script

This is one of the most important fields. It represents a shell script executed by Gitlab Runner, which must execute many commands during the build process, such as install dependencies, package, deploy, etc. Again, in our case, the addition of node images means that commands such as NPM install and NPM run build can be executed.

# dependency mirror
image: Node: 10.16.0

# Define stages
stages:
  - build
  - deploy

# Define job-build projects
job1 - build Phase:
  You need to install dependencies before you start
  stage: build
  script:
    - yarn install
    - yarn build
    - echo "finish build stage"

# define job
job2 - Publishing to an online environment:
  stage: deploy
  script:
    - echo "finish deploy stage"
Copy the code

Yarn install and YARN Build are two new commands in job1. If you are familiar with front-end development, you need to install dependencies and package and compile before project production. Next push to the remote repository to see the effect:

• Building a successful

• Build content for the first step

As can be seen from the screenshot above, the pushed code triggers the automatic build again, which is successful and the first step of the build stage executed the three scripts we wrote in turn, which is very perfect ~

Before_script and after_script

Before_script – a script executed before the script is executed, after_script – a script executed after the script is executed.

If yarn install depends on the script, put it in before_script, so build phase only build command, more appropriate. The answer is no, because the two commands are in the outer layer, which means that they are executed before each stage. That is, each job executes before_script before executing its own script. Yarn install only needs to execute this command once.

I thought about looking for the best scenario to demonstrate, but felt that the scenarios used are really few and not very appropriate, of course, if just to understand the function, then output a string is ok, here I think we can set taobao mirror before before_script.

# dependency mirror
image: Node: 10.16.0


before_script:
  - echo "======== before script ========"
  - npm config set registry https://registry.npm.taobao.org

after_script:
  - echo "======== after script ========"

# Define stages
stages:
  - build
  - deploy

# Define job-build projects
job1 - build Phase:
  You need to install dependencies before you start
  stage: build
  script:
    - yarn install
    - yarn build
    - echo "finish build stage"

# define job
job2 - Publishing to an online environment:
  stage: deploy
  script:
    - echo "finish deploy stage"
Copy the code

Take a look at the results:

• The build phase

• The Deploy phase

You can see that, exactly as described, both scripts are executed at each build stage-stage.

only && except

The next two keywords are also important, so let’s take a look at some examples:

• Step 1: Create a new branch:branch-a
• Second, modify the content to push to remote

Regardless of whether the build was successful or not, now there is a problem. Why? You can’t trigger a build every time you push a branch to a remote. You can’t. Only and except come in handy.

• Only: Only the trigger builds that meet the conditions are allowed
• Except: Builds are triggered except for certain things

For demonstration purposes, I’ve agreed that only the master push will trigger, and only the master branch code will trigger the build.

# dependency mirror
image: Node: 10.16.0

before_script:
  - echo "======== before script ========"
  - npm config set registry https://registry.npm.taobao.org

after_script:
  - echo "======== after script ========"

# Define stages
stages:
  - build
  - deploy

# Define job-build projects
job1 - build Phase:
  You need to install dependencies before you start
  stage: build
  script:
    - yarn install
    - yarn build
    - echo "finish build stage"
  only:
    - master

# define job
job2 - Publishing to an online environment:
  stage: deploy
  script:
    - echo "finish deploy stage"
  only:
    - master
Copy the code

[Note] : The only field needs to be defined separately for each job stage, because there is no guarantee that you will be required for each job stage.

Push the code up and it will no longer trigger the automatic build, so let’s verify again by adding an MR to the branch-a branch and merging it into the master branch.

• Merge Request

• Merge && CI

As you can see, the branch commit does not trigger the CI, but the Merge to the master does, as expected.

There’s a lot more to only and except, like specific branches, specific tags, etc. I don’t want to say too much here, in short, it can satisfy you any complex scenes and operations, you go to the combination of good.

variables

We can define some common variables in advance and use them in job scripts. A simple example:

variables:
    DOCKER_HUB_URL: "https://custom.dockerhub.com"
    
# Define job-build projects
job1-build:
  You need to install dependencies before you start
  stage: build
  script:
    - yarn install
    - yarn build
    - echo $DOCKER_HUB_URL
    - echo "finish build stage"

Copy the code

In addition to custom variables, the system also has many built-in constants: see here for details

artifacts

This parameter is also important. After the current job is successfully built, the file or folder in the build list can be transferred to another job (not necessarily the next job). That is, the file content is transferred between two jobs.

To see why this is important, let’s take a look at an example. We added a new command in Job2 to view the current build directory:

# define job
job2 - Publishing to an online environment:
  stage: deploy
  script:
    - ls
    - echo "finish deploy stage"
  only:
    - master
Copy the code

The following output is displayed:

If you know React, create-react-app will generate a /build folder when it is packaged. The contents in this folder are generally the final online content. Artifacts are also used because the joB1 build was successful, so you need to pass the joB1 build folder /build to JoB2.

# dependency mirror
image: Node: 10.16.0

before_script:
  - echo "======== before script ========"
  - npm config set registry https://registry.npm.taobao.org

after_script:
  - echo "======== after script ========"

# Define stages
stages:
  - build
  - deploy

# Define job-build projects
job1-build:
  You need to install dependencies before you start
  stage: build
  script:
    - yarn install
    - yarn build
    - echo "finish build stage"
  only:
    - master
  artifacts:
    paths:
        - build/

# define job
job2-deploy:
  stage: deploy
  script:
    - ls
    - echo "finish deploy stage"
  only:
    - master
  dependencies:
    - job1-build
    
Copy the code

• job1-build

Add artifacts to Job1 and set paths: build/ folder.

• job2-deploy

Job2 added Dependencies: Job1-build, which indicates that the job depends on artifacts passed by job1-build.

Push to remote view effect:

As you can see from the above figure, job2 can now retrieve the /build folder that was generated after the successful build of Job1.

.gitlab-ci.yml

The final configuration file looks like this:

# dependency mirror
image: Node: 10.16.0

before_script:
  - echo "======== before script ========"
  - npm config set registry https://registry.npm.taobao.org

after_script:
  - echo "======== after script ========"

# Define stages
stages:
  - build
  - deploy

# Define job-build projects
job1-build:
  You need to install dependencies before you start
  stage: build
  script:
    - yarn install
    - yarn build
    - echo "finish build stage"
  only:
    - master
  artifacts:
    paths:
        - build/

# define job
job2-deploy:
  stage: deploy
  script:
    - ls
    # # # # # # #
    # This is where you can actually release the project
    # # # # # # #
    - echo "finish deploy stage"
  only:
    - master
  dependencies:
    - job1-build
    
Copy the code

In fact, the real automated construction deployment line is to add more complex scripts in job2-deploy. For example, what the author did is to package the Docker image and push it to the Docker Hub and then deploy the project through K8S. Of course, stage and job can also be added. The specific details are different for each person and each company, so you can modify them by yourself.

Principles of project construction process system

After the introduction of the front end of the basic Gitlab UI can be simple automated deployment online, but in the company when many people develop to standardize some, the following is my personal (personal opinion, do not like spray) feel more standardized reasonable process.

• Develop projects in parallel, and launch master branch each time
• New branch development based on master branch (single player && multiplayer)
• Before the branch development is completed and ready to go online, Merge Request, code-review is ok, and Masters will Merge to master
• Gitlab CI is automatically built and deployed online when the master branch changes.

Therefore, the Gitlab project configuration should be as follows:

• The master branch is being added.gitlab-ci.ymlThen set it to disallow push
• The master branch can only be merged by the administrator using the Merge Request

The optimization

It is emphasized that the optimization here refers to my own optimization in practice, and does not represent the general type, so you can adopt and try as needed.

• npm -> yarn

There is no doubt about this, as more and more projects have moved from NPM -> YARN in the front-end development experience, and the speed of yarn installation and the use of caching are indeed more suitable for CI.

• Mirror Source Settings

# Set the mirror source to Taobao mirror
- yarn config set registry 'https://registry.npm.taobao.org'
# If you are using Node-sass, which is special because it often fails to install, it is safe to reset it
- yarn config set sass_binary_site 'https://npm.taobao.org/mirrors/node-sass/'
Copy the code

• usegitlab-ci 的 cache

cache:
  key: your-project-name
  paths:
    - $(pwd)/.yarn-cache
    - node_modules/
Copy the code

And the installation command becomes -yarn install –pure-lockfile –cache-folder $(PWD)/. Yarn-cache

With the following configuration, dependency packages will be cached at build time so that the next installation will be faster

Here are some optimized screenshots:

• With NPM: 278s

• Switch to YARN: 196s

• Increase cache: 1s

Of course, 1s here refers to the case where our dependency has not changed, and if it does, it must be reinstalled. But overall, it’s a lot of speed.

conclusion

A very simple article, just to let you can simply understand the front-end automation build, as well as some personal summary. For example, the Docker deployment of Gitlab Runner and actual projects varies from person to person, and not everyone uses Docker front-end deployment, so I don’t give too much introduction. I think it is OK to know Gitlab CI as an introduction