Front-end engineering and automation of build processes

This is the fifth day of my participation in the August More text Challenge. For details, see: August More Text Challenge

Front-end engineering and automation of build processes

background

In the development of individual small projects, we often directly write the code directly to the server to run on 😂, but in the team development, the above quick and violent approach is obviously not correct. In team development, we often encounter these problems:

1. specification

   Code has no specification, everyone’s style is different, and the quality of code delivery is out of control; There is no specification for committing, and no way for everyone to know what development is committed

2. The efficiency of

   Constant repetition of work, no accumulation of technology and precipitation

3. process

   Lack of r&d process, no product requirements documentation, no iterative requirements management

4. Project quality

   The test function depends on manual discovery and regression, time-consuming and laborious; Code that is not standardized results in poor project quality

5. The deployment of

   Complex manual construction and deployment; Dependence is not uniform; No version tracking, no rollback

The overall result is that the code looks like a lump of 💩💩, giving people a confused and uncomfortable feeling.

engineering

These problems can be solved by introducing an engineered system.

Engineering 🛠 : for the purpose of improving efficiency, reducing cost and ensuring quality, through a series of specifications, processes and tools to achieve research and development efficiency, automation, quality assurance, service stability, early warning and monitoring.

To quote one of the most popular images on the Internet:

A simple DevOps process is cost-effective as a starting point for building engineering on a small team.

I will not discuss Git Flow here because the general infrastructure within the team uses a mature framework and Git Flow has a specification for each platform.

Automate CI/CD building

With a continuous approach to software development, you can continuously build, test, and deploy iterative code changes. This iterative process helps reduce the chances that you will develop new code based on previous versions that are flawed or failed. With this approach, you can strive to reduce human intervention from developing new code to deployment, or even not at all.

The adoption of CI/CD has changed how developers and testers distribute software.

First it was waterfall, then Agile, and now DevOps, which is the path for modern developers to build great technologies. With the rise of DevOps, there are new approaches to Continuous Integration, Continuous Delivery, and Continuous Deployment.

The focus of continuous integration is to bring together the work of individual developers into a code repository. Typically, this is done several times a day, with the primary goal of catching integration errors early and making the team more cohesive and collaborative. The goal of continuous delivery is to minimize the friction inherent in deployment or release. Its implementation typically automates every step of a build deployment so that the code can be safely released at any time (ideally). Continuous deployment is a higher degree of automation that automatically builds/deployable whenever a significant change is made to the code.

Before introducing the following, recognize some key words:

Pipeline, Pipeline

A Pipeline is actually equivalent to a build task, which can contain multiple processes, such as install dependencies, run tests, compile, deploy test servers, deploy production servers and other processes. Any merge of submit or merge requests can trigger a Pipeline

• A Pipeline is equivalent to a (build task) and contains multiple stages
• Any merge of push submit or merge request can trigger a Pipeline

 +------------------+  trigger  +----------------+               
 |   Commit / MR    +---------->+    Pipeline    |
 +------------------+           +----------------+
Copy the code

Stages – The construction phase

Stages represent the construction phase. Multiple Stages can be defined in a Pipeline.

 // .gitlab-cli.yml
 stages:
   - echo
   - build
   - deploy
Copy the code

• Sequential: All Stages run sequentially, that is, when one Stage is completed, the next Stage can start
• Success: Pipeline will be successful only after all Stages are completed
• Failure: As long as there is a failure, subsequent Stages will not be executed and the entire Pipeline will fail

 +--------------------------------------------------------+
 |  Pipeline                                              |
 |  +-----------+     +------------+      +------------+  |
 |  |  Stage 1  |---->|   Stage 2  |----->|   Stage 3  |  |
 |  +-----------+     +------------+      +------------+  |
 +--------------------------------------------------------+
 ​
Copy the code

Jobs – the work of building

Jobs represents the build work, the work performed within a Stage. We can define multiple Jobs in Stages

• Order: Jobs in the same Stage are executed in parallel
• Success: A Stage is successful only when all Jobs in the same Stage are successfully executed
• Failure: If a Job fails, the Stage fails and the entire Pipeline fails

+-------------------------------------------------+ | stage1 | | +-----------+ +------------+ +------------+ | | | Job 1  | | Job 2 | | Job 3 | | | +-----------+ +------------+ +------------+ | +-------------------------------------------------+Copy the code

YAML basis

0. scalar

   • Null: the value is represented by ~

   • Reference:

     • & : Establish anchor points
     • * : Used to refer to the content & represents
     • << : indicates merge to the current data

 defaults: &defaults --------------------------------------------------- &
   adapter:  postgres
   host:     localhost
 development:
   database: myapp_development
   < < : *defaults ------------------------------------------------------- *
 test:
   database: myapp_test
   < < : *defaults
 Is equivalent to
 defaults:
   adapter:  postgres
   host:     localhost
 development:
   database: myapp_development
   adapter:  postgres ------------------------------------------------- * said & Substitute content
   host:     localhost
 test:
   database: myapp_test
   adapter:  postgres
   host:     localhost
Copy the code

2. Array: Use a set of lines beginning with a conjunction to form an array

 stages:
   - build
   - develop
   - release
Copy the code

3. The rest of the consciously consult.

Within the group, we used GitLab CI/CD, an overview of the CI/CD process is as follows:

0. Make sure you have a GitLab Runner, such as Docker, on your server.
1. Create a.gitlab-cli.yml file in the project root directory.

In the.gitlab-cli.yml file, we can configure the gitlab CI/CD specific instructions, such as:

• The runner should execute the structure and order of the and job
• What decision should Runner make in a particular situation

When the code is pushed to the Gitlab repository or mere-request, Gitlab will parse the. Gitlab-cli. Yml file and call the corresponding runner to execute the job in the stage of pipeline.

During development, when we need to verify that.gitlab-cli.yml has syntax errors, use CI Lint. If you want to deploy your code on a private server, you also need an SSH key to access it.

The common parameters in the.gitlab-cli.yml configuration are as follows:

So far, let’s analyze the project used. Gitlab – CLI.yml (desensitization done) :

 # .gitlab-cli.yml
 # Define the build scenario
 stages:
   - build
   - develop
 # Define variables
 variables:
   REGISTRY: https://pkg.xxx.com/abc/api/npm/npmjs
   DOCKER_IMAGE: pkg.xxx.com/docker/test_ubuntu18.04:latest
   NODE_IMAGE: pkg.xxx.com/docker/language/node:lts-slim
   NAME: release/${CI_PROJECT_NAME}
   DEV_NAME: ${CI_PROJECT_NAME}-test
   TAG: ${CI_COMMIT_TAG}
   DIR: pkg.xxx.com/docker-pro/paas
 Job name: build
 build:
   stage: build
   # docker mirror
   image:
     name: ${NODE_IMAGE}
   # Execute script
   script:
     - yarn config set registry $REGISTRY
     - yarn
     - yarn build
   # Pass the dependencies generated by this job to the next task
   artifacts:
     paths:
       - dist/
       - deploy/
   # The job is executed using this runner
   tags:
     - gitops-k8s-product-runners
   # is available on develop and tags branches
   only:
     - develop
     - tags
 ​
 develop:
    # docker mirror
   image:
     name: ${DOCKER_IMAGE}
   stage: develop
    # available on the Develop branch
   only:
     - develop
   when: on_success
   # Define which job the job depends on
   dependencies:
     - build
   Execute the script before executing the script
   before_script:
     - export VERSION=latest
     - export NAME=$DEV_NAME
     - echo "before_script"
     - cp deploy/* .
     - ls
   # Execute script
   script:
     - docker build -t=$DIR/$NAME:$VERSION .
     - docker push $DIR/$NAME:$VERSION
     - docker rmi -f $DIR/$NAME:$VERSION
   # The job is executed using this runner
   tags:
     - gitops-k8s-product-runners
Copy the code

In addition to the.gitlab-cli.yml file, we also found a.gitignore file. Let’s continue to explore the contents of this file:

Gitignore:

Some files we want Git to ignore and not keep track of in our repository include many automatically generated or platform-specific files, as well as other local configuration files, such as:

0. A file containing sensitive information
1. Compiled code such as.dllor.class.
2. System files, such as.DS_StoreorThumbs,db.
3. Files that contain temporary information, such as logs, caches, etc.
4. Generated files such asdistfolder

You can use git rm to stop tracking a file, such as git rm –cached, but that’s too cumbersome. You just need to use the.gitignore file to tell Git not to track which files.

Basic rules for.gitignore files:

0. Any hash (#) The opening lines are comments.
1. The ‘ ‘character can escape special characters.
2. /Character indicates that the rule applies only to files and folders in the same folder.
3. An asterisk (*) represents any number of characters (zero or more).
4. Two asterisks (天安门事件) represents any number of subdirectories.
5. A question mark (?) instead of zero or one character.
6. An exclamation mark (!) reverses specific rules (that is, includes any files excluded by the previous schema).
7. Blank lines are ignored, so you can use them to add space and make your document easier to read.
8. Add at the end/The entire directory path is ignored.

Example:

 # dependencies
 /node_modules
 /npm-debug.log*
 /yarn-error.log
 # /yarn.lock
 /package-lock.json
 ​
 # production
 /dist
 /build/dist
 ​
 # misc
 .DS_Store
 ​
 # umi
 /src/.umi
 /src/.umi-production
 /src/.umi-test
 /.env.local
 public/config.js
Copy the code

At this point, when we push the project code submission to the functional branch in the remote repository in GitLab, the push triggers the project’s CI/CD pipeline. Then, Gitlab runs CI/CD:

• Run automated scripts (sequential or parallel) :

  • Build or test our project code
  • Changes can be previewed in the Review App

Work as expected after implementation:

• Obtain code reviewed and approved

• Merge the feature branch onto the default branch

  • GitLab CI/CD automatically deploys the changed code to the production environment.

If something goes wrong, you can roll back the changes.

This workflow shows the important steps in the GitLab process, and we can visualize all the steps in the GitLab UI.

conclusion

Following an engineered development process and forming a specification can greatly improve team efficiency.

Reference links:

• docs.gitlab.com/

• Juejin. Cn/post / 698221…

• Git-scm.com/docs/gitign…