background

K8S is now used in the company for service deployment, and the internal testing environment also provides the K8S environment. When multiple people on the team develop multiple requirements at the same time, it is inevitable that multiple requirements will be tested at the same time. In the Intranet K8S test environment, only one service can be deployed at a time. If multiple requirement branches are merged into one branch, it will inevitably affect each other, and it is inefficient to use the company’s TDC service to handle each update manually. In order to solve this problem, I set up multi-branch test environments respectively for static websites and direct websites on the Intranet physical machine of the company, and cooperated with Gitlab CI in automatic deployment to reduce manual deployment time.

Practice line

At present, I have two projects, one is a normal static website, the other is a website directly out of the server. They’re different in terms of deployment, static sites where we’ve developed and packaged the static files and deployed them to the server and configured Nginx access to the specified directory. A direct project on the server needs to start a service and configure Nginx to forward requests to the corresponding service.

So for the above situation we need the following things:

  1. Output corresponding static files or start different services according to different branches;
  2. By configuringNginxRules to forward requests to corresponding files or services by accessing information on the URL;
  3. configurationGitlab CIAnd write automatic scripts for automatic deployment.

How do you distinguish between branches

A static web site

For static sites, we ended up deploying them in a directory on the server and accessing them through Nginx forwarding. This can be done if we can package different branches into different directories and access them through a ·Nginx configuration.

Each development requirement in a project takes a unique branch name, and the CI_COMMIT_REF_SLUG field is provided when Gitlab performs CI. The value of this field is obtained by converting all branch names to lowercase and all characters except 0-9 and a-z to -, and can be used in urls.

This way we simply pass CI_COMMIT_REF_SLUG into the package script as a parameter, create a new directory under the specified directory, and transform the package file through the past to complete the first phase of the automation script.

The server goes straight out of the website

Server rendered sites need to start services to spit out pages, so our Nginx configuration also points to the corresponding service. The problem is that the project port number is fixed. If we manually change the project port number for each branch, one might go online and forget to change it back. Second, manually confirm whether the corresponding port is occupied. Obviously not reasonable.

It just so happens that the company’s projects are all deployed with Docker, which is installed on K8S. One of the features of Docker is to expose the port inside the container to the outside. With this feature, we can expose different access ports to the outside without changing the project startup port:

docker run  --name $CI_COMMIT_REF_SLUG -P -d <image>:$CI_COMMIT_REF_SLUG
Copy the code

Here we focus on the -p parameter, which randomly assigns a system port to the current container.

However, we need to fix the bug during the test, and redeploy after each commit. We want to maintain the existing port rather than change it every commit. We can obtain the port number of the existing service every time we restart the service:

HOSTPORT=$(docker inspect --format='{{(index (index .NetworkSettings.Ports "3000/tcp") 0).HostPort}}' $CI_COMMIT_REF_SLUG)
Copy the code

This solves the problem of exposing different ports for different branches to start services.

Nginx rule configuration

A static web site

Nginx server_name = Nginx server_name But we can use the re to match subdomains:

server { listen 80; server_name ~^(.+)? \.static\.test\.com$; set $www_root $1; root /data/vhosts/static.test.com/$www_root/; }Copy the code

This is an example of Nginx configuration for my static project. The final link to my static site in the test environment is:

http://$CI_COMMIT_REF_SLUG.static.test.com
Copy the code

According to the regular expression above, when Nginx receives a request, it gets our branch alias and forwards the request to the specified branch directory in the project directory.

The server goes straight out of the website

The configuration of the server side directly out of the site and static site is the same idea, just change the access directory to access services:

server { listen 80; server_name ~^(.+)? .server.test.com$; root html; . set $port $1; Location ~ / {proxy_pass http://127.0.0.1:$port; . }}Copy the code

The final access address is:

http://$port.server.test.com
Copy the code
Rules of the whistle

Because the two projects are opened inside the client in actual production, the two urls given in the test are not the same as the configuration of the client. The whistle configuration rules can be used for transformation, and it supports HTTPS, which is very convenient:

https://static.test.com http://$CI_COMMIT_REF_SLUG.static.test.com
https://server.test.com http://$port.server.test.com
Copy the code

Even after using Whistle, the server can go directly to the website without configuring Nginx and access the corresponding service directly through IP:

https://server.test.com http://ip:$port
Copy the code

The deployment scripts

A static web site

Static web site deployment scripts are relatively simple, just package the code and sync it to the appropriate folder:

#! /bin/sh
CI_COMMIT_REF_SLUG="$1"
BUILD_COMMAND="yarn install --registry https://registry.npm.taobao.org; yarn build;"
TARGET_DIR="/data/vhosts/static.test.com/$CI_COMMIT_REF_SLUG/"
#Start the Docker package file
docker run -v $(pwd)/:/app -w  /app node:10 /bin/sh -c "$BUILD_COMMAND" || exit 1
#To create a specified directory, -p indicates that the directory is recursively created. No error message is reported if the directory already exists
mkdir -p $TARGET_DIR
#Synchronize packaged files to corresponding directories
rsync -av --delete-after ./dist/ $TARGET_DIR
Copy the code
The server goes straight out of the website

The server deployment script is relatively complex, mainly reflected in the update of the container processing:

#! /bin/sh
BUILD_COMMAND="yarn install --registry https://registry.npm.taobao.org; yarn build;"
#The first argument passed later when the script is executed
CI_COMMIT_REF_SLUG="$1"

#Compile package project
docker run -v $(pwd)/:/app -w  /app node:10 /bin/sh -c "$BUILD_COMMAND" || exit 1
#Package the project into the Docker image with the image name attached to CI_COMMIT_REF_SLUG
#To run this command, you need to have a Dockerfile in the project root path, which is not listed here
docker build -t game-frontend:$CI_COMMIT_REF_SLUG .
#Checks whether the container corresponding to the current branch is running
if [ ! "$(docker ps -q -f name=$CI_COMMIT_REF_SLUG)" ]; 
then
		#If a container exists but has exitedif [ "$(docker ps -aq -f status=exited -f name=$CI_COMMIT_REF_SLUG)" ]; Then # delete docker rm $CI_COMMIT_REF_SLUG from docker rm $CI_COMMIT_REF_SLUG $commit_ref_slug -p -d game-frontend:$CI_COMMIT_REF_SLUG HOSTPORT=$(docker inspect --format='{{(index (index.networkSettings. Ports "3000/ TCP ") 0). HOSTPORT}}' $CI_COMMIT_REF_SLUG) echo "Port number: $HOSTPORT" else	#Gets the port number if a container already existsHOSTPORT=$(docker inspect --format='{{(index (index .NetworkSettings.Ports "3000/tcp") 0).HostPort}}' $CI_COMMIT_REF_SLUG) echo"  #Stop the container and delete it
  docker stop $CI_COMMIT_REF_SLUG
  docker rm $CI_COMMIT_REF_SLUG
  #Start a new container based on the previous port number
  docker run --name $CI_COMMIT_REF_SLUG -p $HOSTPORT:3000 -d game-frontend:$CI_COMMIT_REF_SLUG
fi
Copy the code

Configuring automated Deployment

With the above preparations completed, the Gitlab CI is ready to be configured.

Registered Gitlab Runner

We can find CI/CD under Setting in Gitlab project page and install Gitlab Runner as instructed. I won’t go into that here.

Write. Gitlab – ci. Yml
cache:
  untracked: true
stages:
  - dev
before_script:
  - git lfs pull
dev:
  stage: dev
  script:
  # Here is the script written above
  - chmod +x ./scripts/test_dev.sh
  - ls -lsa ./scripts/test_dev.sh
  - ./scripts/test_dev.sh $CI_COMMIT_REF_SLUG
  tags:
  # Specify the Gitlab Runner to use
  - test-dev
Copy the code

At this point, the multi-branch test environment for automated deployment is configured.

Problems encountered

  1. When executing the deployment scriptGitlab runnerPermission issues arise when you need to create a directory somewhere other than its working directory.

Gitlab-runner = gitlab-runner = gitlab-runner = gitlab-runner = gitlab-runner = gitlab-runner = gitlab-runner = gitlab-runner = gitlab-runner

gitlab-runner:x:0:0:GitLab Runner:/home/gitlab-runner:/bin/bash
Copy the code
  1. There is a permission problem when executing the script to pull the docker warehouse image of the company, but I have clearly logged in on my machine. This is becausegitlab runnerReading is/home/gitlab-runner/.docker/Under the account configuration we need inciLogin again indocker.