demand

In the Intranet environment, the company needs to build a CI/CD environment of GITLab by itself

The environment

  • windowsServer2008R
  • Centos 7.5.1804

Implementation plan

Plan a

Install centos on Windows Server2008R virtual machine, and make the virtual machine can access the network by bridging

  • Install centos7.5 using hyper-v or vmware
  • Docker is installed in centos7.5
  • Docker run gitlab, gitlab-runner

The results of

Gitlab takes up a lot of resources, and the official website recommends at least 4G memory for running. WinServer itself needs to run a lot of JAVA services, so we allocated 4G memory to the virtual machine, but it only ran docker-GitLab, and the virtual machine is very prone to deadlock. So I abandoned this solution and kept only Gitlab-Runner on the virtual machine and used it to store data backups for GitLab

This solution is the same as solution 2 except that vm installation, network processing, shared mounting and other aspects need to be handled separately

Scheme 2

  • Install docker in centos on Linux server
  • Docker runs Gitlab, GitlabRunner

The results of

This solution has more relevant information and is relatively simple to get started, which does not involve the performance, network, mount and other problems of virtual machines. If conditions are available, the server will be on the server. The consideration of simple and reliable use of Docker is based on the Intranet environment. Through the production of images can be more convenient to transplant and maintenance of the current running well

Install Docker

Prepare the environment

Due to the internal network environment, it is relatively troublesome to install Docker offline. Multiple RPM packages need to be downloaded for installation and update. The packages to be used are listed here. In mirrors.163.com/centos/7/os… Download the following RPM installation package:

Audit - libs - python - 2.8.5-4. El7. X86_64. RPM checkpolicy - 2.5-8 el7. X86_64. RPM libcgroup - 0.41-21. El7. X86_64. RPM Libsemanage - python - 2.5-14. El7. X86_64. RPM libtool - LTDL - - 22 2.4.2. El7_3. X86_64. RPM Policycoreutils - python - 2.5-33. El7. X86_64. RPM python - IPy - 0.75-6. El7. Noarch. RPM setools - libs - 3.3.8-4. El7. X86_64. RPM Audit - 2.8.5-4. El7. X86_64. RPM audit - libs - 2.8.5-4. El7. X86_64. RPM libselinux - 2.5-14.1. El7. X86_64. RPM Libselinux - python - 2.5-14.1. El7. X86_64. RPM libselinux - utils - 2.5-14.1. El7. X86_64. RPM libsemanage - 2.5-14. El7. X86_64. RPM Libsepol - 2.5-10. El7. X86_64. RPM policycoreutils - 2.5-33. El7. X86_64. RPMCopy the code

In download.docker.com/linux/cento… Download docker installation package:

Docker - ce - 17.12.0. The ce - 1. El7. Centos. X86_64. RPMCopy the code

In rpm.pbone.net/index.php3/… Download the container-selinux installation package from

The container - selinux - 2.9-4. El7. Noarch. RPMCopy the code

Here, we did not divide it into update packages and install packages according to the method in the reference post, because in the actual debugging, there would be a interdependence between the installation and update packages, leading to the update or installation failure, so I directly use the indiscriminate forced installation to override the installation.

The installation

Download the dependencies into a package first

tar cf docker-ce.offline.tar *.rpm
Copy the code

Then copy the compressed package to the Intranet environment and decompress it for installation

Tar // Decompress sudo RPM -ivh --force *. RPM to install sudo RPM -uvh *. RPM --force *Copy the code

Here use force install –force to overwrite the installation, if the prompt does not rely on, then upload the installation can be

Start the docker

systemctl start docker 
systemctl enableDocker // Set docker version to boot // Verify that docker is successfully installedCopy the code

If the following information is displayed, the Docker is successfully installed

Client: Version: 17.12.0 -CE API Version: 1.35 Go Version: go1.9.2 Git commit: c97c6d6 Built: Wed Dec 27 20:10:14 2017 OS/Arch: linux/amd64Copy the code

Ii. Install GITlab-CE

Prepare the environment

We first pulled the mirror image from the Internet machine, and here we pulled the mirror image of Gitlab-Runner

docker pull gitlab/gitlab-ce
docker pull gitlab/gitlab-runner
docker images
Copy the code

There is no restricted version here, I have gitLab version 12.10.3 here

REPOSITORY             TAG                 IMAGE ID            CREATED             SIZE
gitlab/gitlab-runner   latest              f726de7cf9ee        11 days ago         443MB
gitlab/gitlab-ce       latest              f2e48729e35c        3 weeks ago         2GB
Copy the code

The installation

Then we need to save the image and upload it to the Intranet machine

Docker save gitlab IMAGE ID > gitlab.tar docker save gitlab-runner IMAGE ID > gitlab-runner load < gitlab.tar docker load < gitlab-runner.tar docker imagesCopy the code

If the image list is displayed, the import is successful

REPOSITORY             TAG                 IMAGE ID            CREATED             SIZE
gitlab/gitlab-runner   latest              f726de7cf9ee        11 days ago         443MB
gitlab/gitlab-ce       latest              f2e48729e35c        3 weeks ago         2GB
Copy the code

PS: If the image name is None, run the docker tag imageid name:tag command to rename the image

Environment configuration

To facilitate upgrades and backups, put the configuration in Gitlab out of the container and prepare three folders under the/MNT /gitlab directory

mkdir -p /mnt/gitlab/etc
mkdir -p /mnt/gitlab/log
mkdir -p /mnt/gitlab/data
Copy the code

Then write the gitlab running script start.sh in the /gitlab folder

#! /bin/sh
GITLAB_HOME=/mnt/gitlab     Create gitLab local directory
docker stop gitlab           # Stop before gitLab container
docker rm gitlab             Delete the previous GitLab container
docker run \
--detach \
--hostname XXX.XXX.XXX.XXX `The gitlab domain name is not configured on the Intranet, so use the server IP address to access '\
-p 8443:443 `# container port 443 maps to host port 8443 (HTTPS) '\
-p 8090:80 `# container port 80 is mapped to host port 8090 (HTTP)
-p 2222:22 `# Container port 22 mapped to host port 2222 (SSH) '\
--name gitlab `# container name '\
--restart unless-stopped `# Container restarts automatically
-v $GITLAB_HOME/etc:/etc/gitlab `# mount local directory to container configuration directory '\
-v $GITLAB_HOME/log:/var/log/gitlab `# mount local directory to container log directory '\
-v $GITLAB_HOME/data:/var/opt/gitlab `# mount local directory to container data directory '\
gitlab/gitlab-ce                       # Mirror used: version
Copy the code

Gitlab configuration

1. Modify the/MNT/gitlab/etc/gitlab rb
vim /mnt/gitlab/etc/gitlab.rb

external_url 'http://XXX.XXX.XXX.XXX:8090'  Change external_url to the domain name or IP address of the deployment machine
gitlab_rails['gitlab_shell_ssh_port'] = 2222 # Change the SSH port to 2222 due to port mapping, which is consistent with container configuration, so that the normal SSH address can be obtained during clone

Copy the code
2. Modify the/MNT/gitlab/data/gitlab – rails/etc/gitlab yml
vim /mnt/gitlab/data/gitlab-rails/etc/gitlab.yml
Copy the code

Find the keyword ## Web Server Settings and change the value of host to the mapped external host IP address and port, which will show the gitlab clone address

## Web server settings
host XXX.XXX.XXX.XXX
port 8090
Copy the code

Start the gitlab

Run the script

sh start.sh
docker ps
Copy the code

The operation information of GitLab indicates that the operation is successful

CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                 PORTS                                                 NAMES
993236646a3e        gitlab/gitlab-ce    "/assets/wrapper"11 days ago Up 2 hours (healthy) 22/ TCP, 0.0.0.0:8090->80/ TCP, 0.0.0.0:8443->443/ TCP gITlabCopy the code

The IP address of http://gitlab is 8090. If you log in as user root, a message is displayed asking you to set the root password. After that, you can log in to the administrator page as user root

Install gitlab-Runner

I have seen the post that it is better not to install Runner and Gitlab on the same machine. The specific reason is not detailed, but we will configure multiple Runners according to the server resources later, so it is not considered here. CI/CD facility construction will be completed in Docker first, and the specific installation and registration methods of other environments are basically the same

Prepare the environment

The gitlab-Runner image has been loaded in the previous step, so we will leave out the instructions in this step.

configuration

Again, prepare a configuration folder under/MNT /gitlab-runner

mkdir -p /mnt/gitlab-runner/etc
Copy the code

Prepare the gitlab-runner script start.sh under /gitlab-runner

$ GITLAB_HOME = /mnt/gitlab-runner     Create gitlab-runner local directory
$ docker stop gitlab-runner           # Stop before gitLab container
$ docker rm gitlab-runner             Delete the previous GitLab container
$ docker run -d \
--restart unless-stopped  \             The container restarts automatically
-v $GITLAB_HOME/etc:/etc/gitlab-runner \        Mount the local directory to the container configuration directory
gitlab-runner                      # Mirror used: version
Copy the code

Start the gitlab – runner

Run start.sh to start the gitlab-Runner container

sh start.sh
docker ps
Copy the code

At this point, we can see both the GitLab and gitlab-Runner containers running

registered

Then we need to register runner, we first register a shared runner test, the process is the same, first open the GitLab console, Then go to Admin Area > Overview > Runners and find Set up a shared Runner manually, duplicate the URL and token that Runners need to register, and type the command on the console to start registering

docker exec -it gitlab-runner bash Enter the container bash
gitlab-runner register # registered runner

Enter GitLab address
Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):
http://XXX.XXX.XXX.XXX:8090/

Enter the GitLab Token
Please enter the gitlab-ci token for this runner:
1Lxq_f1NRfCfeNbE5WRh

Enter the description of Runner
Please enter the gitlab-ci description forThis runner: can be empty# set Tag, which can be used to specify that a CI is triggered when a specified Tag is built
Please enter the gitlab-ci tags for this runner (comma separated):
shared-runner

# Select true, which can be used to execute code directly after upload
Whether to run untagged builds [true/false] :true

Select false and press Enter. Default is false
Whether to lock Runner to current project [true/false] :false

# Select runner executor, here we select shell
Please enter the executor: virtualbox, docker+machine, parallels, shell, ssh, docker-ssh+machine, kubernetes, docker, docker-ssh:
shell

Copy the code

test

At this point, we added.gitlab-ci.yml script to the project to test CI/CD with the shared-runner tag just registered

stages:
  - install_deps
  - test
  - build
  - deploy_test
  - deploy_production

cache:
  key: ${CI_BUILD_REF_NAME}
  paths:
    - node_modules/
    - dist/

# install dependencies
install_deps:
  stage: install_deps
  only:
    # - develop
    - master
  script:
    # - npm install
    - echo 'Simulate installation dependency Phase'
  tags:
    - shared-runner

Run the test case
test:
  stage: test
  only:
    # - develop
    - master
  script:
    # - npm run test
    - echo 'Simulate running test Case phase'
  tags:
    - shared-runner
# compiler
build:
  stage: build
  only:
    # - develop
    - master
  script:
    # - npm run clean
    # - npm run build:client
    # - npm run build:server
    - echo 'Simulated compile phase'
  tags:
    - shared-runner
    
Deploy the test server
deploy_test:
  stage: deploy_test
  only:
    # - develop
    - master
  script:
    # - pm2 delete app || true
    # - pm2 start app.js --name app
    - echo 'Simulate deployment test server phase'
  tags:
    - shared-runner

# Deploy the production server
deploy_production:
  stage: deploy_production
  only:
    # - develop
    - master
  script:
    # - bash scripts/deploy/deploy.sh
    - echo 'Simulate the production server deployment phase'
  tags:
    - shared-runner
    
Copy the code

Then, we can check whether the project is successful in the CI/CD position in the project code base. Status is passed for success. Click status to enter the pipeline and check the details and logs of each stage

Pit point

Fatal: Unable to access 'http://...

In the pipeline log, if runner fails to pull git library, it is likely that runner’s Clone address is incorrect. We need to modify/MNT /gitlab-runner/etc/config.toml to find [[runners]] and add clone_URL to solve the problem that runner can’t pull git library code

clone_url = "http://XXX.XXX.XXX.XXX:8090/"
Copy the code

Four, backup,

The default backup directory of GitLab is /var/opt/ GitLab /backups. You can change the default backup directory of /etc/gitlab/gitlab.rb

gitlab_rails['backup_path'] = '/var/opt/gitlab/backups'
Copy the code

Manual backup

Gitlab -rake gitlab:backup:createexec -it gitlab gitlab-rake gitlab:backup:create
Copy the code

Can see we map the directory/MNT/gitlab/data backup/backups in a new package

Automatic backup

We first in/MNT/gitlab/data/backups, adding an automatic backup execution script auto_backup. Sh

docker exec gitlab gitlab-rake gitlab:backup:create
Copy the code

We manually execute the test script is written correctly, see the generated backup package

sh auto_backup.sh
Copy the code

Then, the crontab service is used to add scheduled tasks for automatic backup. We configure the crontab service to add scheduled backup tasks

crontab -e
Copy the code
Run the automatic backup script at 23:00 every day
0  23   * * *   /mnt/gitlab/data/backups/auto_backup.sh
Copy the code

Then, you need to restart the CRon service

Reload the cron configuration file
systemctl reload crontab 
Restart the CRon service
systemctl restart crontab
Copy the code

After a scheduled task is executed, you can view the automatic backup package in the backup directory

You can view scheduled task execution logs in /var/log/cron

tail -300 /var/log/cron
Copy the code

Pit point

Notice The backup command in the automatic backup script file cannot contain the it command. The reason is that a terminal is started after the exec parameter is added. Therefore, scheduled tasks cannot enter any terminal and the command is invalid

/var/spool/mail/root = 755; /var/spool/mail/root = 755; /var/spool/mail/root = 755

If there is no problem with file permissions, then it is probably selinux. Go to /ect/selinux/config and set selinux to disabled. This pitfall will cause root to have no permissions

Dual-machine backup

Periodically copy our backup files to the Linux server via SCP to prevent our server from having problems and be able to retrieve code assets

First we need to pass our SSH public key to the remote server, or execute it if the secret key has not been generated before

[root@centos-linux ~]ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):  # Path where the key is stored
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase):  Enter the password. Once you enter your password, you must enter it every time after that. Please decide whether you need a password based on your security needs. If not, press Enter
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:di6MX8faj+TPOBvZwL60ajEJDgjrcO1peb209P/R3EU [email protected]
The key's randomart image is: +---[RSA 2048]----+ | | | . | | + . E| |. o o . . . . | | + . o +S... o .| | . = .+=o+o + oo| | . .. oo+ooO o +| | .oooB.B . | | .. ooX*= | +----[SHA256]-----+Copy the code

If the key has been generated before, the command line will prompt you to overwrite it and press Enter

Find the public key file id_ras.pub in ~/. SSH and upload it to the remote server

SCP id_rsa.pub Server login username (root) @XXX.XXX.XXx. XXX(server address):/ TMP /id_rsa.pub.gitlabCopy the code

SSH: /home/XXX /.ssh: /home/XXX /.ssh: /home/XXX /.ssh: /home/XXX /.ssh: /home/XXX The ssh-keygen command is recommended

Find the authorized_keys file in the ~/.ssh directory, if not, create one

Create authorized_keys file
touch authorized_keys
Copy the code

Append/TMP /id_rsa.pub.gitlab to authorized_keys

cat id_rsa.pub.gitlab >> ~/.ssh/authorized_keys
Copy the code

Then go back to the GitLab server and log in to the remote backup server using SSH to test whether the public key is successfully paired

SSH user name (root in this example)@ Address of the remote serverCopy the code

If no password is prompted, success

If it fails, check whether the copy process is successful, check whether the appended public key in authorized_keys is the same as the public key of the Gitlab server, and then check the problem through debugging information

Ssh-vvv User name (root in this example)@ Remote server ADDRESSCopy the code

SSH has requirements on directory permissions. The value of ~/. SSH is 700, and that of ~/. SSH /* is 600

First in/MNT/gitlab/data/backups/add an automatic backup of executing scripts auto_scp. Sh

#! /bin/bash
# gitlab Backup file path
BACKUPDIR=/mnt/gitlab/data/backups

Log in to a remote backup server as user root.
RemoteUser=root

# IP address of the remote backup server
RemoteIP=XXX.XXX.XXX.XXX

# Current system date
DATE=`date "+%Y-%m-%d-%H:%M:%S"`

#Log storage path
LogFile=$BACKUPDIR/log/$DATE.log

Gitlab backup files in local backup directory within 1 day with suffix.tar
BACKUPFILE_SEND_TO_REMOTE=$(find $BACKUPDIR -type f -mmin -1440 -name '*.tar')

Create a log file
touch $LogFile

echo "[$DATE] backup gitlab" >> $LogFile

Print the address of the remote server
echo "[$DATE] Remote server address:$RemoteUser@$RemoteIP:$BACKUPDIR" >> $LogFile

Print the filename of each SCP file
echo "[$DATE] Backup files:$BACKUPFILE_SEND_TO_REMOTE" >> $LogFile

Backup to remote server
scp $BACKUPFILE_SEND_TO_REMOTE $RemoteUser@$RemoteIP:$BACKUPDIR

Copy the code

Run the auto_scp.sh command to view backup logs in the Log folder. Go to the backup directory on the remote server to view backup files

Then, add scheduled tasks to crontab

Run the automatic backup script once a day at 23:30
30  23   * * *   /mnt/gitlab/data/backups/auto_scp.sh
Copy the code

After that, you can also do some scheduled operations such as automatic deletion of backup files and automatic recovery of GitLab. For details, please refer to the steps in the references, which will not be described here

5. Remote synchronization

Our requirement is that we have a local library and a remote library that we manage separately, but updates to the local library code need to be pushed to the remote

On the project Repository page, choose Settings->Repository from the left menu bar, select Push to a remote Repository, select Remote Mirror Repository, and fill in Git Repository URL

The URL here needs to carry the account password information

http://[userName]:[password]@XXXX.git
Copy the code

Then submit the code once to test it. If there is an error message, it will be displayed in the Tip popover of Statu

conclusion

Thank you for your reference to the authors of the article, for the construction of a lot of reference and help, here the overall process of the construction of the sorting out, there are wrong places, please correct

Refer to the link

  • CentOS7 Docker is deployed offline
  • Docker installation configuration gitLab detailed process
  • Gitlab, GitLab Runner automatically deploys docker containers
  • Gitlab automatically backs up and deletes periodically
  • GitLab from installation to full automatic backup through-train