This article will show you how to use Docker+Jenkins to achieve continuous integration of Go language projects.

First of all, let’s talk about the general implementation idea:

Pull or clone code from Git server => compile code into binary executable file => Build Docker image => upload Docker image to image repository => Enter the remote application server from the Jenkins server => pull the image from the Docker image repository => Stop and delete the docker container that the project is running => Build and start the Docker container with the new image => delete other old versions of the image => complete

Then talk about some of the techniques used in the practice process:

  • Docker

    The main reason for using Docker is that it is convenient for application transplantation and can effectively solve environmental problems caused by the deployment of the same application on different servers.

  • Docker Compose

    Docker-compose is also a command line tool provided by Docker, which is mainly used to simplify the operation of containers on Jenkins server. It can be directly used instead of shell script.

  • Jenkins

    Jenkins is an automated server and can also be used as a simple CI/CD server.

  • Git

    Git is also used as a code management repository for the project, and uses its Web Hooks as triggers for the Jenkins task to build.

  • Mirror warehouse

    Here choose ali cloud container services as our mirror warehouse, you can choose their own appropriate mirror warehouse.

At the end of the article, the problems that might arise during the practice are listed and the Go project repository used by the practice is provided.

Install Jenkins

Start Jenkins container

  1. Pull Jenkins mirror image

    docker pull jenkins/jenkins:latest
Copy the code

  2. Write the docker-comemess. yml file

    version: "2"
services:
	jks:
		image: jenkins/jenkins:latest
		ports:
			- 8080: 8080
			- 50000: 50000
		volumes:
			- /data/jenkins:/var/jenkins_home
			- /var/run/docker.sock:/var/run/docker.sock
			- /bin/docker:/bin/docker
		container_name: jks
		user: root
Copy the code

  3. Create a/ data/ Jenkins folder for mounting and change the owner of this folder to a user with ID=1000

    mkdir -p /data/jenkins
chown -R 1000:1000 /data/jenkins
Copy the code

  4. Start the Jenkins container with Docker-compose

    docker-compose -f /data/compose/docker-compose.yml up -d jks
Copy the code

Install Jenkins plugins

  1. Enter http://IP:8080 in your browser to go to the Jenkins Management page

  2. Through the command cat/data/Jenkins/secrets/initialAdminPassword obtain initial password, to unlock Jenkins, unlock after skip plug-in installed (without having to worry about packing recommended plug-ins, in the subsequent installation process, All plug-ins recommended after unlocking will be automatically installed)

  3. (Optional) Modify the configuration file and restart the Jenkins container to accelerate plug-in installation

    #Edit the Jenkins configuration file
vim /data/jenkins/updates/default.json

#Run the ex command to replace all plug-in download urls
:1,$s/http:\/\/updates.jenkins-ci.org\/download/https:\/\/mirrors.tuna.tsinghua.edu.cn\/jenkins/g

#Run the ex command to replace the connection test URL
:1,$s/http:\/\/www.google.com/https:\/\/www.baidu.com/g

#Restart Jenkins container
docker-compose -f /data/compose/docker-compose.yml restart jks
Copy the code

  4. Go to the Jenkins -> Plugin Manager page to install the following plug-ins

    • Localization: Chinese (Simplified)
    • Publish Over SSH
    • Gitee
    • Go

Configuration Jenkins

Global Tool Configuration

Go to the Jenkins -> System Administration -> Global Tools configuration page to configure Git and Go

Gitee configuration

On the premise that the Gitee plug-in has been installed, go to Jenkins -> System Management -> System Configuration, find the Gitee configuration, and configure it. The configuration content is roughly as follows:

Configure Publish over SSH

After installing the Publish over SSH plugin, go to Jenkins -> System Administration -> System Configuration and find the Publish over SSH configuration, as shown below:

Passphrase: encryption password used to generate the key. If the value is empty, there is no encryption password

Path to key: specifies the location of the generated private key

Key: The content of the generated private Key. Only one of the Path to Key options is required

Name: Name of the remote server remarks (custom)

Hostname: indicates the address of the remote server

Username: indicates the Username of the remote server

Remote Directory: Indicates the location of the Remote server

There are two main configuration methods. The first is to directly use the account and password of the remote server, and the other is to use the key.

  • Account password mode

    To Use the account and password mode, you do not need to configure the public and private keys. Directly configure the SSH Server, click the Advanced button, and select Use Password Authentication, or Use a different key. Then fill in the Password Passphrase/Password corresponding to the remote server user, and click the Test Configuration button below to Test the connection.

  • The key way to

    1. Enter Jenkins container

      docker exec -it jks /bin/bash
Copy the code

    2. Generate the key

      ssh-keygen -t rsa
Copy the code

    3. Copy the public key to the remote server

      SSH - copy - id - I ~ /. SSH/id_rsa pub 192.168.56.101Copy the code

    4. Set the Path to key option to /root/.ssh/id_rsa

    5. Configure SSH Server

    6. Connect the test

Build tasks

  1. Go to Jenkins -> New Task to create a Jenkins task

  2. General

    Fill in the Gitee link for the project and check discard the old build.

  3. Source code management

    Select the Git option, fill in the information for Repositories, and create the Gitee account password credentials.

  4. Build trigger

    Check the Gitee Webhook trigger build, select the branch to allow trigger build as required, and click the Generate button to the right of the Gitee Webhook password option below. Fill in the WebHook URL and WebHook password for the trigger into the Gitee WebHooks administration for the project.

  5. Build environment

    On the premise that Go plug-in has been installed, check the box of Set Up Go Programming Language Tools and fill in the Go version number used by the project below, as shown below:

  6. build

    For example, in the Jenkins container, the current location is /var/jenkins_home/workspace build task name.

    The construction process can be roughly divided into the following steps:

    • usego buildCommands are compiled into binary executable files
    • Implement what is written in advancedocker-pushScript, compileDocker mirrorAnd upload toThe docker warehouse

    #1. Compile the executable file
go build -o app

#2. Run the shell script to upload the Docker image
chmod +x docker-push && ./docker-push
Copy the code

    Dockerfile contains the following contents:

    FROM golang:1.14.2

COPY app .
COPY conf/conf.toml ./conf/

ENTRYPOINT ["./app"]
Copy the code

    The docker-push script reads as follows:

    #Container name
name=`cat version | awk '{print $1}'`
#Container labels
tag=`cat version | awk '{print $2}'`
#The warehouse of the domain name
domain=registry-vpc.cn-shenzhen.aliyuncs.com
#Warehouse URL
url=colehuang/coletest

#Build the Docker image
docker build -t $name:$tag .

#Obtaining the Mirror ID
id=`docker images | grep $name | grep $tag | awk '{ print $3 }'`

#Upload the image to the Docker image repository
docker login --username=xxx --password=xxx $domain
docker tag $id $domain/$url:$tag
docker push $domain/$url:$tag
Copy the code

    The version file contains the following contents:

    GDD 1.1.2Copy the code

  7. Post-build operation

    The post-build operation process can be roughly divided into the following steps:

    • The projectversionanddocker-pullThe file is transferred to the remote server
    • To enter inPublish over SSHTo the remote server directory configured in thedocker-pullFrom the script,Docker image repositoryPull the submitted new version image and build with itDocker container

    The content of the docker-pull script is as follows:

    #Container name
name=`cat version | awk '{print $1}'`
#Container labels
tag=`cat version | awk '{print $2}'`
#The warehouse of the domain name
domain=registry.cn-shenzhen.aliyuncs.com
#Warehouse URL
url=colehuang/coletest

#Pull an image from the Docker image repository
docker login --username=xxx --password=xxx $domain
docker pull $domain/$url:$tag

#Stop the Docker container in which the image is runningline=`docker ps | grep $name` if [ -n "$line" ]; Then echo "There is a running $name container and it is being stopped..." Docker stop $name echo "$name
#Delete the Docker container for the imageline=`docker ps -a | grep $name` if [ -n "$line" ]; Then echo "$name"; Docker rm $name echo "$name
#Start the container
docker run --rm --name $name -p 8008:8006 -d $domain/$url:$tag

IFS=$'\n'

#Deleting Unnecessary Mirrorsimages=`docker images | grep $domain/$url` for i in $images do echo $i t=`echo "$i" | awk '{print $2}'` if [ "$t" ! = "$tag" ]; then id=`echo "$i" | awk '{print $3}'` docker rmi $id fi doneCopy the code

  8. Basically completed

    When all of the above is configured, Git WebHooks are triggered when code is pushed locally to a Git remote repository, enabling automatic updates and deployment to the server project.

Contents summary

This article briefly explains how to use Docker and Jenkins to realize the continuous integration of Go language project. At the end of this article, we talk about the points needing attention in the practice process.

First of all, when the Jenkins server pulls the code of the remote repository by git pull command, if it is a private repository, it may need to input the Git account and password. The method used here is to record the Git account and password by setting the git global variable.

git config --global credential.helper store
Copy the code

Then in the final post-construction operation, the author will have a phenomenon: when the remote server obtains the image from the Docker image warehouse for the first time, the pull speed is slow and timeout occurs because the remote server does not have other images that the image depends on locally. The solution to this problem can be to change the timeout period or go to the remote server and manually pull it again.

Finally, attach a code repository consistent with the content of the demo project: gitee.com/hkail/go-do…

You may need to modify the following items in practice:

Docker-push script file: domain repository domain name, url repository URL, username and password after docker login command

In the docker-pull script file: mapping port number after the docker run command is used in the docker-pull script file

Finally, thank you for your patience