Jenkins Pipeline + GitLab + Docker to achieve CI/CD function

Jenkins Pipeline + GitLab + Docker to achieve CI/CD function

Create a task

As with the previous manual build task, a parameter is added to mark whether the build or rollback operation is performed

Let’s start writing the pipeline script for the build operation

The first stage of preparation

This phase involves checking out the code from gitlab to local for subsequent operations

stage('Source') {
       git branch: 'master'.credentialsId: 'a4f6f384-4d9c-4dcf-bc1c-c3df536cdbdc'.url: '[email protected]:jeff/citest2.git'
       def container_name = 'citest'
       def git_tag = sh(script: 'git describe --always --tag'.returnStdout: true).trim() + '_manual_' + env.BUILD_NUMBER
       def container_full_name = container_name + The '-' + git_tag
       def repository = 'registry.cn-shanghai.aliyuncs.com/xxx/' + container_name + ':' + git_tag
       println 'container_full_name: ' + container_full_name 
       println 'repository: ' + repository
}
Copy the code

Click Apply test effects to build the task output

using credential a4f6f384-4d9c-4dcf-bc1c-c3df536cdbdc
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url [email protected]:jeff/citest2.git # timeout=10
Fetching upstream changes from [email protected]:jeff/citest2.git
 > git --version # timeout=10
using GIT_SSH to set credentials 
 > git fetch --tags --progress [email protected]:jeff/citest2.git +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 87a5f818724b3954e87363ec271b095355f11d69 (refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 87a5f818724b3954e87363ec271b095355f11d69
 > git branch -a -v --no-abbrev # timeout=10
 > git branch -D master # timeout=10
 > git checkout -b master 87a5f818724b3954e87363ec271b095355f11d69Commit message: "Fix executable file not running with packaged image" > git rev-list --no-walk 87a5f818724b3954e87363ec271b095355f11d69 # timeout=10[Pipeline] sh + git describe --always --tag [Pipeline] echo container_full_name: Citest - version 1.3.3 _manual_4 [Pipeline] echo the repository: Registry.cn-shanghai.aliyuncs.com/xxx/citest:v1.3.3_manual_4 [Pipeline]} [Pipeline] / / stage/Pipeline} [Pipeline] / /  node [Pipeline] End of Pipeline Finished: SUCCESSCopy the code

Check that the output is ok, and the warehouse checkout is done here

Note: The point here is that if the repository checked out by the code requires authentication information, you need to explicitly pass the credentialsId to use the credentialsId configured in Jenkins. This ID can also be customized. If you do not need to authenticate the repository, you need to explicitly pass the credentialsId to use the credentialsId configured in Jenkins. You can use the Checkout SCM directly to checkout code

The second step is to build the Docker image to Aliyun Registry

node {
   stage('Source') {
        git branch: 'master'.credentialsId: 'a4f6f384-4d9c-4dcf-bc1c-c3df536cdbdc'.url: '[email protected]:jeff/citest2.git'
        def container_name = 'citest'
        def git_tag = sh(script: 'git describe --always --tag'.returnStdout: true).trim() + '_manual_' + env.BUILD_NUMBER
        container_full_name = container_name + The '-' + git_tag
        repository = 'registry.cn-shanghai.aliyuncs.com/xxx/' + container_name + ':' + git_tag
        println container_full_name
        println repository
   }
   
   stage('Build') {
        docker.withRegistry('https://registry.cn-shanghai.aliyuncs.com'.'ali_docker_registry') {
            def customImage = docker.build(repository)
            customImage.push()
            
            sh 'docker images | grep citest | awk \'{print $1":"$2}\' | xargs docker rmi -f || true'
            sh 'docker rmi -f `docker images | grep \'
      
       \' | awk \'{print $3}\'` || true'
      }}}Copy the code

Here add a new step, build the docker image of the code repository of pull, and then push it into Ali Cloud Registry. The authentication confidence here is also saved in Jenkins’ global credential, and the id is defined as ali_docker_registry

** Note: ** This variable, which uses a def declaration for each stage, is not accessible from other stages, but can be accessed if the declaration is removed

After the manual test ali cloud mirror warehouse successfully created the information mirror

The third step tells the target machine to automatically pull the image and execute it

Here are three ideas

1. Connect to the target machine through SSH, and then execute the pull command and clean up the operation, which is not very different from the previous operation, just wrapped with pipeline
2. By configuring the target machine as a Jenkins node machine and only executing the specified job, which is then executed by the specified node in the later deployment stages, an explicit SSH connection process is reduced, but the pipeline is written more
3. Configure the remote docker server TSL remote access, the advantage is that pipeline docker library can be directly used to complete the connection and operation, the official demo is the same way, but did not find the way to delete the image, The main purpose of this plug-in is probably for build purposes rather than for automatic deployment

So ultimately, the first idea is used to continue with the automated deployment feature

Connect to other hosts through SSH Pipeline Steps

Here, the connection information of the remote host is saved to the global Jenkins credential for easy use, and then SSH pipeline Steps is used to complete the SSH connection action. See SSH Steps for Jenkins Pipeline

First, install the SSH Pipeline Steps plug-in

Then add the connection user information and password for a remote host connection

Then proceed to write the pipeline script to perform the automatic deployment operation remotely using SSH

stage('deploy') {
       
    withCredentials([usernamePassword(credentialsId: 'pro4_ssh'.passwordVariable: 'password'.usernameVariable: 'userName')]) {

        def remote = [:]
        remote.name = 'pro4'
        remote.user = userName
        remote.host = '47.111.111.111'
        remote.password = password
        remote.allowAnyHosts = true

        writeFile file: 'poll.sh'.text: """ docker ps | grep citest | awk '{print \$2}' >> /data/jenkins/mi_test_history echo /data/jenkins/mi_test_history docker ps | grep citest | awk '{print \$1}' | xargs docker kill || true docker images | grep citest | awk '{print \$1":"\$2}' | xargs docker rmi -f || true docker login --username=yourName --password=yourPassword registry.cn-shanghai.aliyuncs.com docker pull ${repository} docker run -d ${repository} """

        sshPut remote: remote, from: 'poll.sh'.into: '. '
        sshScript remote: remote, script: 'poll.sh'
        sshRemove remote: remote, path: 'poll.sh'}}Copy the code

Note that if you want to use a concatenation variable in a string you must use “wrap the contents of the string instead of”

Manual build task to view the result, the mirror successfully run, and record the previous mirror records

Parameter verification Completes the rollback function

No problem with the deployment. Migrate the rollback script directly to the current pipeline writing. The complete script is shown below

node {
    
    if (env.CTRL_BUILD == 'true') {
    
        stage('Source') {
            git branch: 'master'.credentialsId: 'a4f6f384-4d9c-4dcf-bc1c-c3df536cdbdc'.url: '[email protected]:jeff/citest2.git'
            def container_name = 'citest'
            def git_tag = sh(script: 'git describe --always --tag'.returnStdout: true).trim() + '_manual_' + env.BUILD_NUMBER
            def container_full_name = container_name + The '-' + git_tag
            repository = 'registry.cn-shanghai.aliyuncs.com/xxx/' + container_name + ':' + git_tag
            println container_full_name
            println repository
        }
       
        stage('Build') {
            docker.withRegistry('https://registry.cn-shanghai.aliyuncs.com'.'ali_docker_registry') {
                def customImage = docker.build(repository)
                customImage.push()
                
                sh 'docker images | grep citest | awk \'{print $1":"$2}\' | xargs docker rmi -f || true'
                sh 'docker rmi -f `docker images | grep \'
      
       \' | awk \'{print $3}\'` || true'
      
            }
        }
       
        stage('deploy') {
          
            withCredentials([usernamePassword(credentialsId: 'pro4_ssh'.passwordVariable: 'password'.usernameVariable: 'userName')]) {
           
                def remote = [:]
                remote.name = 'pro4'
                remote.user = userName
                remote.host = '47.101.137.187'
                remote.password = password
                remote.allowAnyHosts = true
                
                writeFile file: 'poll.sh'.text: """ docker ps | grep citest | awk '{print \$2}' >> /data/jenkins/mi_test_history echo /data/jenkins/mi_test_history docker ps | grep citest | awk '{print \$1}' | xargs docker kill || true docker images | grep citest | awk '{print \$1":"\$2}' | xargs docker rmi -f || true docker login --username=yourName --password=yourPassword registry.cn-shanghai.aliyuncs.com docker pull ${repository} docker run -d ${repository} """
    
                sshPut remote: remote, from: 'poll.sh'.into: '. '
                sshScript remote: remote, script: 'poll.sh'
                sshRemove remote: remote, path: 'poll.sh'}}}else {
        stage('rollback') {
            withCredentials([usernamePassword(credentialsId: 'pro4_ssh'.passwordVariable: 'password'.usernameVariable: 'userName')]) {
           
                def remote = [:]
                remote.name = 'pro4'
                remote.user = userName
                remote.host = '47.111.111.111'
                remote.password = password
                remote.allowAnyHosts = true
                
                writeFile file: 'roll.sh'.text: ''' popline(){ LC_CTYPE=C l=`tail -"${2:-1}" "$1"; echo t`; l=${l%t}; truncate -s "-${#l}" "$1"; printf %s "$l"; } last=`popline /data/jenkins/mi_test_history || ""` if [ -n "$last" ]; then docker ps | grep citest | awk '{print $1}' | xargs docker kill || true docker images | grep citest | awk '{print $1":"$2}' | xargs docker rmi -f || true docker login --username=yourName --password=yourPassword registry.cn-shanghai.aliyuncs.com docker pull $last docker run -d $last echo "rollback to $last success" else echo "nothing to rollback" fi '''
    
                sshPut remote: remote, from: 'roll.sh'.into: '. '
                sshScript remote: remote, script: 'roll.sh'
                sshRemove remote: remote, path: 'roll.sh'}}}}Copy the code

Use a build and a rollback to view the results on the target host

[root@izuf69bpbvay7p0ozdhq6bz jenkins]# cat mi_test_history Registry.cn-shanghai.aliyuncs.com/xxx/citest:v1.3.3_manual_37 [root @ izuf69bpbvay7p0ozdhq6bz Jenkins] # docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES fd5a375be004 13 seconds registry.cn-shanghai.aliyuncs.com/xxx/citest:v1.3.3_manual_40 ". / test "line Up 12 seconds beautiful_hoover 16 df7cae17a3 docker - elk_kibana "/ usr/local/bin/kiba..." 2 months ago Up 2 months docker-elk_kibana_1 AD8056879f70 docker-elk_logstash "/usr/local/bin/docker-elk_logstash" 2 months ago Up 2 months Docker-elk_logSTASH_1 236f6ED9a404 Docker-elk_elasticSearch "/usr/local/bin/docker-elk_elasticSearch" 2 months ago Up 2 months docker-elk_elasticsearch_1 [root@izuf69bpbvay7p0ozdhq6bz jenkins]# cat mi_test_history [root@izuf69bpbvay7p0ozdhq6bz jenkins]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 12f23fec4a5f Registry.cn-shanghai.aliyuncs.com/xxx/citest:v1.3.3_manual_37 ". / test "9 seconds line Up 8 seconds eager_napier 16 df7cae17a3 docker - elk_kibana "/ usr/local/bin/kiba..." 2 months ago Up 2 months docker-elk_kibana_1 AD8056879f70 docker-elk_logstash "/usr/local/bin/docker-elk_logstash" 2 months ago Up 2 months Docker-elk_logSTASH_1 236f6ED9a404 Docker-elk_elasticSearch "/usr/local/bin/docker-elk_elasticSearch" 2 months ago Up 2 months docker-elk_elasticsearch_1 [root@izuf69bpbvay7p0ozdhq6bz jenkins]#Copy the code

The build and rollback operations were successful