Application deployment is a process that development, testing, and launching must face, especially with the emergence of microservice architecture, operation and maintenance deployment is gradually separated from single deployment, and becomes more complex.

However, put aside the multi-language, multi-environment, clustered, distributed deployment. Just talk about incremental and full deployment

1. Incremental and full deployment

Deployment, with the exception of initial deployment of the project, is ideally an update of what the new release changes

1.1 Incremental Deployment

1.1.1 Introduction to Incremental Deployment

Incremental deployment generally refers to extracting the incremental parts (including code, executable files, or configuration) between the current version and the version to be deployed during each deployment, and only updating the incremental parts during the deployment.

1.1.2 Common Deployment process

  1. Use code management tools (SVN, GIT, and so on) to extract the increments between the two versions and incorporate incremental changes in other aspects.
  2. Follow the incremental deployment, write deployment scripts and prepare incremental deployment packages (including obfuscated code, etc.).
  3. Distribute and deploy incremental deployment packages to target environments that already run the previous version to upgrade the system.

1.1.3 Advantages of Incremental Deployment

  1. Fast deployment speed. Only the incremental part is updated each time to shorten the deployment time
  2. Reduce the variation. Reduce the amount of change to the entire system, and some configurations do not need to be updated every time
  3. Improve security. Because each detachment increment is updated, the entire code is not leaked

1.1.4 Disadvantages of Incremental Deployment

  1. Incremental deployment If other external deployment environments are required, the deployment efficiency is reduced

    Incremental deployment is not

  2. Multiple deployment environments require high repeatability

  3. Incremental deployments become unfriendly to rollback operations

1.2 How to Choose Incremental or Full Volume

Most of the existing automated deployments are full deployment, but full deployment also has some disadvantages. But filtering can be done with a few strategies:

  • You can deploy all configurations and materials (such as deployment packages and external configuration files) in advance to improve efficiency and speed
  • Use grayscale publishing or load balancing to reduce the impact of full deployment on application availability

For the vast majority of state-independent deployment units (applications, modules, microservices, etc.) in modern systems, full deployment should generally be the best choice. State-related deployment units (databases, etc.) still fit the incremental deployment logic.

2. Get to the point

The previous section covered some scenarios for both incremental and full deployment. How to do incremental deployment with a shell script and Git Log

2.1 Prerequisites

  • Java project

  • Maven for management

  • Git serves as a code repository

2.2 shell script

Shell newbie, not perfect enough to write, light spray.

2.2.1 Module of the entire shell script

  • Preparing the Git Environment
  • Maven compiles the project you want to build
  • Create an incremental deployment folder
  • Retrieve the project Target directory
  • Git diff retrieves the difference between two commits, and then copies the corresponding files to an “incremental folder”

2.2.2 Git Environment Preparations

# git environment

if[[!-d ".git"]].then
    ECHO error: please init  Git Repository
    exit 1;
fi

if [[ ! -z ${branch}]].then
    git checkout ${branch}
fi

Get the default commit-hash
if [[ -z "$begin_hash" ]] && [[ -z "$end_hash"]].then
    for p in $(git log --pretty=oneline -2) ; do
        if [[ ${#p} -eq40]];then
            if [[ -z ${begin_hash}]].then
                begin_hash=${p}
            else
                end_hash=${p}
                break
            fi
        fi
    done
fi

is_begin_has=false

# Whether the current commit is the latest
if [[ $(git log --pretty=oneline -1) == *${begin_hash}*]];then
    is_begin_has=true
fi

If compile is not supported in maven's original configuration, the build will fail.
if [[ ${is_begin_has} = false]].then
    project_path=$(pwd)
    project_name=${project_path##*/}
    cd. build_project_name=${project_name}_build_temp_project
    if[[!-d ${build_project_name}]].then
        mkdir ${build_project_name}
    fi
    \cp -rf  ${project_name}/.  ${build_project_name}
    cd ${build_project_name}
    git reset --hard ${begin_hash}
fi
Copy the code
2.2.2.1 Verify whether git repository code is available
if[[!-d ".git"]].then
    ECHO error: please init  Git Repository
    exit 1;
fi
Copy the code
2.2.2.2 Check whether branch switchover is required
if [[ ! -z ${branch}]].then
    git checkout ${branch}
fi
Copy the code
2.2.2.3 Do YOU need to set the default build COMMIT value

If you do not assign –begin_hash= and –end_hash= to a build, then the last two COMMITS are used by default for incremental deployment.

Git log –pretty=oneline -2 to get the hash of the last two commits

Get the default commit-hash
if [[ -z "$begin_hash" ]] && [[ -z "$end_hash"]].then
    for p in $(git log --pretty=oneline -2) ; do
        if [[ ${#p} -eq40]];then
            if [[ -z ${begin_hash}]].then
                begin_hash=${p}
            else
                end_hash=${p}
                break
            fi
        fi
    done
fi
Copy the code
2.2.2.4 Verify whether the begin_hash value passed is the latest commit hash of the current branch

If the current branch is not the latest COMMIT hash, roll back to the corresponding COMMIT to build the project

if [[ $(git log --pretty=oneline -1) == *${begin_hash}*]];then
    is_begin_has=true
fi
Copy the code
2.2.2.5 If begin_hash is not the latest commit hash

If the value of begin_hash is not the latest commit hash. You need to roll back to the corresponding COMMIT for build compilation.

  1. Copy existing projects to the new directory environment
  2. Reset the project to the new directory environment for building the project
if [[ ${is_begin_has} = false]].then
    project_path=$(pwd)
    project_name=${project_path##*/}
    cd. build_project_name=${project_name}_build_temp_project
    if[[!-d ${build_project_name}]].then
        mkdir ${build_project_name}
    fi
    \cp -rf  ${project_name}/.  ${build_project_name}
    cd ${build_project_name}
    git reset --hard ${begin_hash}
fi
Copy the code

2.2.3 Maven compiles the project to be built

Compile the project and generate the corresponding class file and related configuration file

mvn clean compile -q -DskipTest
Copy the code

If the local repository is not configured in Maven, you can reconfigure it by scope and systemPath, for example:

<dependency>
    <groupId>cn.catalpaflat</groupId>
    <artifactId>core</artifactId>
    <version>1.0.0</version>
    <scope>system</scope>
    <systemPath>${project. The basedir} / lib/core - 1.0. The jar</systemPath>
</dependency>
Copy the code

2.2.4 Creating an Incremental Deployment Folder

To prevent incremental folders from being deleted or committed to a Git repository, you can unify them into a single directory and ignore them with.gitignore. You can compare the differences between each incremental deployment

build_path=build-path/
current_date=`date +%Y%m%d%H%m%s`

if[[!-d "$build_path$current_date"]].then
    mkdir -p ${build_path}${current_date}
else
    rm -rf ${build_path}${current_date}
    mkdir -p ${build_path}${current_date}
fi
Copy the code

2.2.5 Retrieve the project Target directory

If the project is a Maven project and a Java project, because there are multiple Maven modules, you need to retrieve the compiled code path under each module for copying files such as class.

default_target_paths=()
default_java_file=java

module_index=0
# Retrieve the current project for maven multi-module development, recursively retrieve it, and set its compiled code location (only Java type is provided).
obtain_module() {for module in ` cat ./pom.xml | grep '<module>' | awk -F '>' '{print $2}' | awk -F '<' '{print $1}' `
    do
        cd ${module}

        if[[!-d "/pom.xml"]].then
           module_exist=`cat ./pom.xml | grep '<module>' | awk -F '>' '{print $2}' | awk -F '<' '{print $1}'`
           if [[ -z ${module_exist}]].then
                if[[!-d "/target"]].then
                    if [[ -z The $1]].then
                        default_target_paths[module_index]=${module}/target/classes
                    else
                        default_target_paths[module_index]=The $1/${module}/target/classes
                    fi
                    ((module_index++))
                fi
           else
                 if [[ -z The $1]].then
                      obtain_module ${module}
                 else
                      obtain_module The $1/${module}
                 fi
           fi
        fi
        cd.done
}

obtain_module
Copy the code

2.2.6 Retrieve and copy the change files to the increment folder

  1. Git diff –name-only Check for file differences between two commits
  2. Copy the commit compiled code of begin_hash to the incremental folder for later packaging and deployment
# by git diff - name - only file differences between two commit, and after begin_hash code to compile, the differences of file copy to the folder "delta", in preparation for the subsequent incremental deployment

for file_path in $(git diff --name-only ${begin_hash} ${end_hash});do
    package_path=${file_path%/*}
    file_name=${file_path##*/}
    file_type=${file_name##*.}
	Check whether the verification folder where the file resides is created
    if [[ ${package_path}! = *. *]];then
          if[[!-d ". /${build_path}${current_date}/$package_path"]].then
                mkdir -p ./${build_path}${current_date}/${package_path}
           fi
    fi
	# if the Java
    if [[ ${file_type} = ${default_java_file}]].then
        module_path=${package_path##*java}
        file_class_name=${file_name%.*}
        module_type=${package_path%%/*}
		Check which Maven module path you are in
        for default_target_path in ${default_target_paths[@]}; do
            target_module_path=$(echo ${default_target_path} | awk -F '/target/' '{print $1}')
            file_target_module_path=$(echo ${package_path} | awk -F '/src/' '{print $1}')
            file_target_package_path=$(echo ${package_path} | awk -F '/src/main/java/' '{print $2}')
            default_module_type=${default_target_path%%/*}
            if [[ ${target_module_path} = ${file_target_module_path}]].then
            	Check the target directory of the maven module and run the cp operation
                cp -afx ${default_target_path}/${file_target_package_path}/${file_class_name}/ *.${build_path}${current_date}/${package_path}
            fi
        done

    else
    # Non-java files, directly copy files to the corresponding directory
        if [[ ${package_path}! = *. *]];then
            if[[!-d ". /${build_path}${current_date}/$package_path"]].then
                mkdir -p ./${build_path}${current_date}/${package_path}
            fi
        else
             package_path=${package_path%/*}
        fi

        cp -afx ${file_path}. /${build_path}${current_date}/${package_path}

    fi
done
Copy the code

Source through train!!

So far, the rough version of version 1.0 is preliminarily complete and visually ready to use, hahaha