preface

Jenkins, as the CI/CD tool we commonly use today, has many features. I currently use Jenkins the most for deployment packaging, pre-release checking, automated testing, unit testing, Lint checking, etc. These scenarios are common in a complete “production line”.

For example, when developing the AAR library, we needed to publish the AAR to the Maven repository (perhaps SNAPSHOT at this point) every time a change was incorporated into the main branch of the AAR library, and this process was obviously automated. We could have used Jenkins, To hook the merge behavior, the main branch of the Git repository, to automatically package, publish, and notify specific developers.

The above is just a scenario in the actual development, but according to the organizational structure of our company, we may be divided into many teams, and each team is divided into many groups (there may be other groups under the group). At this time, there may be many Jenkins jobs. With the increase of jobs, There are certainly many reusable patterns to be found in the Jenkins Job configuration. This article will not talk too much about Jenkins and some concepts of pipeline. The purpose of this article is to explain how to reference Groovy files in pipeline to achieve the reuse of some common pipeline functions

Pipeline

Pipeline is one of the most commonly used models in Jenkins. As its name implies, pipeline allows us to define our jobs just like product pipeline. For example, we can regard a release process as an assembly line.

This diagram roughly represents what needs to be done from the development phase (COMMIT) to the final release phase. The entire process can be viewed as a pipeline, where each step is a stage. The corresponding Jenkins Job will be executed according to stages in the execution process, and the logs of each stage and whether the execution is successful can be clearly seen in combination with the Blue Ocean plug-in.

JenkinsFile

Pipelines come in two forms, both implemented using Groovy: Declarative Pipeline, Scripted Pipeline. Check out the Scripted Pipeline for any differences in how it’s written. This article will be Scripted for you as an example.

In addition, Pipeline can be configured in two ways: the first way is to create a Pieline Job directly in Jenkins, and then directly write Pipeline script ** in the configuration page, as shown in the following figure:

In addition to this approach, we can define the pipeline script in the project and then configure the Git repository to get the pipeline script.

The second method is recommended, because it is very flexible. After modifying the project code, JenkinsFile can be directly changed. Otherwise, you need to go to the Jenkins configuration page every time to modify it.

Import the Groovy

The above made a general introduction to Pipeline. As mentioned above, as the number of projects increases, there are many scenarios that need to be reused, such as the following pipeline:

node {
  // Run unit tests
  stage('Unit Test') {
      sh './run_unitTest'
  }

  // Export the report and delete the report folder
  stage('Test Result Reporter') {
      sh './export_reporter'
      archiveArtifacts './reporter'
      sh 'rm -rf reporter'
  }

  stage('... ') {}}Copy the code

Obviously, in practice, many of our projects need the stage of unit test and export report. At this time, we should be able to fulfill the amount of these two stages, otherwise, each Job pipeline needs to be written again. Reuse can be achieved in several ways:

  1. Import the common pipeline steps into the Global Pipeline LIbraries. If you are interested, go to the Jenkins Pipeline Extension Shared Library

  2. Since pipeline is written with Groovy scripts, we can extract the common steps to form a common code base. When we need to reference the stages in the public library, we just need to load the corresponding Groovy script. The advantage of this is that we can modify it at any time. It also eliminates the need to upload the Jenkins shared library, which is what we’ll talk about today.

With the second option described above, we can define a public script unit_test.groovy:

def runUnitTest() {
  sh './run_unitTest'
}

def exportReporter() {
  sh './export_reporter'
  archiveArtifacts './reporter'
  sh 'rm -rf reporter'
}
Copy the code

Our pipeline can be changed to:

node {
		
    def unitTestModule = load('unit_test.groovy')
    // Run unit tests
    stage('Unit Test') {
        unitTestModule.runUnitTest()
    }

    // Export the report and delete the report folder
    stage('Test Result Reporter') {
    	unitTestModule.exportReporter()
    }
    
    stage('... ') {}}Copy the code

And you can see it’s a lot cleaner.

Note that the load() parameter above requires a groovy path. Assuming that the common library and pipeline are in the same repository, we can import groovy files directly in this way. However, we might not want to embed a pipeline public library in the project. Ideally, the public library is a separate repository, and we can also load the corresponding Groovy, which requires some extra work:

  1. First need to install Jenkins plug-in: Pipeline Remote Loader plugin

  2. With this plug-in, you can implement Groovy files that are imported directly into other repositories as follows:

    node {
		
  def unitTestModule = fileLoader.fromGit('your_groovy_path'.'git_repository_url'.'branch'.'credentialsId'.'node_label')
   // Run unit tests
  stage('Unit Test') {
       unitTestModule.runUnitTest()
  }

   // Export the report and delete the report folder
  stage('Test Result Reporter') {
   	unitTestModule.exportReporter()
  }

  stage('... ') {}}Copy the code

    When the Pipeline Remote Loader plugin is installed, fileloader.fromgit () can be used to load groovy from the repository. So what happens when you need to import multiple Groovy files? You can use the following methods:

    fileLoader.withGit('git_repository_url'.'branch'.'credentialsId'.'node_label') {
   def unitTestModule = load('your_unit_test_groovy_path')
   def otherModule = load('your_other_groovy_path')... }Copy the code

    As you can see, if you want to import multiple scripts, you can use withGit.

FileLoader has five apis:

  • fromGIt(String libPath, String repository, String branch, String credentialsId, String labelExpression )

    Load a single Groovy file from the specified Git repository

  • withGit(String repository, String branch, String credentialsId, String labelExpression)

    Load a set of Groovy files from the specified Git repository

  • fromSVN(String libPath, String repository, String credentialsId, String labelExpression)

    Load a single Groovy file from the specified SVN repository

  • withSVN(String repository, String credentialsId, String labelExpression)

    Load a set of Groovy files from the specified SVN repository

  • load(String libPath)

    Load groovy files according to the path

Parameter Description:

  • LibPath: Relative path of groovy files
  • Repository: Git or SVN address
  • Branch: indicates a branch
  • CredentialsId: credentialsId that has been added to Jenkins to pull code to Git or SVN, usually in the form of user name + password or private key. Specific how to match can search, there are a lot of online.
  • LabelExpression: Specifies whether node is specified to run this operation.

conclusion

As can be seen, groovy files can be freely referenced by Pipeline Remote Loader plugin, which can help us achieve Pipeline script reuse. You can choose which reuse method to use according to your own project.