First posted on Jenkins Chinese community

Test code coverage using JaCoCo statistics functionality?

I know something about JaCoCo but not very well. “Knowing” refers to the fact that JaCoCo has been used in CI practice for unit test code coverage statistics: When code is pushed to the code repository, JaCoCo is used for unit test code coverage statistics and pushes the data to SonarQube. “Not very familiar” means that the application scenario is limited to that, without much research and practice.

Some time ago, a testing colleague asked to use JaCoCo statistics functionality to test code coverage during actual testing. Its main purpose is to enhance confidence in software quality by looking at indicators related to code coverage statistics after functional testing. A search of information has proved that this is feasible.

Since I do not know much about JaCoCo, I refer to the official website for further information.

Learn more about JaCoCo

JaCoCo, Java Code Coverage, is an open source Statistical tool for Java Code Coverage. It was created by the EclEmma team based on lessons learned over the years using and integrating existing libraries.

JaCoCo vision

JaCoCo should provide standard techniques for code coverage analysis in Java VM-based environments. The focus is on providing a lightweight, flexible, well-documented library for integration with a variety of build and development tools.

JaCoCo product features

  • Coverage analysis of instructions (C0), branches (C1), lines, methods, types, and cyclomatic complexity.
  • Based on Java bytecode, it can also work without source files.
  • Simple integration through real-time detection based on Java Agent. Other integration scenarios, such as custom class loaders, can also be implemented through the API.
  • Framework independence: Smooth integration with Java VM-based applications, such as normal Java programs, OSGi frameworks, Web containers, or EJB servers.
  • Compatible with all published versions of Java class files.
  • Support for different JVM languages.
  • Supports several report formats (HTML, XML, CSV).
  • Remote protocols and JMX controls to perform data dump from coverage Agent requests at any point in time.
  • Ant tasks that collect and manage execution data and create structured coverage reports.
  • Maven plugin for collecting coverage information and creating reports in Maven builds.

Nonfunctional feature

  • Simple to use and integrated with existing build scripts and tools.
  • Good performance and minimal runtime overhead, especially for large projects.
  • Lightweight implementation with minimal dependence on external libraries and system resources.
  • Comprehensive documentation.
  • A fully documented API (JavaDoc) and examples for integration with other tools.
  • Regression testing is based on JUnit test cases with complete functional test coverage.

JaCoCo provides Java API, Java Agent, CLI, Ant, Maven, and Eclipse integration. Third parties provide integration with other tools such as Gradle, IDEA, Jenkins, etc.

Leave the theory behind and start doing it

JaCoCo supports statistics on code coverage not only for local services but also for remote services. Unit test coverage statistics are the statistics of code coverage of local services, code and running services on the same machine, the author here by using the JaCoCo Maven plug-in. The function test code coverage statistics is the statistics of remote service code coverage, code and running services are generally not on the same machine, this needs to use JaCoCo Java Agent to achieve.

Note: Actually, the JaCoCo Maven plugin also uses the JaCoCo Java Agent, but the user does not need to directly relate to the Java Agent and its options. The Maven plugin handles this transparently.

1. Download the JaCoCo distribution

Distributions can be downloaded from the JaCoCo website or from a Maven repository (central or private). The distribution package’s lib directory contains the following libraries:

Java application startup script adds JVM parameters related to JacocoAgent

Jacocoagent.jar needs to be pushed to the server where the application is deployed, and I have done this using Ansible. Java application startup scripts need to add JVM parameters like the following:

JAVA_OPTS="$JAVA_OPTS -javaagent:/path/jacocoagent.jar=includes=*,output=tcpserver,append=false,address=$IP,port=$JACOCO_PORT"
Copy the code

This exposes a TCP service to which the client can connect and retrieve execution data files after the application is successfully started.

The related attributes are described as follows:

  • Append: Append =false indicates that a new execution data file is generated each time dump. If append=true, data is appended to an existing execution data file during dump. Output = tcpServer indicates that the agent listens on the connection from the TCP port specified by the adrress and port attributes, and the execution data is written to this connection.
  • Output: If output= tcpClient, it indicates that the agent is connected to the TCP port specified by the adrress and port attributes at startup, and the execution data is written to this connection. If output=file, the execution data is written to the file specified by the DestFile attribute when the JVM terminates. Output The default value is file.
  • Address: Indicates the IP address or host name bound to the tcpServer when output is the tcpClient or the IP address or host name connected to the tcpClient when output is the tcpClient. In tcpServer mode, a value of “*” causes the proxy to accept connections only on the local address. The default value of address is 127.0.0.1.
  • Port: Bound to the port when output is tcpServer, or connected to the port when output is tcpClient. In tcpServer mode, ports must be available, which means that if multiple JaCoCo Agents are running on the same machine, different ports must be specified. The default value of port is 6300.

3. Create and configure Jenkins Pipeline task

The Jenkins task generally consists of several steps: pull code, build, dump application execution data (Jacoco. Exec), parse jacoco. Exec files generated by Jacoco, and then generate coverage reports (HTML format). Pull code here need not say more, configuration from the code repository (SVN/Git) and branch address can be done, relatively simple. MVN clean package-dmaven.test.skip =true There are several ways to dump application execution data: Ant, CLI, and Maven. Since Java applications are built using Maven, the Maven Jacoco Plugin is selected. The Jenkins JaCoCo Plugin is used here to parse the JaCoCo. Exec file generated by JaCoCo and generate the coverage report (HTML format).

Jenkins Pipeline case is as follows:

pipeline {
    agent any
    tools {
        jdk 'JDK1.8'
    }
    stages {
        stage('Checkout'){
            steps{
                git branch: '${GIT_BRANCH}', credentialsId: 'xxx-xxx-xx-xx-xxx', url: '${GIT_URL}'
            }
        }
        stage('Build') {
            steps{
                withMaven(maven: 'maven'){
                      sh "mvn clean package -Dmaven.test.skip=true"
                }
            }
        }
        stage('DumpFromServer'){
            steps {
                withMaven(maven: 'maven'){
                      sh 'MVN org. Jacoco: jacoco -- maven plugin: 0.8.4: dump - Djacoco. Address = ${SERVER_IP} - Djacoco. Port = ${JACOCO_PORT}'
                }
            }
        }
        stage('JacocoPublisher') {
            steps {
                 jacoco()
            }
        }
    }
}
Copy the code

JaCoCo coverage report, partial screenshots are as follows:

conclusion

The way I’ve implemented it doesn’t cover any scenarios, but it’s pretty much the same. See the documentation on the official website for details on how to use the tools, because it’s the most comprehensive. The author hopes that this practice can give some reference to peers with similar demands, of course, the author also hopes to communicate with everyone. At the same time, the author’s JaCoCo practice is not over, and there may be some problems to be solved in the process of use. In the future, we will consider using Jenkins API to automatically generate a corresponding Jenkins task for Java application instances that need statistical function test code coverage. After the Java application instance is destroyed, the Jenkins task is cleaned and other functions are performed.

reference

  • www.eclemma.org/jacoco/inde…
  • www.jacoco.org/jacoco/trun…
  • www.jacoco.org/jacoco/trun…
  • www.jacoco.org/jacoco/trun…
  • www.eclemma.org/jacoco/trun…

Author: Wang Donghui