preface

Who says a programmer can’t pressure test! JMH based benchmark test, to show you how I use code pressure test code (doll warning ⚠) for performance testing do not know friends can first see: program ape also need to know performance testing knowledge summary

What is a benchmark

Benchmark testing refers to the quantitative and comparable testing of a certain performance index of a class of test objects through the design of scientific testing methods, testing tools and testing systems.

Why use benchmarks?

Benchmarks have the following characteristics:

(1) Repeatability: repeatability test can be carried out, which is conducive to compare each test result, get the long-term trend of performance results, and make reference for system tuning and capacity planning before going online.

(2) Observability: through comprehensive monitoring (including the beginning and end of the test, execution machine, server, database), timely understanding and analysis of what happened in the test process.

(3) Displayability: relevant personnel can intuitively understand the test results (web interface, dashboard, line chart tree chart and other forms).

(4) Authenticity: the test results reflect the real situation experienced by customers (real and accurate business scenarios + consistent configuration with production + reasonable and correct testing methods).

(5) Executability: relevant personnel can quickly test, verify, modify and tune (positioning and analysis). So it’s very tedious and very difficult to do a benchmark test that matches a trait. External factors can easily affect the final test results. Especially for Benchmarks in JAVA.

When do YOU need to benchmark

  • You want to know exactly how long a method takes to execute and the correlation between the execution time and the input.
  • Compare throughput of different implementations of interfaces/methods under given conditions;
  • See what percentage of requests are completed in what time;

How do I test Java code performance?

Test code preparation

Before we do performance testing, let’s prepare the test code.

When java8 came out, everyone was encouraging the use of lambda expressions, saying how elegant lambda expressions were and how efficient lambda expressions were. Don’t fall behind! With the quickly!

There must have been some programmer who said,

The next step is to verify that lambda expressions perform better than normal loops by looping through the List to find the maximum value.

Test code:

package com.demo.jmh;

import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import java.util.Optional;

/ * * *@author lps
 * @title: com.demo.jmh.LoopHelper
 * @projectName testdemo
 * @descriptionThe loop utility class implements multiple java8-based ways of looping through a List * to find the maximum value * by iterating through all the values of the List in 5 different ways *@date2020/7/12 15:43 * /
public class LoopHelper {
    /** * use the iterator to loop **@param list
     * @return* /
    public static int iteratorMaxInteger(List<Integer> list) {
        int max = Integer.MIN_VALUE;
        for (Iterator it = list.iterator(); it.hasNext(); ) {
            max = Integer.max(max, (Integer) it.next());
        }
        return max;
    }

    /** * use foreach loop **@param list
     * @return* /
    public static int forEachLoopMaxInteger(List<Integer> list) {
        int max = Integer.MIN_VALUE;
        for (Integer n : list) {
            max = Integer.max(max, n);
        }
        return max;
    }

    /** * use the for loop **@return* /
    public static int forMaxInteger(List<Integer> list) {
        int max = Integer.MIN_VALUE;
        for (int i = 0; i < list.size(); i++) {
            max = Integer.max(max, list.get(i));
        }
        return max;
    }

    /** * Use Java8 parallel streams **@return* /
    public static int parallelStreamMaxInteger(List<Integer> list) {
        Optional max = list.parallelStream().reduce(Integer::max);
        return (int) max.get();
    }

    /** * uses lambda expressions and streams **@return* /
    public static int lambdaMaxInteger(List<Integer> list) {
        returnlist.stream().reduce(Integer.MIN_VALUE, (a, b) -> Integer.max(a, b)); }}Copy the code

Common means of code testing performance

In general, to evaluate the performance of the comparison code, we can use the time indicator to evaluate which method is better to implement with less time.

So we usually write like this:

    public static void main(String[] args) {
        List<Integer>list = new ArrayList<Integer>();
        for (int i = 0; i < 100; i++) {
            list.add(i);
        }
        long start=System.currentTimeMillis();
        System.out.println(LoopHelper.iteratorMaxInteger(list));
        long end=System.currentTimeMillis();
        System.out.println("Current program 1 time:"+(end-start)+"ms");
    }
Copy the code

There is nothing wrong with such a look, but the intermediate error if in the case of large amount of data, the test results are not standard.

Unusual benchmarks based on JMH

What is JMH?

JMH, or Java Microbenchmark Harness, is a suite of tools specifically for code Microbenchmark testing. What is Micro Benchmark? In simple terms, this is benchmark testing at the method level, with an accuracy of microseconds. When you locate a hot method and want to further optimize its performance, you can use JMH to quantify the results of the optimization. What distinguishes the JMH from its competitors, if any, is that it was developed by those within Oracle who implemented THE JIT, knowing full well the impact that JIT and what the JVM calls profile Guided Optimization can have on benchmark accuracy (SMILE).

Test using JMH

Engineering Environment Description

Dependencies/software version
java 1.8
maven-compiler-plugin 3.8.1
jmh 1.23

Depend on the load

Add the following dependencies in pom.xml

<dependencies>
        <! -- https://mvnrepository.com/artifact/org.openjdk.jmh/jmh-core -->
        <dependency>
            <groupId>org.openjdk.jmh</groupId>
            <artifactId>jmh-core</artifactId>
            <version>1.23</version>
        </dependency>

        <dependency>
            <groupId>org.openjdk.jmh</groupId>
            <artifactId>jmh-generator-annprocess</artifactId>
            <version>1.23</version>
            <scope>provided</scope>
        </dependency>

        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-math3</artifactId>
            <version>3.6.1 track</version>
        </dependency>
   </dependencies>
Copy the code

Writing test classes

package com.demo.jmh;


import com.demo.jmh.LoopHelper;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;

import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.TimeUnit;


/ * * *@author lps
 * @title: LoopHelperTest
 * @projectName testdemo
 * @description: Test class *@date2020/7/12 tightly with * /
@BenchmarkMode(Mode.Throughput) / / throughput
@OutputTimeUnit(TimeUnit.MILLISECONDS) // The time unit used for the result
@State(Scope.Thread) // Assign one instance per test thread
@Fork(2) // The number of forks performed
@Warmup(iterations = 4) // Preheat 4 rounds
@Measurement(iterations = 10) // Run 10 rounds of tests
public class LoopHelperTest {
    // Define two list sizes to test for different list sizes
    @Param({"1000"."100000",})private int n;

    private List<Integer> list;

    /** * initialization method, before all benchmarks run */
    @Setup(Level.Trial)
    public void init(a) {
        list = new ArrayList<Integer>();
        for (int i = 0; i < n; i++) { list.add(i); }}@Benchmark
    public void testIteratorMaxInteger(a) {
        LoopHelper.iteratorMaxInteger(list);
    }

    @Benchmark
    public void testForEachLoopMaxInteger(a) {
        LoopHelper.forEachLoopMaxInteger(list);
    }

    @Benchmark
    public void testForMaxInteger(a) {
        LoopHelper.forMaxInteger(list);
    }

    @Benchmark
    public void testParallelStreamMaxInteger(a) {
        LoopHelper.parallelStreamMaxInteger(list);
    }

    @Benchmark
    public void testLambdaMaxInteger(a) {
        LoopHelper.lambdaMaxInteger(list);
    }

    /** * completes the method after all benchmarks have been run
    @TearDown(Level.Trial)
    public void arrayRemove(a) {
        for (int i = 0; i < n; i++) {
            list.remove(0); }}public static void main(String[] args) throws RunnerException {
        Options options = new OptionsBuilder().include(LoopHelperTest.class.getSimpleName()).build();
        newRunner(options).run(); }}Copy the code

The test results

After running, the console outputs the following test results:

Benchmark: Method by which a Benchmark is executed (n) : indicates parameter n Mode Test Mode, in this case throughput CNT: number of runs Score: the final Score Units: indicates the operation times per second
LoopHelperTest.testForEachLoopMaxInteger 1000 thrpt 10 1572.310 + / – ops/ms
LoopHelperTest.testForEachLoopMaxInteger 100000 thrpt 10 15.391 + / – ops/ms
LoopHelperTest.testForMaxInteger 1000 thrpt 10 1555.872 + / – ops/ms
LoopHelperTest.testForMaxInteger 100000 thrpt 10 15.926 + / – ops/ms
LoopHelperTest.testIteratorMaxInteger 1000 thrpt 10 1592.788 + / – ops/ms
LoopHelperTest.testIteratorMaxInteger 100000 thrpt 10 15.151 + / – ops/ms
LoopHelperTest.testLambdaMaxInteger 1000 thrpt 10 295.189 + / – ops/ms
LoopHelperTest.testLambdaMaxInteger 100000 thrpt 10 2.057 + / – ops/ms
LoopHelperTest.testParallelStreamMaxInteger 1000 thrpt 10 55.194 + / – ops/ms
LoopHelperTest.testParallelStreamMaxInteger 100000 thrpt 10 3.818 + / – ops/ms

To facilitate comparison, the data is presented as a discounted graph

1. This is the result when the List is 1000: Iterator > for > foreach > Lambda > ParallelStream

Iterator, for, and foreach perform fairly well, and ParallelStream performs poorly

2. This is the result when the List is 100000: for > foreach > Iterator >ParallelStream >Lambda

Among them, Iterator, for, foreach performance is not much different, Lambda performance is poor

Based on the benchmark results, we can draw the following conclusions:

Lamdba and ParallelStream loops perform worse than Iterator, for, and foreach for small or large volumes of data, but not worse, at least 3-4 times the throughput difference. Iterator, for, and foreach are recommended.

JMH related explanatory notes

@BenchmarkMode

The benchmark mode has four values

  • Throughput(” THRPT “, “Throughput, OPS /time”), Throughput, number of times executed per unit of time
  • AverageTime(” avGT “, “AverageTime”, time/op “), the Average time for each operation
  • SampleTime(” sample “, “Sampling time”), which gives the worst-case time for satisfying the number of percent
  • SingleShotTime(” SS “, “SingleShot Invocation Time”), SingleShotTime is run only once. Warmup is often set to 0 at the same time to test performance on a cold start.
  • All(” All “, “All Benchmark Modes”), All of the above tests, most commonly used

@Warmup

Preheating, due to the JIT, the data after preheating is more stable.

Why warm up? Because of the JVM’s JIT mechanism, if a function is called more than once, the JVM tries to compile it into machine code to speed up execution. To make Benchmark’s results more realistic, you need to warm up.

@Measurement

Test some basic parameters

  • Iterations: specifies the number of tests
  • Time: indicates the time of each test
  • TimeUnit: TimeUnit
  • The number of methods called per op

@Threads

The number of threads tested can be annotated on a class or method

@Fork

Several new Java virtual machines will be forked to reduce the impact, and a number of parameters will need to be set. If the number of forks is 2, the JMH will fork two processes to test.

@outputTimeUnit

The time type of the benchmark

@Benchmark

Method-level annotations for each method to be tested.

@Param

Attribute-level annotations, which can be used to specify multiple cases of a parameter, are particularly useful for testing the performance of a function with different parameter inputs. The @param annotation takes an array of strings that are converted to the corresponding data type before the @setup method executes. For example, if there are two @param annotated fields, the first field has 5 values and the second field has 2 values, then each test method will run 5*2=10 times.

@Setup

Method-level annotations that do some preparatory work before testing, such as initialization parameters

@TearDown

Method-level annotations, which do some finishing work after the test

The name of the describe
Level.Trial The default level. All benchmarks run before/after (a set of iterations)
Level.Iteration Before/after an iteration (set of calls)
Level.Invocation Before/after each method call (not recommended unless you know the purpose)

@State

Set the state of a class to be shared between threads during testing:

  • Thread: The Thread is private
  • Group: Shared by all threads in the same Group
  • Benchmark: Shared by all threads

conclusion

Through the above cases, we based on JMH benchmarking framework can obtain more accurate and complete performance test results, according to the results to evaluate the performance of the program interface/method to help us solve the problem of performance bottlenecks, optimize the system performance, reduce the outage probability, rounded is to improve the work efficiency, reduce the chance to work overtime, another is a promotion and pay increase rounded, Go to the top of your profession.

I wish you all the best in 2020

The resources

www.lagou.com/lgeduarticl…

Blog.csdn.net/yh_zeng2/ar…

www.cnblogs.com/fightfordre…