It has been a long time since the last functional plan of the FunTester test framework was updated. The main factor is that the functionality currently supported by the FunTester test framework meets the requirements of the work. Whether it is distributed performance testing framework, full link performance testing support, and quantitative simulation of online traffic, the basic technical verification has been completed, the rest of the technical solution is adjusted to better meet the needs of the current work, there are no technical obstacles.

While the FunTester testing framework is still being updated, it hasn’t been functional for a long time. Dynamic pressure is currently the most likely to be used in the workplace in the test scenarios envisaged for possible use.

The ultimate goal

The ultimate goal is to flexibly increase, decrease or maintain the test pressure during the continuous pressure test, so that multiple test scenarios can be covered in the process of executing the use case at a time, and the increase of test time caused by starting pressure from 0 can be avoided.

In previous performance test cases, after clicking “Start”, except for the active end, pressure adjustment was rarely carried out in the middle. Dynamic pressure is to intervene pressure in the process of pressure measurement and adjust the pressure generating end during the test. This requirement can be implemented through technologies such as containerization and distribution. My idea is to directly tune tasks running in a single JVM thread pool, which is suitable for non-cluster loading and scenarios with flexible stress requirements, but can also improve performance if applied to daily performance checks.

Main functions:

During the execution of the performance test, the performance test pressure value is dynamically increased or decreased.
Dynamic injection of new flow model tasks during performance test execution.

Such benefits:

Dynamically increase or decrease the execution of tasks, to achieve a trial execution test a variety of pressure purposes.
Dynamically adding and subtracting tasks of different models to dynamically modify the pressure measurement flow.

Long march first step: increase the pause function realization.

Source of demand

In the work, we often encounter a kind of scenario: it is expected that the QPS provided by the system is 10,000, and the number of threads is 100. We design 0 threads as the starting point, and increase the pressure by 10 at the interval of 30 seconds, and the maximum value is 120. (You can also use QPS increment mode; the following Demo uses thread increment).

The reality is that 80 threads have reached a performance bottleneck, and the need is to stay under the pressure of 80 threads for a while to facilitate data collection and field capture.

Train of thought

Basic implementation

Because the use scenario is to terminate the increase and maintain the pressure in the process of increasing pressure, the soft start scheme in the soft start preliminary study of performance test is used. Use the for loop to increase the pressure. Terminating increments are then triggered at some point.

The trigger condition

Expected to be in practice of service performance test and local test script in use, only to realize the function of the local test script idea is through a thread safe variable to mark execution status, whether continuous pressure or need to terminate, chose the other one thread here to complete this task, through from the console input to control the switch. The Demo I am only doing now is a plan with no turning back. Only the termination pressure will continue to increase or end without suspension.

Multithreaded tasks are as follows:

    private static class FunTester implements Runnable {

        @Override
        public void run(a) {
            waitForKey(INTPUT_KEY);
            HOLD.set(1);
            output("Pressure pause"); }}Copy the code

Here from the console waiting for input, if is equal to the com funtester. Config. Constant# INTPUT_KEY. Then set the com funtester. Frame. The execute. HoldConcurrent# HOLD is set to 1, Thus marking that the test mission has entered the termination booster phase.

Multithreaded synchronization

Here refer to Java thread synchronization of the three musketeers, I use the Java. Util. Concurrent. The Phaser, reason is a Java. Util. Concurrent. CountDownLatch is not flexible, unable to meet demand, And Java. Util. Concurrent. Although CyclicBarrier flexible also can satisfy the need of multithreaded synchronization, and flexible but not need to be synchronized increase or decrease the number of threads, they gave up.

Lead to problems

This dynamic pressure model is carried out in the pressurized phase, that is, threads are not fixed, which leads to some problems that have not been solved yet.

Local data statistics distortion.
It does not support multitasking and cannot be servified.

To realize the Demo

Multithreaded task class

First is com. Funtester. Base. Constaint. HoldThread class, realize the com. Funtester. Base. The constaint. ThreadBase section method, Concrete with other implementation differences mainly in com. Funtester. Base. Constaint. HoldThread# before and com funtester. Base. Constaint. HoldThread# after methods for marking multithreaded synchronization objects in c Om. Funtester. Frame. The execute. HoldConcurrent# several details of the phaser operation.

The code is as follows:

package com.funtester.base.constaint;

import com.funtester.frame.execute.HoldConcurrent;
import com.funtester.httpclient.GCThread;
import com.funtester.utils.Time;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

/** * dynamic pressure model, pressurization pause */
public abstract class HoldThread<F> extends ThreadBase<F> {

    private static final long serialVersionUID = -4617192188292407063L;

    private static final Logger logger = LogManager.getLogger(HoldThread.class);

    public HoldThread(F f, int limit, boolean isTimesMode) {
        this.isTimesMode = isTimesMode;
        this.limit = limit;
        this.f = f;
    }

    protected HoldThread(a) {
        super(a); }/ * * * * /
    @Override
    public void run(a) {
        try {
            before();
            long ss = Time.getTimeStamp();
            while (true) {
                try {
                    executeNum++;
                    long s = Time.getTimeStamp();
                    doing();
                    long et = Time.getTimeStamp();
                    short diff = (short) (et - s);
                    costs.add(diff);
                } catch (Exception e) {
                    logger.warn("Mission failed!", e);
                    errorNum++;
                } finally {
                    if ((isTimesMode ? executeNum >= limit : (Time.getTimeStamp() - ss) >= limit) || ThreadBase.needAbort() || status())
                        break; }}long ee = Time.getTimeStamp();
            if ((ee - ss) / 1000 > RUNUP_TIME + 3)
                logger.info("Thread :{}, number of executions :{}, number of errors :{}, Total time :{} s", threadName, executeNum, errorNum, (ee - ss) / 1000.0);
            HoldConcurrent.allTimes.addAll(costs);
            HoldConcurrent.requestMark.addAll(marks);
        } catch (Exception e) {
            logger.warn("Mission failed!", e);
        } finally{ after(); }}@Override
    public void before(a) {
        super.before();
        HoldConcurrent.phaser.register();
    }

    @Override
    protected void after(a) {
        super.after(); GCThread.stop(); HoldConcurrent.phaser.arriveAndDeregister(); }}Copy the code

Perform class

Perform class com. Funtester. Frame. The execute. HoldConcurrent difference with other models is that: 1. No fixed stable pressure, the end of pressurization is stable pressure; 2. Multithreaded tasks accept the output content of the console to control the task stage; 3. When the user has not input com. Funtester. Config. Constant# INTPUT_KEY, there is not natural to stop.

The code is as follows:

package com.funtester.frame.execute;

import com.funtester.base.bean.PerformanceResultBean;
import com.funtester.base.constaint.ThreadBase;
import com.funtester.config.Constant;
import com.funtester.frame.Save;
import com.funtester.frame.SourceCode;
import com.funtester.utils.RWUtil;
import com.funtester.utils.Time;
import org.apache.commons.lang3.StringUtils;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

import java.util.ArrayList;
import java.util.List;
import java.util.Vector;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Phaser;
import java.util.concurrent.atomic.AtomicInteger;

import static java.util.stream.Collectors.toList;

/** * concurrency class, used to start stress scripts */
public class HoldConcurrent extends SourceCode {

    private static Logger logger = LogManager.getLogger(HoldConcurrent.class);

    /** * is used to mark the status */
    public static AtomicInteger HOLD = new AtomicInteger(0);

    /** * start time */
    private long startTime;

    /** * End time */
    private long endTime;

    /** * Task description */
    public String desc;

    /** * Task set */
    public List<ThreadBase> threads = new ArrayList<>();

    /** * Number of threads */
    public int threadNum;

    /** * Total number of execution failures */
    private int errorTotal;

    /** * Total number of failed tasks */
    private int failTotal;

    /** * Total number of executions */
    private int executeTotal;

    /** * is used to record all request times */
    public static Vector<Short> allTimes = new Vector<>();

    /** * Record all markRequest information */
    public static Vector<String> requestMark = new Vector<>();

    /** * thread pool */
    ExecutorService executorService;


    /** * Multithreaded multistage synchronization class, for multithreaded task stage management */
    public static Phaser phaser;

    / * * *@paramThread Thread task *@paramThreadNum Number of threads *@paramDesc Task description */
    public HoldConcurrent(ThreadBase thread, int threadNum, String desc) {
        this(threadNum, desc);
        range(threadNum).forEach(x -> threads.add(thread.clone()));
    }

    / * * *@paramThreads Thread group *@paramDesc Task description */
    public HoldConcurrent(List<ThreadBase> threads, String desc) {
        this(threads.size(), desc);
        this.threads = threads;
    }

    private HoldConcurrent(int threadNum, String desc) {
        this.threadNum = threadNum;
        this.desc = StatisticsUtil.getFileName(desc);
        phaser = new Phaser(1);
        executorService = ThreadPoolUtil.createFixedPool(threadNum);
    }

    private HoldConcurrent(a) {}If there is no threadname, the name of the threadname is desc+ the number of threads used as the threadname, removing the date */
    public PerformanceResultBean start(a) {
        Thread funtester = new Thread(new FunTester());
        funtester.start();
        ThreadBase.progress = new Progress(threads, StatisticsUtil.getTrueName(desc));
        ThreadBase.progress.threadNum = 0;
        new Thread(ThreadBase.progress).start();
        startTime = Time.getTimeStamp();
        for (int i = 0; i < threadNum; i++) {
            if (HOLD.get() == 1) {
                threadNum = i;
                break;
            }
            ThreadBase thread = threads.get(i);
            if (StringUtils.isBlank(thread.threadName)) thread.threadName = StatisticsUtil.getTrueName(desc) + i;
            sleep(RUNUP_TIME / threadNum);
            executorService.execute(thread);
            ThreadBase.progress.threadNum = i + 1;
            logger.info("{} threads have been started!", i + 1);
        }
        phaser.arriveAndAwaitAdvance();
        executorService.shutdown();
        ThreadBase.progress.stop();
        threads.forEach(x -> {
            if (x.status()) failTotal++;
            errorTotal += x.errorNum;
            executeTotal += x.executeNum;
        });
        endTime = Time.getTimeStamp();
        HOLD.set(0);
        logger.info("Total {} threads, shared :{} s, total execution :{}, errors :{}, failures :{}", threadNum, Time.getTimeDiffer(startTime, endTime), formatLong(executeTotal), errorTotal, failTotal);
        return over();
    }

    private static class FunTester implements Runnable {

        @Override
        public void run(a) {
            waitForKey(INTPUT_KEY);
            HOLD.set(1);
            output("Pressure pause"); }}private PerformanceResultBean over(a) {
        Save.saveIntegerList(allTimes, DATA_Path.replace(LONG_Path, EMPTY) + StatisticsUtil.getFileName(threadNum, desc));
        Save.saveStringListSync(HoldConcurrent.requestMark, MARK_Path.replace(LONG_Path, EMPTY) + desc);
        allTimes = new Vector<>();
        requestMark = new Vector<>();
        return countQPS(threadNum, desc, Time.getTimeByTimestamp(startTime), Time.getTimeByTimestamp(endTime));
    }


    /** ** fixQPS <p> </p> </p@paramName Number of threads */
    public PerformanceResultBean countQPS(int name, String desc, String start, String end) {
        List<String> strings = RWUtil.readTxtFileByLine(Constant.DATA_Path + StatisticsUtil.getFileName(name, desc));
        int size = strings.size() == 0 ? 1 : strings.size();
        List<Integer> data = strings.stream().map(x -> changeStringToInt(x)).collect(toList());
        int sum = data.stream().mapToInt(x -> x).sum();
        String statistics = StatisticsUtil.statistics(data, desc, threadNum);
        int rt = sum / size;
        double qps = 1000.0 * name / (rt == 0 ? 1 : rt);
        double qps2 = (executeTotal + errorTotal) * 1000.0 / (endTime - startTime);
        return new PerformanceResultBean(desc, start, end, name, size, rt, qps, qps2, getPercent(executeTotal, errorTotal), getPercent(threadNum, failTotal), executeTotal, statistics);
    }


    /** * for later calculations **@param name
     * @param desc
     * @return* /
    public PerformanceResultBean countQPS(int name, String desc) {
        returncountQPS(name, desc, Time.getDate(), Time.getDate()); }}Copy the code

test

The test script

Simple multithreaded Sleep was used for testing, not the native HTTP interface. The script content is as follows:

package com.funtest.groovytest

import com.funtester.base.constaint.HoldThread
import com.funtester.base.constaint.ThreadBase
import com.funtester.frame.SourceCode
import com.funtester.frame.execute.HoldConcurrent
/** * compression pause */
class HoldTest extends SourceCode {
    public static void main(String[] args) {
        def tester = new FunTester()

        new HoldConcurrent(tester,10."Terminate incremental pressure maintenance stress test").start()

    }

    private static class FunTester extends HoldThread {

        FunTester() {
            super(null.100.true)}@Override
        protected void doing(a) throws Exception {
            sleep(0.1)}@Override
        ThreadBase clone(a) {
            return new FunTester()
        }

    }

}

Copy the code

Console output

Here is the full console output:

INFO->The current user: oker, working directory: / Users/oker IdeaProjects/funtester/SRC /test/ groovy/com/funtest/groovytest /, system coding format: utf-8, Mac OS X system version: 10.16
WARN->Please enter "FunTester" to continue!
INFO->One thread has been started!
INFO->Stop increasing pressure to maintain pressure test progress: 2%, current QPS: 0
INFO->Two threads have been started!
INFO->Termination of increasing pressure to maintain pressure testing progress: ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ 31%, current QPS: 20
INFO->Three threads have been started!
INFO->Termination of increasing pressure to maintain pressure testing progress: ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ 60%, current QPS: 30
FunTester
INFO->Total wait: 9.535 seconds!
INFO->Pressure to suspend
INFO->Four threads have been started!
INFO->Termination of increasing pressure to maintain pressure testing progress: ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ 89%, current QPS: 39
INFO->Termination of increasing pressure to maintain pressure testing progress: ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ ▍ 100%
INFO->Total 4 threads, total time: 13.439 s, total execution :227, error :0, failure :0
INFO->Data saved successfully! Name of the file: / Users/oker IdeaProjects/funtester/SRC /test/ groovy/com/funtest groovytest/long/data/termination of increasing pressure to maintain pressure test 231809 _4
INFO-> ~ ☢ ~ ~ ☢ ☢ ~ ~ ~ ~ ☢ ☢ ~ ~ ~ ~ ☢ ☢ ~ ~ ~ ~ ☢ ☢ ~ ~ ~ ~ ☢ ~ JSON ~ ☢ ~ ~ ☢ ☢ ~ ~ ~ ~ ☢ ☢ ~ ~ ~ ~ ☢ ☢ ~ ~ ~ ~ ☢ ☢ ~ ~ ~ ~ ☢ ~ > {> (1). "rt" : 103, > 1). The "failRate" : 0.0, > 1). "threads" : 4, > 1). The "deviation" : "56.51%" > 1). The "errorRate" : 0.0, > 1). "executeTotal" : 227, > ①. "QPS ":16.891137733462312, > ①. "total":227, > ①." QPS ":38.83495145631068, > ①. "endTime":"2021-09-23 18:09:16", > ①. > 1). The "table", "eJwBLwDQ / + + + Wkp WkquWwkSzml6Dms5Xnu5jlm74hIOW6lOW9k aVsOaNrumHj + S6jiAxMDI0 / eodgA = =" >} ~ ☢ ~ ~ ☢ ☢ ~ ~ ~ ~ ☢ ☢ ~ ~ ~ ~ ☢ ☢ ~ ~ ~ ~ ☢ ☢ ~ ~ ~ ~ ☢ ~ JSON ~ ☢ ~ ~ ☢ ☢ ~ ~ ~ ~ ☢ ☢ ~ ~ ~ ~ ☢ ☢ ~ ~ ~ ~ ☢ ~ ~ ☢ ☢ ~ ~ ~INFO->Too little data to draw! Should be greater than 1024

Process finished with exit code 0

Copy the code

It can be seen that the difference between QPS and QPS2 is more than one times, which is easy to understand, and the sense of both sight of arithmetic sequence. In the next step, we will try our best to implement local flexible increase and decrease of stress value and global task management function, please look forward to it!!

Have Fun ~ Tester!

FunTester test framework architecture diagram
JMeter Chinese Operation Manual with a sense of time
140 interview questions (UI, Linux, MySQL, API, security)
Graphic HTTP brain map
Fiddler Everywhere is the future
JVM command brain map for testing
The Definitive Guide to Java Performance
There is no data to drive automated tests
HTTP asynchronous connection pooling and multithreading practices
Bind mobile phone number performance test
Discussion on assembly point and multi-stage synchronization in performance testing
How to become a Full stack automation engineer
LT browser – responsive web testing tool
Simplify test cases

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

Dynamic model supercharging pause

The ultimate goal

Main functions:

Such benefits:

Source of demand

Train of thought

Basic implementation

The trigger condition

Multithreaded synchronization

Lead to problems

To realize the Demo

Multithreaded task class

Perform class

test

The test script

Console output

Have Fun ~ Tester!

Dynamic model supercharging pause

The ultimate goal

Main functions:

Such benefits:

Source of demand

Train of thought

Basic implementation

The trigger condition

Multithreaded synchronization

Lead to problems

To realize the Demo

Multithreaded task class

Perform class

test

The test script

Console output

Have Fun ~ Tester!

Related Posts

Leetcode 1342. Number of Steps to Reduce a Number to Zero (Python)

SpringCloud Service Consumer (Feign)(Finchley Version)(3)

How far can programmers go in writing?