“This is the 14th day of my participation in the Gwen Challenge in November. Check out the details: The Last Gwen Challenge in 2021.”

An overview,

In this article, we will introduce how to use JMeter+InfluxDB+Grafana to create a visual real-time monitoring of pressure measurements.

Second, the introduction

We often use JMeter for performance testing, and it is difficult to check the performance status of the application in the process of pressure testing in time. We always have to wait until the test is completed to read the Report. If it is a long pressure test, such as pressure testing for 1 or 2 days, it will be even more annoying.

While pressing, we summarize the output in real time through the GenerateSummaryResults listener, and we can see the following

GenerateSummaryResults summarizes a lot of information to be concerned about in a pressure survey, including: Avg (Average), Min (Minimum), Max (Maximum) response time, TPS, error rate, number of active threads, etc. Jmeter. properties default 30-second statistics frequency, if we want to use less than 30 seconds of granular statistics frequency. There would be a lot of output on the console, and it would be hard to see the results.

# jmeter.properties 
# Define the following property to automatically start a summariser with that name
# (applies to non-GUI mode only)
# summariser.name=summary
#
# interval between summaries (in seconds) default 3 minutes
# summariser.interval=30
#
# Write messages to log file
# summariser.log=true
#
# Write messages to System.out
# summariser.out=true
Copy the code

On the other hand, this console output is not a very friendly way to share it with other people, we need a nice simple graph like the HTML Dashboard, but we need to wait for the whole test to complete.

Therefore, we can remove GenerateSummaryResults from the test plan and send the test results to the database for persistence so that we can query the database in SQL to create the presentation chart.

Since JMeter2.13, by providing realize AbstractBackendListenerClient class, you can use (JDBC, JMS, Webservice,…). The Backend Listener sends real-time storage results to the database.

Send to Graphite BackendListenerGraphiteBackendListenerClient, allows the statistical indicators

This feature provides:

  • Real-time data
  • Beautiful chart
  • Ability to compare more than 2 test plans
  • As long as JMeter produces the same Backend, monitoring data can be stored

.

JMeter 3.2 Backend Listener introduced in InfluxDBBackendListenerClient done using UDP or HTTP protocol will be sent to InfluxDB statistical indicators

This feature provides:

  • Real-time data
  • Beautiful chart
  • Ability to compare more than 2 test plans
  • Ability to add comments to charts
  • As long as JMeter produces the same Backend, monitoring data can be stored

.

Second, solutions

JMeter introduces Backend listeners, which are used to send statistical data to the timing database Influxdb in real time during pressure measurement. By configuring Grafana data sources to connect to the Influxdb, we can create cool visual kanban. And can obtain real-time test index data.

Related introduction:

Sequential database: Processes time-tagged (sequential changes in time, i.e. time serialization) data, also known as time series data. Think of it like an SQL table where time is its primary key InfluxDB: an open source distributed temporal, event, and metric database written in Go with no external dependencies. The database is now primarily used to store time-stamp data involving a large amount of data, such as DevOps monitoring data, APP Metrics, loT sensor data and real-time analytics data.

Official website: www.influxdata.com/

Data on the DEMO

> select* from weather
name: weather
-------------
time                    altitude        area    humidity        temperature
1456386985094000000     1000            北      18              17
1456386985094000000     5000            上      20              47
1456386985094000000     5000            北      26              68
1456386985094000000     1000            广      17              83
1456387267668000000     1000            上      12              77
1456387267668000000     1000            北      16              20
1456387267668000000     5000            广      - 3              66
1456387267668000000     5000            上      19              60
Copy the code

Compare with nouns in a traditional database

Noun in influxDB Concepts in traditional databases
database The database
measurement Tables in the database
points A row of data in a table

Some concepts unique to InfluxDB are composed of time stamps, fields, and tags.

Point attribute Concepts in traditional databases
time Each data record time is the primary index in the database (automatically generated)
fields The various recorded values (attributes without indexes) are the recorded values: temperature, humidity
tags Various indexed attributes: region, altitude

JMeter&InfluxDB integration

1. Installation of InfluxDB

1.1. Installation requirements

Network port InfluxDB The following network ports are used by default:

  • TCP port 8086 is used for client-server communication over the INFLUENCE XDB’S HTTP API
  • TCP port 8088 is used for RPC services for backup and restore
  • In addition to the ports above, InfluxDB also provides several plug-ins that may require custom ports. You can modify all port mappings through the configuration file located at/etc/influxdb/influxdb.confDefault location.

Network Time Protocol (NTP)

InfluxDB assigns time stamps to data using the host’s local UTC time and for coordination purposes. Synchronize the time between hosts using the Network Time Protocol (NTP); If the host clock is not synchronized with NTP, the timestamp of data written to the InfluxDB may be inaccurate.

2. Download and install

Wget https://dl.influxdata.com/influxdb/releases/influxdb-1.6.2.x86_64.rpm sudo yum localinstall Influxdb - 1.6.2. X86_64. RPMCopy the code

3. Parameter Description Value InfluxDB

Find InfluxDB configuration file (/ etc/InfluxDB/InfluxDB conf)

vi /etc/influxdb/influxdb.conf
Copy the code

Most of the Settings in influxdb.conf are commented out; All commented out Settings are determined to be internal defaults. Any uncommented Settings in the configuration file override the internal defaults. Note that the local configuration file does not need to contain every configuration setting. If you cannot find the configuration file, use InfluxDB to display configuration commands

Influxd config
Copy the code

Jmeter uses the Graphite protocol to write data to InfluxDB, so it needs to be enabled in the InfluxDB configuration file, as shown below:

[[graphite]]
 enabled = true
 bind-address = ": 2003"
 database = "jmeter"
 retention-policy = ""
 protocol = "tcp"
 batch-size = 5000
 batch-pending = 10
 batch-timeout = "1s"
 consistency-level = "one"
 separator = "."
 udp-read-buffer = 0
Copy the code

After the modification, run the following command to load InfluxDB

There are two ways to start the InfluxDB using a configuration file:

1. Use the -config option to point the process to the correct configuration file

influxd -config /etc/influxdb/influxdb.conf
Copy the code

2. Set environment variables to the path of the INFLUXDBCONFIGPATH configuration file and start the software

echo $INFLUXDB_CONFIG_PATH
/etc/influxdb/influxdb.conf
influxd
Copy the code

InfluxDB operation

[root@zuozewei ~]# influx # Login to database
Connected to http://localhost:8086 version 1.6.2
InfluxDB shell version: 1.6.2
> show databases  View all databases
name: databases
name
----
_internal
> CREATE DATABASE "jmeter" Create database
> use jmeter               Switch database
Using database jmeter
> CREATE USER "admin" WITH PASSWORD 'admin' WITH ALL PRIVILEGES Create a user with administrator privileges
Copy the code

4. JMeter configuration

  1. Create a test plan and add Backend Listeners
  2. Setting the InfluxDB IP address and port Indicates the InfluxDB IP address and port
  3. Run the test, wait a few seconds Run the test, wait a few seconds
  4. Check whether JMeter generates error logs. Check whether JMeter generates error logs

Port Description:

  • Port 8086, the port Grafana uses to fetch data from the database
  • Port 2003, the port on which JMeter sends data to the database

View all the tables in the JMeter database

> show measurements
name: measurements
name
----
delta
jmeter.all.a.avg
jmeter.all.a.count
jmeter.all.a.max
jmeter.all.a.min
jmeter.all.a.pct90
jmeter.all.a.pct95
jmeter.all.a.pct99
jmeter.all.h.count
jmeter.all.ko.count
jmeter.all.ok.avg
jmeter.all.ok.count
jmeter.all.ok.max
jmeter.all.ok.min
jmeter.all.ok.pct90
jmeter.all.ok.pct95
jmeter.all.ok.pct99
jmeter.test.endedT
jmeter.test.maxAT
jmeter.test.meanAT
jmeter.test.minAT
jmeter.test.startedT
total
Copy the code

OK, we have been able to send statistics data successfully to InfluxDB

Grafana and InfluxDB integration

What is Grafana?

Grafana is an open source software with rich metric dashboards and graphics editors for Graphite, Elasticsearch, OpenTSDB, Prometheus, InfluxDB. Simply put, it is a set of open source WEB visualization platform.

Grafana.com/

1. Start the installation

Download and install

Wget https://s3-us-west-2.amazonaws.com/grafana-releases/release/grafana-4.2.0-1.x86_64.rpm sudo yum localinstall Grafana 4.2.0-1. X86_64. RPMCopy the code

Start the

service grafana-server start
Starting grafana-server (via systemctl):                   [  OK  ]
Copy the code

Use your browser to open http://IP:3000/login and visit the Grafana home page

Example Create the InfluxDB data source

Click Save&Test to ensure that the InfluxDB can be connected

Create a dashboard

Add Graph panel

Data binding

Number of threads/User related indicators

  • Test. minat-min Active Threads: specifies the minimum number of active threads
  • Test. maxat-max Active Threads: Specifies the maximum number of active threads
  • Test. meanat-mean Active Threads: Indicates the number of active threads
  • Test. startedT-Started Threads: Indicates the number of Started threads
  • Test. endedt-finished Threads: Indicates the number of Finished threads

Response time indicator

  • .ok.count: number of successful responses to the sampler
  • .h.mount: Number of hits per second
  • .ok.min: the minimum successful response time of the sampler
  • .ok. Max: Maximum successful response time of the sampler
  • .ok.avg: average response time to sampler success
  • .ok. PCT: percentage of successful sampler responses
  • .ko. Count: indicates the number of failed sampler responses
  • .ko.min: minimum response time for sampler failure
  • .ko. Max: Indicates the maximum response time of sampling failure
  • .ko. Avg: Average response time to sampler failures
  • .ko. PCT: percentage of sampler failure responses
  • .a.count: number of sampler responses (sum of ok.count and ko.count)
  • .a.IN: Minimum response time of the sampler (minimum of ok. Count and ko. Count)
  • .a.max: maximum response time of the sampler (maximum of ok. Count and ko. Count)
  • .a.vg: average response time of the sampler (average of ok.count and ko.count)
  • .a.ct: percentage of sampler response (calculated based on total and failed samples)

The default percentile of Backend Listener is 90; 95; 99 “, i.e., the percentiles are 90%, 95% and 99%. Graphite use point (“. ) to split elements, which can be confused with decimal percentile values. JMeter converts any such value, replacing the point (“) with an underscore (” – “). ). For example, “99.9” becomes “99_9”.

By default, JMeter sends metrics for all samplers accumulated under samplerName “all”. If the configuration BackendListenerSamplersList, the JMeter also sends matching index of sample name, the premise is to configure summaryOnly = true

Effect in pressure measurement

Import the Dashboard template

If you were asked to design a nice Dashboard, you probably wouldn’t want to do it from scratch, but the Grafana website offers a rich library of templates that you can find and then extend.

Official website template library: grafana.com/dashboards

Search for Kanban templates

Here, I chose to download the template with 3000+ downloads

Template JSON file

Import the template

Select data source

JMeter Backend Listener Settings

Animation effect during manometry

At this point, my initial pressure measurement visual real-time monitoring is complete.