Author: Aurora senior test engineer — Wang Suhua & Yang Wanwu

1. Demand background

This section describes how to use locUST distributed load testing tool to help you quickly master the basic methods of performance testing and improve work efficiency

2. Explanation of terminology

Definition of performance testing: testing various performance indicators of a system by means of an automated test tool simulating various normal, peak and load conditions

Response time: To measure the performance of a single interface, the response time can be simply understood as the time when the user sends a request to the response data returned by the server.

Number of concurrent users: Specifies the number of users who submit requests to the system at the same time. The requests may be in the same or different scenarios.

Transaction per second (TPS) To measure the performance of the single interface, a transaction can be understood as a process in which a client sends a request to the server and the server responds.

Single push: send a message to push only one user ID (multiple users receive messages by pushing multiple messages)

Multiple Tweets: A push message can push multiple user ids simultaneously. (Multiple users receive messages through a single message)

3. Test strategies

3.1 Demand Analysis:

According to the requirements, the following scenarios need to be tested:

Use the domain list push to see the maximum TPS sent

Use the domain name multiple push to see the maximum TPS sent

3.2 Test Method

1. Preparation for performance test:

Analysis of business scenarios: what are the scenarios and the scope is wide? Discuss with development and product to determine the scope of this test

Analyze test strategy: Find out what test scenarios are designed

Analyze the production environment

Test tool selection: How to test performance

2 Purpose of performance test:

The performance test simulates the scenario pressure in advance to find possible bottlenecks in the system and tune the system to reduce the risk of server downtime

Performance testing can also be used to evaluate the system load capacity of the software under test under different loads, providing performance data decisions for system deployment

3 Performance decline curve analysis:

As the number of users increases, performance declines, as shown in the figure:

The blue line shows TPS, and the yellow line shows response time

In the process of TPS increase, the response time will be in A low state at the beginning, that is, before point A. Then the response time starts to increase somewhat until the business can afford point B, where TPS still has room to grow. The pressure is then increased until the maximum TPS is reached at point C. We increase the pressure further, the response time increases, but the TPS decreases.

​

3.3 Test Tools

Use locUST, a distributed pressure measuring tool

3.4 Test Environment

3.4.1 Flow chart of pressure measurement data

3.4.2 System under test

API services are deployed in dual-node mode, and related dependent services are deployed on the dual-node mode

Server resources: 4 cores 16 GB

Using domain name compression: The domain name is forwarded to the server through LBS

4 Test tool Locust

4.1 briefly

JMeter and LoadRunner are older performance testing tools that use threads as virtual users, while Locust implements concurrent users based on coroutines. Coroutines are smaller units than threads, also known as subthreads, and can run multiple coroutines in one thread. Therefore, Locust based on coroutine can be deployed not only on multiple machines, but also on a single host machine

4.2 Pressure Test Tool: Locust, version: 0.14.5

4.3 Locust Local Host Deployment Procedure

1. First we need to use Python +locust to write the abc.py file that the program needs to run

Script description:

Create a TPush(TaskSet) class that inherits TaskSet, and write the requested interface and related information below. Self. client calls the get and POST methods, just like requests;

@task decorates this method as user behavior, and the parameters in parentheses indicate the execution weight of this behavior: the higher the value is, the higher the execution frequency is. The default value is 1 if not set.

On_start method: this method is called when the impersonator user starts executing the TaskSet class;

The WebsiteUser() class is used to set the basic properties of the generated load:

  • Tasks: Points to the class that defines user behavior;

  • Min_wait: minimum wait time between tasks simulating load, in milliseconds.

  • Max_wait: Indicates the maximum wait time between tasks simulating load, in milliseconds.

2. Create locust run master:

After writing the locust file to run, we need to create a master using the command “locust -f XXXX.py –master”. This master is not involved in creating concurrent users, it is mainly used for listening and collecting statistics.

2.1 For the Web end, we can use python OS system function directly write CMD command to run

2.2 Using no-web, you can also use the System function of the OS in Python to directly write CMD commands to run

Of course, you can also directly enter the command to execute

When we run the py file, we can see the following information, indicating that we have started the locust distributed master node

3. Create a slave for the LOCust to run

We can create a slave node for creating concurrent users by copying the master file and changing the file name to make sure that the slave is running on the same file as the master. We can create as many slaves as our machine has cpus. Typically, a slave uses a SINGLE CPU to create concurrent users

Py abc_slave1.py 3.1 Then enter CMD in the main function of each slave file

3.2 You can also run the slave command

Run multiple slaves, and you can view the following to determine whether the distributed deployment is successful

After all slaves are enabled, you can view the console information of the master. If the following information is displayed, the LocUST distributed deployment is successful

4. Open the Web terminal and view the distributed slave

Landing page

Before starting the simulate test, you need to specify the Number of Total Users to Simulate, Spawn rate per second, and HOST name.

The Spawn rate stops when the Number of total users to Simulate reaches the Number of users.

View the number of slave nodes

It can be seen from the above figure that all slaves are in the running state and the pressure is evenly distributed.

Test results data download

In LocUST, there is no way to set the execution time when using web mode. Therefore, the web mode will continue to run until you click Stop. If you want to stop the web mode automatically after running it for a period of time, you need to add stop_timeout during parameter configuration. Value Indicates the execution time in seconds.

class websitUser(HttpLocust):

Task_set = Login # Defines the set of tasks that need to be executed

Min_wait = 1000 # minimum wait time

Max_wait = 2000

Stop_timeout = 60 # Stop time

5 Locate interface performance bottlenecks on the LOCUST visual interface

Running for a period of time with 100 total users and 1 user per second, we look at the CustWeb interface:

We found that TPS stayed at a certain value (9.8), and then we clicked “Charts” to see the details of TPS, latency and user numbers

It can be seen that before TPS is 10, it increases with the increase of users; However, when the number of users is increased to 10, the TPS is still stable at about 10, but the delay increases.

Then look at the server performance

It can be seen that the CPU is occupied by the program, so we can initially assume that the TPS of the interface is 10, and the bottleneck is on the CPU.

6. Data acquisition operation

Single push interface: /single

Multi-push interface: /mutiplie

7. Data analysis

Memory consumption is less obvious than CPU consumption during pressure measurement. From the perspective of system stability, performance indicators under stable conditions should be considered. Data acquisition needs to be analyzed and the following conclusions are drawn:

Single push: increase the number of users per second by 3, the number of concurrent users is 160, and the CPU consumption is about 80% to continuously measure the following performance indicators.

Multiple push: increase the number of users per second by 2, the number of concurrent users is 120, and the CPU consumption is around 78% to continuously pressure the following performance indicators.

Note: For multiple pushes, considering the size of the body, deviceTokens sets three numbers, which make no difference for interfaces. Resources are consumed only when data is delivered. In addition, with the experience of single push and the system logic of multiple pushes, resource consumption can be easily identified. So less data is collected.)

8. Test conclusions

Single push interface: The TPS test result of the interface is 1757/s, and 90% of the requests can be responded within 0.1 seconds. Because multiple machines are deployed online, this TPS can only be used in the current configuration and deployment, and the main bottleneck is device upgrade.

Multi-push interface: the TPS pressure test result of the interface is 1342/s, and 90% requests can be responded within 0.1 seconds. Because there are multiple machines on the line, this TPS can only be used in the current configuration and deployment, and the main bottleneck is device upgrade.

The data cited in this article only represent service performance in a particular pressure test environment