In this article, I introduce a performance testing tool called Locust. Based on the functions and features of Locust, I will introduce how to use Locust with examples.

An overview of the

Locust provides the following functions and features:

In the Locust testing framework, test scenarios are described using pure Python scripts. There is no need for clunky UI and bloated XML
For systems with the most common HTTP (S) protocol, Locust uses Python requests as a client, making scripting much easier. Locust allows you to test other systems or protocols in addition to HTTP (S) protocols, simply by writing a client for what you’re testing.
In terms of simulating concurrency, Locust is event-driven, using non-blocking IO and Coroutine provided by GEvent to implement concurrent requests at the network layer, enabling a single process to handle thousands of concurrent users. Coupled with Locust’s support for distribution, supporting hundreds of thousands of concurrent users is no dream.
Locust has a simple, clean Web interface that displays real-time testing progress. The load can be changed at any time during the test run. It can also run without the UI, making it easy to use for CI/CD testing.

We all know that the most core part of server-side performance testing tools is the pressure generator, and the key points of the pressure generator are two: one is to simulate real user operations, and the other is to simulate effective concurrency.

Locust can generate loads at a lower cost than LoadRunner or Jmeter, which use threads for one user/one concurrent load. The maximum Jmeter concurrency is limited by JVM size).
Support BDD (behavior-driven development) writing and executing tasks to better simulate the user’s real operation process.

Script Structure

Here is a simple example of how to use locUST:

#! /usr/bin/env python
# -*- coding: utf-8 -*-
# @time :2022/3/26 9:38 am
# @Author:boyizhang
from locust import TaskSet, HttpUser, task, run_single_user


class BaiduTaskSet(TaskSet) :
    """ "Mission Set """
    @task
    def search_by_key(self) :
        self.client.get('/')

class BaiduUser(HttpUser) :
    """ - creates concurrent user instances - creates user instances that execute task sets according to rules.
    # Set of tasks defined
    tasks = [BaiduTaskSet,]
    host = 'http://www.baidu.com'


if __name__ == '__main__':
    # debug: Indicates whether the debugging task can run successfully
    run_single_user(BaiduUser)
Copy the code

From the script, it can be seen that the script mainly contains two classes: BaiduTaskSet and BaiduUser. BaiduTaskSet inherits TaskSet, and BaiduUser inherits HttpUser.

BaiduTaskSet defines the details of tasks performed by users, while BaiduUser(User) is responsible for generating User instances to perform these tasks.

The User class is like a swarm of locusts, and each locust is an instance of the class. In turn, the TaskSet class acts like the brain of the locust, controlling the specific behavior of the locust by testing the corresponding TaskSet for the actual business scenario.

HttpUser(User)

In the User class, there is a client attribute, which corresponds to the request capabilities of the virtual User as a client.

Normally, we don’t use it directlyUserClass because of theclientProperty is not bound to any methods.
In the use ofUserClass, need to inherit firstUserClass, and then in an inherited subclassclientProperty to bind the client implementation class.

For the common HTTP(S) protocol, we can inherit the HttpUser class. HttpUser is the most commonly used user class. It adds a client attribute for making HTTP requests.

itsclientProperty boundHttpSessionClass, andHttpSessionAnd inherit fromrequests.Session. So in testingHTTP(S)The Locust script we can passclientProperty to usePython requestsLibrary of all methods, call methods also withrequestsExactly the same.
Due to therequests.SessionTherefore, the client automatically has the function of state memory between method calls. The common scenario is that the login status can be maintained after logging in to the systemSessionSo that subsequent HTTP request operations can carry the login state.

For protocols other than HTTP(S), we can also use Locust to test,

Although Locust only has built-in support for HTTP/HTTPS, it can be extended to test almost any system. Just based onUserClass implementsclientCan.
We can uselocust-plugins, this is the third party maintenance library, supportKafka,mqtt.webdriverSuch tests.

TaskSet

introduce

TaskSet implements scheduling algorithms for tasks executed by user instances, including task sequence planning, next task selection, task execution, sleep waiting, and interrupt control. From there, we can describe the business test scenario in a very succinct way in the TaskSet subclass, organizing and describing all the actions (tasks) and configuring the weights for different tasks.

There are two ways to define task information in a TaskSet subclass, the @Task decorator and the Tasks attribute.

using@taskA decorator

from locust import TaskSet, task, constant

class MyTaskSet(TaskSet) :
    def on_start(self) :
        """ Triggered when the user starts executing this task set :return: """
        print("task is running")
    def on_stop(self) :
        """ Triggered when the user stops executing this task set :return: ""
        print(("task is stopped"))
    @task(2)
    def task1(self) :
        print("User instance (%r) executing my_task1" % self)
    @task
    def task2(self) :
        print("User instance (%r) executing my_task2" % self)   
Copy the code

Adopt the Tasks attribute

You can use a list or dict. If you use list, the weight is 1:1

from locust import User, task, constant

class MyTaskSet(TaskSet) :

    def on_start(self) :
        """ Triggered when the user starts executing this task set :return: """
        print("task is running")
    def on_stop(self) :
        """ Triggered when the user stops executing this task set :return: ""
        print(("task is stopped"))
        
    def task1(self) :
        print("User instance (%r) executing my_task1" % self)
    def task2(self) :
        print("User instance (%r) executing my_task2" % self)   
    tasks = {task1:2, task2:1}
    If it is a list, the permissions for each task are 1:1
    # tasks = [task1, task2]
Copy the code

In both of these ways of defining task information, the weight attribute is set so that task1 is executed twice as often as Task2. If the weight of the task is not specified, the ratio is 1:1.

The on_start() and on_stop() methods override TaskSet’s on_start() and on_stop() methods, respectively. Fired when the user starts and stops executing this task set, respectively.

TaskSet nesting – Real simulated user scenarios

The tasks of the TaskSet class can be other TaskSet classes, allowing them to nest any number of levels. This allows us to define simulated user behavior in a more realistic way.

class NestTaskSet(TaskSet) :
    @task(3)
    def get_index_page(self) :
        print("get_Index_page")
    @task(7)
    class get_forum_page(TaskSet) :
        @task(3)
        def get_view_detail(self) :
            print('get_view_detail')
        @task(1)
        def create_forum(self) :
            print('create_forum')
        @task(1)
        def stop(self) :
            print('exit forum page')
            self.interrupt()


    @task(1)
    def get_info(self) :
        print('get info')
Copy the code

from locust import HttpUser, TaskSet, task, between

class ForumThread(TaskSet) :
    pass

class ForumPage(TaskSet) :
    # wait_time can be overridden for individual TaskSets
    wait_time = between(10.300)
    
    # TaskSets can be nested multiple levels
    tasks = {
        ForumThread:3
    }
    
    @task(3)
    def forum_index(self) :
        pass
    
    @task(1)
    def stop(self) :
        self.interrupt()

class AboutPage(TaskSet) :
    pass

class WebsiteUser(HttpUser) :
    wait_time = between(5.15)
    
    # We can specify sub TaskSets using the tasks dict
    tasks = {
        ForumPage: 20,
        AboutPage: 10,}# We can use the @task decorator as well as the  
    # tasks dict in the same Locust/TaskSet
    @task(10)
    def index(self) :
        pass
Copy the code

The important thing to note about Tasksets is that they never stop performing their task and you need to call the taskset.interrupt () method manually to stop execution.

In case 1 above, without the stop method, once the user enters get_forum_page, there is no exit from this class and only the task under get_forum_page will be executed.

scripting

Case 1:

Baidu search traffic is relatively large, now want to baidu search interface pressure test, how to write pressure test script?

#! /usr/bin/env python
# -*- coding: utf-8 -*-
# @time :2022/3/27 5:15 PM
# @Author:boyizhang
import random

from locust import TaskSet, task, FastHttpUser, HttpUser,run_single_user
from locust.clients import ResponseContextManager
from locust.runners import logger



class BaiduTask(TaskSet) :
    @task
    def search_by_baidu(self) :

        wd = random.choice(self.user.share_data)
        path = f"/s? wd={wd}"

        with self.client.get(path,catch_response=True) as res:
        # If you want different parameters of the same interface to be placed in the same group, you can use the following method
        # with self.client.get(path,catch_response=True,name="/s? wd=[wd]") as res:
            res: ResponseContextManager
            # If not, mark failure
            ifres.status_code ! =200:
                res.failure(res.text)


    def on_start(self) :
        logger.info('hello')
    def on_stop(self) :
        logger.info('goodbye')

class Baidu(HttpUser) :
    host = 'https://www.baidu.com'

    tasks = [BaiduTask,]
    share_data = [Po Xiaoyi.'boxiaoyi'.'Performance test'.'locust']

if __name__ == '__main__':
    run_single_user(Baidu)
Copy the code

In this case, by defining a list share_data in a subclass of HttpUser, one element of the list share_data can be randomly selected as an interface entry parameter when executing the task set.

Script execution

After lifting the veil of the first layer of mystery of Locust: script structure introduction, continue to talk about the implementation of Locust in combination with the case.When load tests are started, they are defined by the userNumber of usersAs well asSpawn rateGenerate user instances.

The specified TaskSet is executed by the user instance
The user instance will be selectedTaskSetOne of the tasks to perform
After execution, the thread puts the user to sleep for a specified period of time (user-defined)wait_time )
When the dormancy is over, theTaskSetSelect a new task to execute
Wait again, and so on.

This is how Locust performs.

Implement way

Command line execution

You can run the locust -h command to view the locust command-line parameters. You can also see: Locust command line parameter parsing for details.

$ locust -f example.py --headless --users 10 --spawn-rate 1 -H http://www.boxiaoyi.com -t 300s

Copy the code

-f: specifies the Locust script to execute
–headless: Disable the Web interface (using a terminal) and start testing immediately. Use -u and -t to control the number of users and the running time
-u/–users: indicates the peak number of concurrent Locust users. The main--headless 或--autostart Use together. You can change it during testing by typing W, W (generate 1, 10 users) and S, S (stop 1, 10 users)
-r/–spawn-rate: rate at which users are generated (users per second). The main and- - headless 或 - - autostartUsed together
-t/–run_time: stops at a specified time, for example, 300 seconds, 20 meters, 3 hours, or 1h30m. Only with--headless 或 --autostartUse together. The default is always running.
— Autostart: Start testing now (without Web UI). Use -u and -t to control the number of users and the running time. You can use both the terminal and the Web UI page to observe

The command line execution support, along with parameter support, can be integrated into the CI/CD process, but it is important to specify –run_time, otherwise the process will not exit automatically.

Web UI execution

$ locust -f example.py
Copy the code

Once Locust is started, open your browser and point to ithttp://localhost:8089. The following page is displayed: Click Start Swarming to start the load test.

Execution strategy

Single machine to perform

Single-node execution, that is, one Locust process is executed. Can refer to the case above

Distributed execution

A single process running Locust can simulate fairly high throughput. For a simple test plan, it should be able to make hundreds of requests per second, or thousands if FastHttpUser is used. But if your test plan is complex or you want to run more load, you’ll need to scale to multiple processes, or even multiple machines.

We can start a Locust instance with the –master flag and multiple work instances with the –worker flag.

If the worker process and master process are on the same machine, it is recommended that the number of workers do not exceed the number of CPU cores on the machine. Once exceeded, the effect may decrease rather than increase.
It can be used if the worker process is not on the same machine as the master process--master-hostPoint them to the IP/ host name of the machine where the master process is running.
Master and worker machine instances must have copies of locusfiles when Locust is distributed.
The Master instance runs the Locust Web interface and tells workers when to start/stop users. The worker runs the user and sends statistics back to the master instance. The Master instance itself does not run any users.

Pay attention to the point

Because Python cannot fully leverage more than one kernel per process (see GIL), it is generally good to run one Worker instance per processor kernel on Worker machines to take advantage of all their computing power.
There is almost no limit to the number of users each Worker instance can run. Locust/ GEvent can run thousands or even tens of thousands of users per process, as long as the user’s total request rate /RPS is not too high.
Locust logs a warning if it is running out of CPU resources.

How to use distributed?

Start Master instance:

locust -f my_locustfile.py --master
Copy the code

Then on each Worker (XXX is the IP of the master instance, or if your Worker is on the same machine as the master machine, omit this parameter entirely) :

locust -f my_locustfile.py --worker --master-host=xxx
Copy the code

Other parameters:

–master: Set locust to master mode. The Web interface will run on this node.
–worker: Set locust to worker mode.
–master-host=X.X.X.X: optional with –worker setting master node hostname /IP (default 127.0.0.1)
–master-port=5557: Optionally used with –worker to set the port number of the master node (default: 5557).
–master-bind-host=X.X.X.X: optional with –master. Determine the network interface to which the master node will be bound. The default is * (all available interfaces).
–master-bind-port=5557: Optionally with –master. Determine the network port on which the master node will listen. The default value is 5557.
–expect-workers=X: use –headless when starting the master node. The master node will then wait for X worker nodes to connect before starting the test.

Use Docker to perform distribution

version: '3'

services:
  master:
    image: locustio/locust
    ports:
      - 8089: 8089
      - 5557: 5557
    volumes:
      - ./:/myexample
    command: -f /myexample/locustfile.py WebsiteUser --master -H http://www.baidu.com

  worker:
    image: locustio/locust
    links:
      - master
    volumes:
      - ./:/myexample
    command: -f /myexample/locustfile.py WebsiteUser --worker --master-port=5557
Copy the code

Start the

$ docker-compose -d -f myexample/run_locust_by_docker.yml up --scale worker=3
Copy the code

Results analysis

During the Locust test, we can see the results running in real time on the Web interface. The following indicators are shown: concurrency, RPS, failure rate, and latency. In addition, trend charts of some indicators are also shown, such as Case 1- Figure 3.

Execute case 1: locust -f locustfile.py, and you can see the following results on the Web page:

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

Performance Tools Locust Learning – Basics

An overview of the

Script Structure

HttpUser(User)

TaskSet

introduce

TaskSet nesting – Real simulated user scenarios

scripting

Case 1:

Script execution

Implement way

Command line execution

Web UI execution

Execution strategy

Single machine to perform

Distributed execution

How to use distributed?

Use Docker to perform distribution

Results analysis

Performance Tools Locust Learning – Basics

An overview of the

Script Structure

HttpUser(User)

TaskSet

introduce

TaskSet nesting – Real simulated user scenarios

scripting

Case 1:

Script execution

Implement way

Command line execution

Web UI execution

Execution strategy

Single machine to perform

Distributed execution

How to use distributed?

Use Docker to perform distribution

Results analysis

Related Posts

[Wei] The company survived for 24 days

Don’t call yourself a programmer, my ten years of IT career summary

PHP source code protection – ANALYSIS of PHP encryption schemes & decryption and restoration