- 💜 Hello everyone, I am Chen Dasheng, from Chongqing, non-academic training background, halfway out of the programmer.
- 💜 advocates freedom, open source and sharing.
- 💜 has not yet nuggets any creative title, efforts to move forward, always become the light in their hearts, come on
1. Why do we need tools
Let’s take a look at the definition of tool: it originally refers to an instrument used in the work, later extended to the means to achieve, accomplish or promote something. (– Explanation from Baidu)
1. From the point of view of human evolution, the ability to make and use tools is the most fundamental difference between humans and apes because tools help us become more productive and efficient.
2. Imagine if you didn’t use a tool for performance testing?
We can analyze it from the perspective of the definition of performance testing. Performance testing refers to the testing of various performance indicators of the system through the simulation of various normal, peak and abnormal load conditions by automated testing tools. Without tools, manual performance testing has the following disadvantages:
A) Testing requires a large amount of resources:
In order to simulate multi-load, concurrent scenarios that require multiple people to work together, testing often does not have many resources, and even if resources are available, the effectiveness of manual work is greatly reduced, and some scenarios cannot be completed by manual work alone.
B) Very poor repeatability:
Performance testing often requires repeated tuning and test execution, and it’s hard to imagine doing it all manually without the help of tools.
C) Poor test accuracy:
Due to the need to simulate a variety of load and concurrent scenarios, if the manual operation, there will inevitably be errors, and compared to tools or programs such errors will be greater, the test results are also very large.
D) Poor collection, collation and presentation of results:
If there is no tool, manual data collection will also have a large error compared to tools.
Second, the relationship between performance testing and performance testing tools
1. Performance test belongs to system test from the test stage, which has no direct relationship with the specific tool used. Using tools is only a method and means to improve the efficiency and accuracy of performance testing. In essence, there is no real difference between using tools for other things.
2. Performance testing is not equal to Loadrunner, LR is just one of the performance testing tools, and it is not a panacea, in some cases it does not come in handy.
3, the difference between automated testing tools and performance testing tools: performance testing tools are generally based on communication protocol (the client and server exchange information to comply with the convention), it can not care about the system UI, while automation is the use of object recognition technology, focus on the UI interface. Automation cannot or is difficult to load, but it is easy through protocol.
Three, performance test tool selection reference:
Usually in a company or project, we do some research on any tool we choose in order to find the right tool for the company or project. Performance testing tools are no exception. They can be considered from the following aspects:
1. Cost:
A) Cost of tools: tools are usually divided into commercial closed source (FEI) and non-commercial (MIAN) source (FEI). Commercial tools usually have powerful functions and charge for them, so they can provide after-sales service. If there are problems, professionals can help to deal with them. Open source tools are usually free and have limited functions. The organization that maintains the tools is also voluntary, so if there is a problem, it needs to solve it by itself. Choosing a commercial or open source tool depends on the company, such as the size of the company, the cost it is willing to bear, and the overall situation of the project. Generally speaking, large companies can usually afford the cost of tools and will consider buying commercial tools. Smaller companies may opt for open source tools due to financial pressures.
B) Learning cost: Using any tool requires learning, so there will be learning cost (such as time), so we need to take the learning cost of project team members into consideration when choosing tools. If there are two tools A and B that can meet the testing requirements of the project team, and if most people can use tool A while only A few people can use tool B, it is recommended to give priority to tool A. In general, it is desirable for testers to be familiar with a commercial (performance) tool for the process, an open source and free (performance) tool, and familiar (performance) scripting languages are basic requirements.
2. Supported Protocols:
Performance testing is usually closely related to protocols. For example, B/S systems usually use HTTP protocol for information exchange between clients and server vendors, while C/S systems usually use socket protocol for information exchange. When selecting tools, consider the protocol used by the project. Whether a test tool can meet the test requirements and achieve satisfactory test results is the most basic problem to be considered when choosing a test tool.
3. Vitality:
At present, there are many performance testing tools, such as LR, Jmeter and other popular tools, which have a lot of information on the Internet, but some niche tools may have little information on the Internet. If encountered in the process of tool use is very hand problems, seeking solution or help to record, and the mass of the tool will be relatively have the edge point, after all, the more people use, the more information, so their problems perhaps others would have encountered and solved, instant nobody met before, due to the use of many studies, Through the help of community or forum, I believe there will always be a master to help solve.
4. Cross-platform:
Needless to say, look at why JAVA continues to be popular.
5. Ability of secondary development
The tool supports secondary development, can better fit the testing needs, more aspects of statistical results data, and can be well integrated with the company’s existing systems.
Four, common performance testing tools:
Performance testing tools. Theoretically, all tools used in performance testing can be called performance testing tools, which are usually divided into the following categories:
Description:
- Server-side performance testing tools: Need to support generating stress and load, recording and generating scripts, setting up and deploying scenarios, generating concurrent users and placing constant stress on the system.
- Web front-end performance testing tools: need to be about the processing process of the specific page to be displayed by client tools such as the heart browser.
- Mobile terminal performance testing tool: the same as web terminal performance testing tool also needs to care about page processing, and specific data collection functions, such as: mobile PHONE CPU, memory, power, startup time and other data records.
- Resource monitoring tool: This is used to collect data during performance testing and to present the results in a good way.
PS: This article mainly summarizes the server-side performance testing tools LR, Jmeter and kylinTOP, and will also carry out a simple comparison of these three tools.
5. Characteristics of common performance testing tools
- JMeter: Multi-threaded model, very scalable, but not so high manufacturing pressure. It is good for suppressing some Tomcat services, or some back-end interfaces. The disadvantages of JMeter are that the pressure is not precisely controlled, it is difficult to adapt to high concurrency, and because it is written in JAVA, it is a resource drain.
- LoadRunner: More like an emulator, it is suitable for scenarios where the front-end structure is complex, such as simulating 100 user logins. LoadRunner provides good support for non-technical people. LoadRunner does not apply to back-end interfaces.
- KylinTOP: is a domestic set of performance testing, automated testing (UI, interface, APP), business & interface monitoring in one of the test platform, support cross-platform (WINDOWS/LINUX/SOLARIS/ Kirin /MAC, etc.) run.
JMeter, LoadRunner and kylinTOP
| describe | JMeter | LoadRunner | kylinTOP |
|---|---|---|---|
| Architecture principles | Through the intermediate proxy, monitor and collect concurrent client instructions, put them to generate scripts, and then send to the application server, and then monitor the application server feedback process | With JMeter | BS architecture, enterprise-level platform, supporting simultaneous operation of multiple people; Support project management, module management, user management, script use case management, parameter file management; Supports historical comparison of multiple run reports and multiple run history comparison of a single interface |
| The installation | Simple, decompression, more flexible | The LoadRunner installation package is large and cumbersome to install, and the tool itself is relatively bulky | All the way to foolproof installation, remember the installation directory location |
| Supported protocols | It supports a variety of protocols: HTTP, HTTPS, SOAP, FTP, Database via JDBC, JMS, etc., but it is not comprehensive enough compared to LR. For this reason, JEMter is relatively flexible and portable | The supported protocols are numerous and comprehensive, but this makes the tool itself cumbersome and inflexible | Supports multiple protocols, such as HTTP/HTTP2, RTSP, RTMP, Socket, and JAVA customization |
| Script recorded | Provides a function to generate test scripts by recording using local ProxyServer (ProxyServer), and also supports badboy recording to regenerate JMeter scripts | With powerful recording function, can directly record playback | Supports browser proxy recording and packet capture recording |
| Concurrency model | Increase the number of concurrent users by increasing the number of thread groups or by setting the number of loops | Support multiple concurrency models by selecting what scenario to set in the scenario and then selecting virtual number of users | Powerful and flexible concurrency model, including users per second, CAPS per second, target model, see the link for more details |
| Distributed testing | Support, can set up multiple agents, through remote control to achieve multiple machines concurrent pressure | With JMeter | Support control of multiple pressure machines to achieve distributed concurrent testing |
| Resource monitoring | Implemented through the JMeterPlugins plug-in and ServerAgent | Provides resource monitoring functions | The Monitor agent supports resource indicators such as CPU, memory, I/O, swap area, disk read/write, and throughput |
| The report analysis | Generate HTML reports through integration with Ant | It supports the generation of HTML and Word reports | You can merge index curves, filter curves, zoom in and out curves, compare different test results, analyze logs, and generate HTML reports |
| Virtual IP | Does not support | support | Support system virtual IP addresses and tool virtual IP addresses (support more IP addresses and lower resource usage) |
| Network simulation | Does not support | support | Supports limiting upload/download bandwidth for each user |
| scalability | Open source, can be modified according to the requirements of the source | By extending the function library | Support for JAVA language extensions that address proprietary algorithms or business processes such as encryption and decryption, JavaScript computation |
| Learning costs | Mainly self-study official website information | Online information and relevant training a lot of, buy the legal version of the word, still have technical support | Part of the function paid, we judge whether available, you can go to see the price |
Vi. Summary:
Commercial performance tools are much better than open source free software in terms of ease of use (script generation), concurrency model, and statistical indicators. They can greatly improve work efficiency, reduce the difficulty of use, and have much richer statistical indicators.
The resources
If you have to work but often feel a lot of difficult, feel that their testing aspects of learning is not enough fine want to continue to learn, want to transition afraid of learning not, can pay attention to me [programmer yifan], the public number can receive the latest software testing factory interview information and I along the way a full set of learning resources.
- ✔️ If this article is useful to you, please like 👍🏻 and follow me
- ✔️ we’ll see you next time! 👋 👋 👋
🌻 Recommended reading:
- Write an interface test tool, let the student sister see my ability
- “Performance testing Technical Notes Series 1” performance indicators industry reference | collector’s edition
- ✨ combat series ✨ 1️ micro channel small program automated test practice (attached Python source code) ❤️
- “I have been doing test development for more than a year, and my monthly salary has changed from 5K to 24K.