Introduction to Performance Testing
Before entering the benchmark section, you need to introduce some concepts of performance testing to help you read the results better.
- What is performance testing
Performance test is to test various performance indexes of the system by simulating various normal, peak and abnormal load conditions with test tools.
Objective: To verify whether the software system can achieve the performance index proposed by users, find the performance bottleneck in the system and optimize it.
Performance testing related concepts
The response time
The time from the time the request is initiated to the time the request response is received
Network transmission time: T1 t4
Server processing time: T2 T3
Concurrency/Number of concurrent requests/Number of concurrent users
- concurrent
Narrow sense: do the same thing at the same time broad sense: do different things at the same time, mix the scene
Performance testing: simple narrow concurrency, and then broad concurrency, that is, simple interface performance testing, business performance testing, and mixed business scenario performance testing
-
Concurrency: indicates the number of users sending requests to the server per unit time
Number of concurrent users: the number of performance test virtual users that simulate real users sending requests to the server
The number of system users: in a narrow sense, can be understood as the number of registered system users; Broadly speaking, it can be understood as all the users who have accessed the system
Number of online users: indicates the number of users who are accessing the system and may not have pressure
Throughput and throughput rate
An important indicator to measure network performance
Throughput: The amount of data transferred over the network (number of customer requests handled)
Throughput rate: Indicates the amount of data successfully transmitted on the network in a unit of time (seconds, minutes, hours, and days), such as requests/second or pages/second
TPS (Transaction Per Second) number of transactions Per Second
Transaction: Can be viewed as an action or a set of actions, such as login, which starts and ends as a transaction.
TPS: Measures the ability of the system to handle transactions or transactions, that is, the ability of the server to handle customer requests, the number of transactions handled per second, commonly used in LoadRunner, set up transactions, and then count how many defined transactions the system can successfully complete per unit of time.
Hits Per Second
Hits: Indicates the number of HTTP requests received by the Web Server.
Click rate: The number of HTTP requests submitted by users to the Web Server per second per unit time.
Distinguish the number of mouse clicks: for example, if you request a Web page with three images, the number of clicks requested to the Web Server is 1+3=4, and one click of the mouse allows you to visit the Web page, with only one click
Server Resource Usage
Server resource usage refers to the resource utilization of the system under load. The lower the resource usage, the better the system. For example, CPU usage, memory usage, Cache query command rate, and disk I/O read/write rate.
CPU common counters:
| counter | Counter analysis |
|---|---|
| %Processor Time | If this value consistently exceeds 95%, the bottleneck is the CPU. |
| Processor Queue Length | Refers to the number of threads in the queue to be processed. If there are always more than two threads in the processor queue, it usually indicates that the processor is blocked. In addition, the Server Work Queues that track the computer will display the Server Work Queues\Queue Length of the current Length, indicating possible processor congestion if the Queue Length is consistently greater than 4. |
| %User Time | Represents database operations that consume CPU, such as sorting and executing aggregate functions. If the value is high, consider increasing the index, and try to reduce the value by simple table joins, horizontal splitting of large tables, and so on. |
| %Privileged Time | This counter is a time-specific value, the percentage of time (CPU kernel time) spent processing thread execution code in privileged mode. If the value of this parameter and the value of the “Physical Disk” parameter are consistently high, it indicates an I/O problem. Consider replacing the disk system. |
| %DPC Time | CPU usage of a NIC: The lower the value, the better. If the value is greater than 50% and the Processor: %Processor Time is very high, it indicates that the network is affected by the CPU and cannot work properly |
Common memory counters:
| counter | Counter analysis |
|---|---|
| Available Mbytes | Number of available physical memory. If the value of Available Mbytes is small, it indicates that there may not be enough total memory on the computer, or that a program has not freed memory. |
| Page/sec | Indicates the number of pages taken from disk due to a hardware page error, or pages taken from disk due to a page error, or pages written to disk due to a page error to free memory space. In general, if the Page/ SEC continues to be higher than a few hundred, then swapping activities should be further investigated and memory may need to be increased to reduce the need for Page breaks. A large value of Page/ SEC does not necessarily indicate a memory problem, but may be caused by running a program that uses memory-mapped files. |
| Page read/sec | The number of times a Page file must be read in order to resolve a reference to memory. The lower the value, the better. A value greater than 5 indicates disk reads rather than buffer reads. |
| Page Faults/sec | Refers to the number of software Page failures per second, and Page/ SEC simply indicates that the data is not immediately available in the specified memory. |
| Cache Bytes | The File System Cache, by default, is 50% of available physical memory. If a memory leak is suspected, Monitor Memory \Available Bytes, Process\working Set, and Process\Handle Count, If the memory processes Process\Private Bytes\Process\Working Set and Process\Handle Count are suspected, and if a kernel-mode Process is suspected to have caused the leak, Then you should also monitor Memory\ Pool Nonpaged Bytes, Memory\ Pool Nonpaged Allocs, and Process (process_name) Nonpaged Bytes |
| Pages/sec | Number of pages retrieved per second, which should be less than 1 page per second |
| Committed Bytes | Confirm virtual memory in bytes, physical memory that confirms that space is reserved on the disk page file. Each physical disk can have one or more page files. This counter displays only the last value observed; It’s not an average. This refers to how much virtual memory is being used. Virtual memory is the amount of memory that occupies disk space, not physical memory. |
Common counters for physical disks:
| counter | Counter analysis |
|---|---|
| %Disk Time | Refers to the percentage of time the selected disk drive takes to service read or write requests. If all three counters are large, the disk is not the bottleneck. If only %Disk Time is large and the other two are moderate, hard disks may be a bottleneck. If the value continues to exceed 80%, memory leaks are possible. |
| Avg.Disk Queue Length | It refers to the queue length of the current disk. In layman’s terms, a counter reflects how long it took the disk to complete a request, and a higher value indicates that the disk controller is continually retrying the disk due to a failure. These failures increase the average disk transfer time. |
| Average Disk Read /Write Queue Length | Average number of read (write) requests (queue) |
| Disk Reads(Writes)/sec | The total number of disk read/write operations per second on the physical disk must be less than the maximum disk capacity |
| Average Disk sec/Read | The average time in seconds to read data from this disk |
| Average Disk sec/Transfer | The average time in seconds to write data to the disk. Generally speaking, if the value is less than 15ms, 15 to 30ms is good, and 30 to 60ms is acceptable. If the value is greater than 60ms, you need to consider replacing disks or using RAID |
| Bytes Total/sec | The rate at which bytes are sent and received, including frame characters. To determine whether the network connection speed is a bottleneck, compare the value of this counter with the current network bandwidth. |
Common thread counters:
| counter | Counter analysis |
|---|---|
| Context Switch/sec | If you decide to increase the size of the thread byte pool, you should monitor both the inetinfo and dllHost process counters for instantiation. Adding threads might increase the number of context switches so that performance drops rather than increases. If the upper and lower switching values of multiple instances are very high, the thread byte pool should be reduced. |
Common process counters:
| counter | Counter analysis |
|---|---|
| Process/ %Processor Time | The amount of processor time consumed by the processor. If the Server is dedicated to SQL Server, the maximum acceptable limit is 80% to 85% |
| Page Faults/sec | Compare process page failures with system page failures to determine the impact of the process on system page failures |
| Working set | The most recent memory pages used by processing threads reflect the amount of memory used by each process. If the server has enough free memory, pages are left in memory, and when free memory falls below a certain threshold, pages are eliminated from memory. |
| Process/Private Bytes | The number of bytes allocated by a process that cannot be shared with other processes. This counter is used to determine whether a process leaks memory during a performance test. We can focus on monitoring the inetInfo process’s Private Bytes. If the process’s Private Bytes counter continues to increase during the performance test, or if the process’s Private Bytes count remains high for some time after the performance test is stopped, It indicates that the application has memory leaks. |
Common service buffer counters:
| counter | Counter analysis |
|---|---|
| File Cache Hits | The specific value that the file cache hit |
| File Cache Hits % | The ratio of cache hits to total cache requests reflects how well IIS’s file cache Settings are working. For a mostly static web site, this value should be around 80%. |
| File Cache Flushes | The number of times the file cache has been refreshed since the server started. If the refresh is too slow, memory will be wasted. If the flush rate is too fast, objects in the Cache are frequently discarded and regenerated, which does not play the role of Cache. By dividing File Cache Hits by File Cache Flushes, you can calculate the ratio of Cache hit ratio to Cache flush rate. By observing these two values, you can get an appropriate refresh value |
| Connection Refused | A lower value is better; a higher value indicates a bottleneck in the network adapter or processor. |
Common network counters:
| counter | Counter analysis |
|---|---|
| Bytes Total/sec | The speed of sending and receiving bytes, including frame characters, to determine whether the network connection speed is a bottleneck, can be divided by the value of this counter and the current network bandwidth, the result should be less than 50%. |
Apache common counters:
| counter | Counter analysis |
|---|---|
| Apache CPU Usage apache | CPU usage of the server |
| Kbytes Sent/sec | Number of bytes sent per second by the server |
| Hits/sec | Click through rate per second for Apache services |
| #Busy Workers | Apache service usage |
| #Idle Workers | Apache service idle rate |
Mysql > select * from user where user = ‘user’;
| counter | Counter analysis |
|---|---|
| Threads_connected | Indicates how many clients are currently connected to the Mysql server, whether the number of connections is too many, and whether there are network problems. It is dynamic. If Threads_connected == MAX_connections, the database system cannot provide more connections. If Threads_connected == MAX_connections, the database system will refuse to create new connections. If Threads_connected == MAX_connections, the database system will display an error message if the application does not do too much error handling. |
| Threads_running | If the database is overloaded, you will get a value that is growing. This value can also be less than a preset value. This value is fine if it exceeds the limit for a short time, but if it exceeds the default and does not fall back within 5 seconds, some other values should be monitored as well. |
| Aborted_clients | The number of clients that were abnormally interrupted (because the client connected to the Mysql server was not properly disconnected or shut down). For some applications there is no impact, but for others this value may be tracked, so an abnormal disconnection may indicate a problem for some applications |
| Questions | The number of queries obtained per second. It can also be the total number of queries, depending on the type of command you want to get different values. |
| Handler_* | These values are worth tracking if you want to monitor low-level database load. If the Handler_read_rnd_next value is significantly different from the normal value, there may be optimization or index problems. Handler_rollback indicates the number of queries that were rolled back from the transaction. |
| Opened_tables | Indicates the number of missed table cache hits. If the value is large, you need to increase the table_cache value. |
| Select_full_join | Without primary key join execution, the value may be zero. This is a good way to catch development errors because such queries can degrade system performance. |
| Select_scan | The number of full-table search queries executed, and if this value continues to grow, it indicates optimization, lack of necessary indexes, or other problems. |
| Select_queries | The number of queries that exceed this value (–long-query-time) or that do not use indexes. There are minor conflicts for all queries. If this value increases, it indicates a performance problem. |
| Threads_created | This value is generally low. A high value could indicate that thread_cache needs to be increased, or that you are experiencing a persistent increase in connections, indicating a potential problem. |
| Pending normal aio reads | This value is the size of innoDB IO request queries. If the value is outside the 10-20 range, there may be some bottlenecks. |
Performance bottleneck
1. Performance bottlenecks on hardware:
The bottleneck refers to the CPU, memory, and disk read/write bottleneck.
2. Performance bottlenecks in application software:
Generally, it refers to server operating system bottlenecks (parameter configuration), database bottlenecks (parameter configuration), Web server bottlenecks (parameter configuration), middleware bottlenecks (parameter configuration), and so on
3. Performance bottlenecks in applications:
This generally refers to developers, applications developed (such as SQL statements, database design, business logic, algorithms, etc.).
4. Performance bottlenecks on operating systems:
It refers to the operating system such as Windows and Linux. For example, the physical memory is insufficient or the virtual memory is incorrectly configured. If the virtual memory is incorrectly configured, the swap rate of the virtual memory greatly decreases and the response time of behaviors greatly increases.
5. Performance bottlenecks on network devices:
This refers to devices such as firewalls, dynamic load balancers, and switches.
The causes of performance bottlenecks are complex and need to be located one by one. The preceding information is for reference only.
The resources
www.cnblogs.com/insane-Mr-L…
www.cnblogs.com/JcHome/p/10…
www.cnblogs.com/wyf0518/p/1…
NEXT
JMH Framework Based Performance Testing: Does Lambda Really Perform better than For?