1. Introduction

Recently, when I was writing code in the company, I made a complicated design for a project because I was worried about the performance problems caused by the large amount of data processed by Redis. But I wonder, is Redis processing really time-consuming? According to Amdar’s law, we should find the point of time first. At present, the processing time of the interface is about 100ms-200ms. In order to find the most time-consuming link, I did a performance test on Redis.

Experiment 2.

2.1 environment

  1. Docker Redis
  2. Redis Tool is a benchmark pressure test Tool.
  3. Linux command:top.iftop(Top By Network Traffic)

Step 2.2

Redis has officially announced that THE QPS of Redis can reach 10W/s, and the performance bottleneck is generally the bandwidth of the network layer.

Therefore, we designed an experiment to verify this idea.

  • Local pressure test
  • LAN stress test

2.2.1 Pressure test of the machine

  1. We installed Redis on an Ubuntu host Docker with the default configuration.
  2. Enter the command in Terminal, the default is 50 concurrent clients, observe the working condition of the computer once, and wait for the end once
root@ubuntu:~# redis-benchmark -p 6379
Copy the code
  1. Computer working conditions:

hardware describe
CPU Us and SY both increased to 97%+. Due to Redis single thread, redis-server process will not break 100%
Network IO No flow
  1. Data and simple conclusions (see appendix at the bottom of this article for details)
indicators describe
QPS with O(1) time complexity 5W+, 99% of requests are returned within 1ms
Time complexity O(n) QPS (range query :range) 8K -1.5W+, 99% of requests are returned within 4ms

2.2.2 LAN Stress Test

  1. We installed Redis on an Ubuntu host Docker with the default configuration.
  2. On the LAN, send a stress test request to Ubuntu through a Mac. The LAN configuration is common home Settings, and the router is Mi Router 4
  3. Enter the command in Terminal, the default is 50 concurrent clients, observe the working condition of the computer once, and wait for the end once
➜ ~ redis-benchmark-h home.qpm.comCopy the code
  1. Computer operation

hardware describe
CPU Utilization rate is stable at 27%+
Network IO Traffic is monitored as shown in the figure above
  1. Data and simple conclusions (see appendix at the bottom of this article for details)
indicators describe
QPS with O(1) time complexity 8K +, 99% of requests are returned within 40ms, Total peak network traffic: 10-12MB
Time complexity O(n) QPS (range query :range) 1K-3K, 99% of requests are returned within 180ms, and the longest time is 1700ms
Network traffic, TPS in writing classes Server -> client about 3M/s
Network traffic, query in range class Server -> client bandwidth table shows 40M/s, but Mac says only 9M/s received

2.3 Test Conclusions

As Redis officials say, network bandwidth is often a bottleneck for Redis

It can be seen from the above data that the performance of Redis is completely untapped under a poor network configuration. In the case of network congestion, Redis performance jitter will be more obvious.

2.4 result

Since network bandwidth can be a bottleneck, Pipeline interaction can be prioritized in the case of heavy use of Redis requests. When a piece of code involves a for loop + Redis, it is easily the worst bottleneck for the system, and unless the bandwidth has caught up to the CPU’s processing speed (which is unlikely), pipelines can be very useful in providing efficiency.

Attention should be paid to the design of Redis data structure to avoid excessive data pulling or redundant data storage, which will also consume bandwidth and reduce the service performance of Redis middleware

3 reference

  1. Redis stress test tool Benchmark
  2. Network traffic testing tool ifTOP IFStats

Appendix 4

  • Unit pressure test report.log

  • Remote pressure test report.log