Today, we take a look at a question that many people are concerned about: “Why is single-threaded Redis so fast?”
First of all, I would like to clarify a fact with you. We usually say that Redis is single-threaded, which mainly means that Redis network IO and key/value pair reads and writes are completed by one thread. This is also the main process of Redis to provide external key/value storage service. But other Redis functions, such as persistence, asynchronous deletion, and cluster data synchronization, are actually performed by additional threads.
So, strictly speaking, Redis is not single-threaded, but we generally refer to Redis as single-threaded high performance, which is “cool”. Next, I will also refer to Redis as single-threaded mode. Also, this prompts you to follow up with the question, “Why single threads? Why is a single thread so fast?”
To understand this, we need to take a closer look at Redis’s single-threaded design mechanism and multiplexing mechanism. Later when you tune Redis performance, you can more specifically avoid actions that would cause Redis to block on a single thread, such as executing complex commands.
So, without further ado, let’s take a look at why Redis is single-threaded.
Why does Redis use single threads?
To better understand why Redis uses single threads, we need to understand the overhead of multithreading.
Overhead of multithreading
When writing daily programs, we often hear a saying: “Using multithreading, can increase system throughput, or can increase system scalability.” Indeed, for a multithreaded system, with a reasonable allocation of resources, it is possible to increase the number of resource entities in the system to handle requests, thus increasing the number of simultaneous requests that the system can handle, that is, throughput. The left figure below is what we expect when we adopt multithreading.
But notice that often, when we do multithreading, if we don’t have a good system design, we actually get something like this. When we first increase the number of threads, throughput increases, but as we increase the number of threads further, throughput slows down and sometimes even drops.
Why does this happen? A key bottleneck is that there is often a shared resource, such as a shared data structure, that can be accessed simultaneously by multiple threads. When multiple threads modify the shared resource, additional mechanisms are required to ensure that the shared resource is correct, and this additional mechanism incurs additional overhead.
In the case of Redis, I mentioned in the last class that Redis has a List data type and provides LPOP and LPUSH operations. Assume that Redis has A multi-threaded design, as shown in the figure below. Now you have two threads, A and B. Thread A LPUSH A List and increments the queue length by one. At the same time, thread B performs LPOP on the List and subtracts the queue length by one. To ensure that the queue length is correct, Redis needs to serialize the LPUSH and LPOP of threads A and B so that Redis can record their changes to the List length without error. Otherwise, we might end up with the wrong length. This is the problem of concurrent access control of shared resources in multithreaded programming.
Has always been a multi-threaded concurrent access control is a difficult problem in the development, if there is no elaborate design, for example, simply use a coarse-grained mutex, can appear not ideal result: even if increased the threads, most of the thread is waiting for the mutex, get access to a Shared resource serial, parallel variable system throughput is not along with the increase with the increase of the thread.
Moreover, multithreaded development generally introduces synchronization primitives to protect concurrent access to shared resources, which also reduces the debuggability and maintainability of system code. To avoid these problems, Redis goes straight to single-threaded mode.
Now that you understand why Single-threaded Redis is used, let’s take a look at why single-threaded Redis achieves high performance.
Why is single-threaded Redis so fast?
In general, single-threaded processing power is much worse than multi-threaded processing power, but Redis can use the single-threaded model to achieve hundreds of thousands of processing power per second. Why? In fact, it’s a combination of Redis’s many design choices.
On the one hand, most of Redis operations are done in memory, coupled with its use of efficient data structures, such as hash tables and hop tables, which is an important reason for its high performance. On the other hand, Redis adopts the multiplexing mechanism, so that it can concurrently process a large number of client requests in network IO operations, and achieve high throughput. Next, we will focus on the multiplexing mechanism.
First, we need to understand the basic IO model of network operations and potential choke points. After all, Redis uses a single thread for IO, and if the thread is blocked, it can’t be multiplexed.
Basic IO model and choke points
Do you remember the network framework SimpleKV that I introduced in the first class?
In order to handle a Get request, SimpleKV needs to listen to the client (bind/listen), establish a connection with the client (accept), read the request from the socket (RECV), parse the request sent by the client (parse). Read the key value data (GET) according to the request type, and finally return the result to the client, namely write data back to the socket (send).
The following figure shows this process, where Bind/Listen, Accept, RECv, Parse, and Send are network IO processing, and GET are key-value data operations. Since Redis is single-threaded, the most basic implementation is to perform these operations in sequence in a single thread.
However, there are potential choke points in the network IO operation here, namely Accept () and recv(). When Redis listens for a connection request from a client, but fails to establish a connection, it blocks the accept() function, causing other clients to fail to establish a connection with Redis. Similarly, when Redis reads data from a client via recv(), Redis blocks at recv() if the data never arrives.
This causes the entire Redis thread to block and cannot process other client requests, which is inefficient. Fortunately, however, the socket network model itself supports non-blocking mode.
Non-blocking mode
The non-blocking mode setting of Socket network model is mainly reflected in three key function calls. If you want to use Socket non-blocking mode, you must understand the return type and setting mode of these three functions. Now, let’s focus on them.
In the Socket model, different operation calls return different socket types. The socket() method returns an active socket and calls listen() to convert the active socket to a listening socket, which listens for connection requests from clients. Finally, the accept() method is called to receive the incoming client connection and return the connected socket.
For listening sockets, we can set non-blocking mode: when Redis calls Accept () but no connection request ever arrives, the Redis thread can go back and process other operations instead of waiting forever. Note, however, that a listening socket already exists when accept() is called.
While the Redis thread does not have to continue to wait, there is always a mechanism to continue to wait for subsequent connection requests on the listening socket and notify Redis when a request is made.
Similarly, we can set non-blocking mode for connected sockets: after Redis calls recv(), if no data ever arrives on the connected socket, the Redis thread can also return to process other operations. We also need mechanisms to continue listening for the connected socket and notify Redis when data arrives.
This ensures that the Redis thread does not wait at a choke point, as in the basic IO model, nor does it cause Redis to fail to process connection requests or data that actually arrive.
Here comes the IO multiplexing mechanism in Linux.
High performance I/O model based on multiplexing
The IO multiplexing mechanism in Linux refers to the process of multiple I/O streams by a single thread, and is often referred to as the SELECT /epoll mechanism. In simple terms, this mechanism allows multiple listening sockets and connected sockets to exist simultaneously in the kernel, in the case that Redis only runs on a single thread. The kernel always listens for connection requests or data requests on these sockets. As soon as a request arrives, it is handed over to the Redis thread for processing, which implements the effect of one Redis thread processing multiple IO streams.
The following figure shows the Redis IO model based on multiplexing. The multiple FDS in the figure are just the same as the multiple sockets. The Redis network framework calls the epoll mechanism to have the kernel listen for these sockets. At this point, the Redis thread does not block on a particular listener or connected socket, that is, on a particular client request processing. Because of this, Redis can connect to multiple clients at the same time and process requests, increasing concurrency.
In order to notify the Redis thread when a request arrives, SELECT /epoll provides an event-based callback mechanism that calls the corresponding handler for different events.
So how does the callback mechanism work? In fact, select/epoll triggers an event whenever it detects that a request has arrived on the FD.
These events are put into an event queue, which is continuously processed by a single Redis thread. This way, Redis doesn’t have to keep polling to see if a request actually happened, which can avoid wasting CPU resources. At the same time, Redis invokes the corresponding handler function when processing the event in the event queue, which implements event-based callback. Because Redis is always processing event queues, it can respond to client requests in a timely manner, improving Redis response performance.
For your convenience, LET me explain the connection request and read data request as examples.
These two requests correspond to Accept and Read events, respectively, for which Redis registers Accept and GET callbacks. When the Linux kernel listens for a connection request or Read request, the Accept and Read events are triggered, and the kernel calls back the corresponding Accept and GET functions of Redis for processing.
It’s like a patient going to the hospital. Each patient (equivalent to a request) needs to be triaged, temperature checked, registered, etc., before the doctor can actually diagnose them. If all this work is done by doctors, doctors will be less efficient. As a result, hospitals have set up a triage desk that handles the pre-diagnosis work all the time (similar to the Linux kernel listening for requests) and then passes it on to the doctor for actual diagnosis. In this way, even a single doctor (equivalent to a single Redis thread) can improve efficiency.
However, it is important to note that the multiplexing mechanism applies even if different operating systems are deployed in your application scenario. Because there are many different implementations of this mechanism, including Select and Epoll implementations based on Linux, KQueue implementations based on FreeBSD, and Evport implementations based on Solaris, you can choose the appropriate multiplexing implementation based on the actual operating system running Redis.
summary
Today, we focused on three questions about Redis threads: “Is Redis really single-threaded?” “Why single thread?” “Why is a single thread so fast?”
Now, we know that Redis single threading refers to its network IO and data read and write operations using a single thread, and one of the core reasons for single threading is to avoid the concurrency control problems of multi-threaded development. Single-threaded Redis also achieves high performance, closely related to the multiplexed IO model, because it avoids the potential choke points of accept() and send()/ recV () network IO operations.
Figure that out, and you’re way ahead of a lot of people. If you have a friend who doesn’t know these questions, you are welcome to share them with him/her and solve their confusion.
In addition, AS you may have noticed, the stable version of Redis 6.0 was released in May 2020, which introduced a multi-threaded model. So, how does this multithreaded model relate to the IO model we talked about in this class? Will complex concurrency control issues be introduced? How much will Redis 6.0 improve? I’ll talk to you about these questions later in the course.
Each lesson asking
In this lesson, I have a quick question for you. Do you see any potential performance bottlenecks in the “Redis basic IO model” diagram? Welcome to write down your thoughts and answers in the comments area, and we will exchange and discuss together.