An overview,
As we all know, Redis is a high performance data storage framework. In the design of high concurrency system, Redis is also a relatively key component, which is a big weapon for us to improve system performance. It is more and more important to deeply understand the principle of high performance of Redis. Of course, high performance design of Redis is a systematic project, involving a lot of content. This paper focuses on IO model of Redis and thread model based on IO model.
We started with the origin of IO, talking about blocking IO, non-blocking IO, multiplexing IO. Based on multiplexing IO, we also sort out several different Reactor models and analyze their advantages and disadvantages. Based on Reactor model, we start the analysis of IO model and thread model of Redis, and summarize the advantages and disadvantages of Redis thread model, as well as the subsequent multi-thread model of Redis. The focus of this article is the Redis thread model design idea comb, smooth out the design idea, is all things.
Note: The code in this article is pseudo code, mainly for illustration, should not be used in production environment.
Ii. History of network IO model
We often say network IO model, mainly including blocking IO, non-blocking IO, multiplexing IO, signal driven IO, asynchronous IO, this paper focuses on Redis related content, so we focus on the analysis of blocking IO, non-blocking IO, multiplexing IO, to help you better understand the subsequent Redis network model.
Let’s look at the picture below;
2.1 blocking IO
We often say that blocking IO is actually divided into two kinds, one is single-thread blocking, one is multi-thread blocking. There are actually two concepts, blocking and threading.
Block: the current thread is suspended until the result of the call is returned. The calling thread will return only after the result is returned.
Threads: The number of threads for system calls.
Things like establishing a connection, reading, and writing all involve system calls and are themselves a blocking operation.
2.1.1 Single-thread Blocking
When a client request comes in, the server uses the main thread to handle the connection, read, write, and so on.
The following code simulates the single-thread blocking pattern;
import java.net.Socket;
public class BioTest {
public static void main(String[] args) throws IOException {
ServerSocket server=new ServerSocket(8081);
while(true) {
Socket socket=server.accept();
System.out.println("accept port:"+socket.getPort());
BufferedReader in=new BufferedReader(new InputStreamReader(socket.getInputStream()));
String inData=null;
try {
while((inData = in.readLine()) ! =null) {
System.out.println("client port:"+socket.getPort());
System.out.println("input data:"+inData);
if("close".equals(inData)) { socket.close(); }}}catch (IOException e) {
e.printStackTrace();
} finally {
try {
socket.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
}
Copy the codeWe are going to simulate the single-threaded blocking pattern with two clients making connection requests simultaneously. The main thread is blocked on the read method of the previous connection. The main thread is blocked on the read method of the previous connection.
We try to close the first connection and see what happens with the second one. What we want to see is that the main thread returns and the new client connection is accepted.
We can see from the log that after the first connection is closed, the second connection request is processed, that is, the second connection request is queued until the main thread wakes up to receive the next request, as we expected.
It’s not just, why?
The main reason is that the accept, read, and write functions are blocked. When the main thread is called by the system, the thread is blocked, and other clients cannot respond to the connection.
Through the above process, we can easily find the defects of this process. The server can only handle one connection request at a time, and the CPU is underutilized and the performance is low. How to take full advantage of the CPU’s multi-core features? It comes naturally to me — multithreaded logic.
2.1.2 Multi-thread Blocking
For engineers, the code explains everything, directly to the code.
BIO multithreading
package net.io.bio;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.ServerSocket;
import java.net.Socket;
public class BioTest {
public static void main(String[] args) throws IOException {
final ServerSocket server=new ServerSocket(8081);
while(true) {
new Thread(new Runnable() {
public void run(a) {
Socket socket=null;
try {
socket = server.accept();
System.out.println("accept port:"+socket.getPort());
BufferedReader in=new BufferedReader(new InputStreamReader(socket.getInputStream()));
String inData=null;
while((inData = in.readLine()) ! =null) {
System.out.println("client port:"+socket.getPort());
System.out.println("input data:"+inData);
if("close".equals(inData)) { socket.close(); }}}catch (IOException e) {
e.printStackTrace();
} finally{ } } }).start(); }}}Copy the codeAgain, we make two requests in parallel;
Both requests are accepted, and the server adds two new threads to handle the connection and subsequent requests from the client.
We use multithreading to solve, the problem of the server can handle only one request at the same time, but at the same time brings about a problem, if the client connection is long, the server will create a large number of threads to handle requests, but the thread itself is the consumption of resources, create, context switches are the consumption of resources, and how to solve?
2.2 a non-blocking
Would we reduce the number of threads on the server if we put all the sockets in a queue, rotated all the sockets in a single thread, and pulled them out when they were ready?
Let’s take a look at the code, pure non-blocking mode, we rarely use, to demonstrate the logic, we simulated the code as follows;
package net.io.bio;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.ServerSocket;
import java.net.Socket;
import java.net.SocketTimeoutException;
import java.util.ArrayList;
import java.util.List;
import org.apache.commons.collections4.CollectionUtils;
public class NioTest {
public static void main(String[] args) throws IOException {
final ServerSocket server=new ServerSocket(8082);
server.setSoTimeout(1000);
List<Socket> sockets=new ArrayList<Socket>();
while (true) {
Socket socket = null;
try {
socket = server.accept();
socket.setSoTimeout(500);
sockets.add(socket);
System.out.println("accept client port:"+socket.getPort());
} catch (SocketTimeoutException e) {
System.out.println("accept timeout");
}
// Simulate non-blocking: poll connected sockets, wait 10MS for each socket, process data if there is any, return no data, continue polling
if(CollectionUtils.isNotEmpty(sockets)) {
for(Socket socketTemp:sockets ) {
try {
BufferedReader in=new BufferedReader(new InputStreamReader(socketTemp.getInputStream()));
String inData=null;
while((inData = in.readLine()) ! =null) {
System.out.println("input data client port:"+socketTemp.getPort());
System.out.println("input data client port:"+socketTemp.getPort() +"data:"+inData);
if("close".equals(inData)) { socketTemp.close(); }}}catch (SocketTimeoutException e) {
System.out.println("input client loop"+socketTemp.getPort());
}
}
}
}
}
}
Copy the codeSystem initialization, waiting for connection;
Two client connections are initiated, and the thread polls the two connections for data.
After the two connections enter the data separately, the polling thread finds that the data is ready and begins the relevant logic processing (single-threaded or multi-threaded).
With a flow chart to help explain (the system actually uses the file handle, this time to replace Socket, convenient for everyone to understand).
The server has a thread that polls all sockets to confirm whether the operating system has completed the relevant event. If so, it will return the processing. If there is no further polling, let’s think about it together. And what’s the problem with that.
CPU idling, system calls (each polling involves a system call, using kernel commands to verify that data is ready), waste resources. Is there a mechanism to solve this problem?
2.3 IO multiplexing
There is no special thread on the server side to do polling operation (application side is not the kernel), but is triggered by events, when there is related to read, write, connection events, actively call up the server thread for related logic processing. The relevant codes are simulated as follows;
IO multiplexing
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.ServerSocketChannel;
import java.nio.channels.SocketChannel;
import java.nio.charset.Charset;
import java.util.Iterator;
import java.util.Set;
public class NioServer {
private static Charset charset = Charset.forName("UTF-8");
public static void main(String[] args) {
try {
Selector selector = Selector.open();
ServerSocketChannel chanel = ServerSocketChannel.open();
chanel.bind(new InetSocketAddress(8083));
chanel.configureBlocking(false);
chanel.register(selector, SelectionKey.OP_ACCEPT);
while (true) {int select = selector.select();
if(select == 0){
System.out.println("select loop");
continue;
}
System.out.println("os data ok");
Set<SelectionKey> selectionKeys = selector.selectedKeys();
Iterator<SelectionKey> iterator = selectionKeys.iterator();
while (iterator.hasNext()){
SelectionKey selectionKey = iterator.next();
if(selectionKey.isAcceptable()){
ServerSocketChannel server = (ServerSocketChannel)selectionKey.channel();
SocketChannel client = server.accept();
client.configureBlocking(false);
client.register(selector, SelectionKey.OP_READ);
// Continue to receive connection events
selectionKey.interestOps(SelectionKey.OP_ACCEPT);
}else if(selectionKey.isReadable()){
/ / get a SocketChannel
SocketChannel client = (SocketChannel)selectionKey.channel();
// Define the buffer
ByteBuffer buffer = ByteBuffer.allocate(1024);
StringBuilder content = new StringBuilder();
while (client.read(buffer) > 0){
buffer.flip();
content.append(charset.decode(buffer));
}
System.out.println("client port:"+client.getRemoteAddress().toString()+",input data: "+content.toString());
// Clear the bufferbuffer.clear(); } iterator.remove(); }}}catch(Exception e) { e.printStackTrace(); }}}Copy the codeCreate two connections simultaneously;
Two connections are created without blocking;
Non-blocking receive read and write;
With a flow chart to help explain (the system actually uses the file handle, this time to replace Socket, convenient for everyone to understand).
Multiplexing of the operating system, of course, there are several kinds of way to accomplish this, we often use the select (), epoll patterns here don’t do too much explanation, interested can look at the related documents, the development of the IO back and asynchronous, events, such as pattern, we are here but more detail, we are more in order to explain the development of Redis thread model.
NIO thread model interpretation
We talked about blocking, non-blocking, IO multiplexing, which is Redis?
Redis uses THE IO multiplexing model, so we will focus on how to better implement the multiplexing model into our system. Inevitably, we will talk about the Reactor model.
First of all, let’s do some related nouns;
Reactor: similar to the Selector in NIO programming, responsible for I/O event distribution;
Acceptor: the branch logic in NIO that processes a connection after receiving an event;
Handler: Operations such as message read and write processing.
3.1 Single-reactor single-thread model
Processing flow
- The Reactor listens for connection events and Socket events and sends them to acceptors for connection events and to handlers for Socket events.
advantages
-
The model is relatively simple, all the processing is in one connection;
-
The Reactor is responsible for multiplexing and event distribution, Acceptor is responsible for connection events, and Handler is responsible for Scoket read and write events.
disadvantages
-
With only one thread, connection processing and business processing share the same thread and cannot take full advantage of the multi-core CPU.
-
The system performs well when the traffic is not heavy and service processing is fast. However, when the traffic is heavy and I/O events are time-consuming, performance bottlenecks may occur.
How to solve the above problems? Since the business processing logic may affect the system bottleneck, can we separate the business processing logic and hand it over to the thread pool to deal with, on the one hand, reduce the impact on the main thread, on the other hand, make use of the advantages of CPU multi-core. This point we hope to understand thoroughly, convenient for our subsequent understanding of Redis from single-thread model to multi-thread model design ideas.
3.2 Single-reactor multithreaded model
Compared with single-reactor single-thread model, this model just transfers the processing logic of business logic to a thread pool.
Processing flow
-
The Reactor listens for connection events and Socket events and sends them to acceptors for connection events and to handlers for Socket events.
-
After the Handler completes the read event, it wraps it into a task object and hands it off to the thread pool for processing, handing the business processing logic to other threads for processing.
advantages
-
Let the main thread focus on the processing of common events (connect, read, write), further decoupled by design;
-
Take advantage of CPU multi-core.
disadvantages
- It seems that this model is perfect, but if there are many clients and traffic is very heavy, the processing of common events (read and write) can also become the bottleneck of the main thread, because each read and write operation involves system calls.
Is there any good way to solve the above problems? Through the above analysis, have you found a phenomenon, when a point becomes the bottleneck point of the system, try to take it out and hand over to other threads to deal with, then this kind of scene is applicable?
3.3 Multi-reactor multi-thread model
Compared with the single-reactor and multi-thread model, this model only takes the read and write processing of the Scoket out of the mainReactor and gives it to the subReactor thread.
Processing flow
-
The mainReactor main process is responsible for monitoring and processing connection events. When an Acceptor completes the connection process, the main process assigns the connection to the subReactor.
-
The subReactor is responsible for monitoring and processing the Socket assigned by the mainReactor. When a Socket event comes, it is handed to the corresponding Handler for processing.
After the Handler completes the read event, it wraps it into a task object and hands it off to the thread pool for processing, handing the business processing logic to other threads for processing.
advantages
-
Let the main thread focus on connection events processing, child threads focus on read and write events blowing, further decoupled from the design;
-
Take advantage of CPU multi-core.
disadvantages
- The implementation can be complex and can be considered in scenarios where single-machine performance is extremely important.
Fourth, Redis thread model
4.1 an overview of the
We talked about the history of the IO network model and the REACTOR model for IO multiplexing. Which reactor model does Redis use? Before we answer that question, let’s comb through a few conceptual questions.
There are two types of events in the Redis server, file events and time events.
File events: The file can be interpreted as socket-related events, such as connection, read, write, etc.
Time time: Can be understood as scheduled task events, such as some periodic RDB persistent operations.
This article focuses on socket-related events.
Figure 4.2 model
First let’s look at the thread model diagram of Redis service.
IO multiplexing is responsible for monitoring each event (connection, read, write, etc.). When an event occurs, the corresponding event is put into a queue, and the event distributor distributes the event according to the event type.
If it is a connection event, it is dispatched to the connection reply handler; Redis commands such as GET and SET are distributed to the command request processor.
After the command is processed, the command reply event is generated, and then the event queue, the event dispatcher, the command reply handler, and the client response are generated.
4.3 Interaction between a client and a server
4.3.1 Connection Process
The connection process
-
The Redis server main thread listens on the fixed port and binds the connection event to the connection reply handler.
-
After the client initiates a connection, the connection event is triggered, and the IO multiplexer wraps the connection event into an event queue, which is then distributed by the event dispatch handler to the connection reply handler.
-
The connection reply handler creates the client object as well as the Socket object. We will focus on the Socket object here and generate ae_readable events, which are associated with the command handler to indicate that the Socket is interested in subsequent readable events, that is, to start receiving commands from the client.
-
The current process is handled by a single main thread.
4.3.2 Command Execution Process
SET Command execution process
-
The client initiates the SET command, the IO multiplexer listens to the event (read event), wraps the data as an event and throws it into the event queue (event is bound to the command request handler in the previous process);
-
The event distribution handler dispatches the event to the corresponding command request handler according to the event type.
-
Command request processor, read data in Socket, execute command, and then generate AE_writable event, and bind command reply processor;
-
After the IO multiplexer detects the write event, the data is packaged as an event and thrown into the event queue. The event distribution processor sends the data to the command response processor according to the event type.
-
The command replies to the processor, writing data to the Socket and returning it to the client.
4.4 Advantages and disadvantages of the model
From the above process analysis, we can see that Redis adopts the single-thread Reactor model. We also analyzed the advantages and disadvantages of this model. Then why does Redis adopt this model?
Redis itself
Command execution is based on memory operations and business processing logic is fast, so command processing can be done in a single thread to maintain a high performance.
advantages
- The benefits of Reactor’s single-threaded model, see above.
disadvantages
-
The disadvantage of Reactor single thread model is also reflected in Redis, the only difference is that business logic processing (command execution) is not the bottleneck point of the system.
-
As traffic increases, IO operations become increasingly time-consuming (read operations, which read data from the kernel to applications). Write operation, data in the application to the kernel), the bottleneck of the system becomes apparent when a certain threshold is reached.
How does Redis solve it?
Ha ha ~ take the time point out from the main thread? Is that what the new version of Redis does? Let’s take a look.
4.5 Redis multi-threaded mode
The multi-threading model of Redis is a little different from “multiple Reactor multi-threading model” and “single Reactor multi-threading model”, but it uses the ideas of both models as follows.
The multi-thread model of Redis is to multithread IO operation, its logic processing process (command execution process) is still single thread, with the help of the single Reactor idea, and different in implementation.
I/O multithreading is consistent with the idea of a single Reactor derived from a multiple Reactor, which is to separate THE I/O operation from the main thread.
Command execution process
-
The client sends a request command, triggering a read-ready event. The main thread of the server puts the Socket (in order to simplify the understanding cost, the Socket is used to represent the connection) into a queue. The main thread is not responsible for reading;
-
The IO thread reads the client request command through the Socket. The main thread is busy polling and waiting for all I/O threads to complete the reading task. The IO thread is only responsible for reading but not executing the command.
-
The main thread executes all the commands at once, just like a single thread, and then puts the returned connection on another queue, which the IO thread writes out (as the main thread does).
-
The main thread is busy polling, waiting for all I/O threads to finish writing.
Five, the summary
To understand a component, more is to understand his design ideas, to think about why to do this design, what is the background of this technology selection, what is the reference significance for the subsequent system architecture design and so on. Everything, I hope you have reference significance.
Vivo Internet Server Team -Wang Shaodong