Xiao Lin, coming.

This time, I’ll illustrate Reactor and Proactor, two high-performance network patterns.

Don’t underestimate these two things, especially Reactor model. Many common open source software in the market adopt this scheme, such as Redis, Nginx, Netty and so on. Therefore, learning the design idea of this model is not only helpful for us to understand many open source software, but also can be used in the interview.

Start!

evolution

If you want a server to serve multiple clients, the most straightforward way is to create threads for each connection.

In fact, it is possible to create a process, the principle is the same, the difference between a process and a thread is that the thread is lighter, the cost of creating a thread and switching between threads is smaller, in order to describe the brief, the following are threads as an example.

After processing the business logic, threads will also be destroyed when the connection is closed. However, the continuous creation and destruction of threads will not only incur performance costs, but also waste resources, and it is not realistic to create tens of thousands of threads to cope with tens of thousands of connections.

How to solve the problem? We can use the “resource reuse” approach.

Instead of creating a thread for each connection, you create a “thread pool,” assigning connections to threads, and then one thread can handle business for multiple connections.

However, this raises a new question: how can threads efficiently handle multiple connections?

When a connection is connected to a thread, the thread uses the “read -> Business processing -> send” process. If there is no data to read, the thread blocks the READ operation (socket by default blocks I/O), but this blocking method does not affect other threads.

However, with the introduction of thread pools, a thread has to process multiple connections. If a thread has no data to read from one connection, it blocks, and the thread cannot continue to process other connections.

The simplest way to solve this problem is to make the socket non-blocking and then poll the thread to see if it has data. This would solve the blocking problem, but it would do it in a crude way because polling is cpu-intensive. And the more connections a thread processes, the less efficient polling becomes.

The problem is that the thread does not know if there is any data to read from the current connection, so it needs to be tested each time with read.

Is there a way for a thread to initiate a read request only when there is data on the connection? The answer is yes, it is I/O multiplexing.

I/O multiplexing uses a system call function to listen for all the connections we care about, which means you can monitor many connections in a single monitor thread.

The familiar select/poll/epoll is the multiplex system call provided by the kernel to the user state. Threads can fetch multiple events from the kernel through a single system call function.

PS: If you want to know the difference between SELECT /poll/epoll, you can check out this article written by Kobayashi: Promise me this time, I/O multiplexing!

How does select/poll/epoll get network events?

To get an event, we first pass the connection we care about to the kernel, which then checks it:

  • If no event occurs, the thread simply blocks on the system call, rather than training to call the read operation to determine whether there is data, as in the previous thread pool scenario.
  • If an event occurs, the kernel will return the connection that generated the event, the thread will return from the blocked state, and then process the connection in the user state.

Is I/O multiplexing the reason why open source software can perform well on networks today?

Yes, it is basically based on I/O multiplexing. If you have used I/O multiplexing interface to write network programs, you must know that you write code in a process-oriented way. The efficiency of such development is not high.

Therefore, based on the idea of object-oriented, the I/O multiplexing layer encapsulation, so that users do not have to consider the details of the underlying network API, only need to focus on the writing of application code.

They gave this model a name that was initially hard to understand: the Reactor Model.

Reactor translates to “Reactor”, which you might think of as a nuclear Reactor in physics, but actually it doesn’t.

This is a reaction to an event, which means that once an event occurs, the Reactor has a reaction/response.

In fact, the Reactor pattern is also called the Dispatcher pattern, which I think is more appropriate. It is the I/O multiplexing pattern that listens for events, receives them, and dispatches them to a process/thread based on the event type.

The Reactor model is mainly composed of Reactor and processing resource pool, which are responsible for the following:

  • Reactor is responsible for monitoring and distributing events, including connection events, read and write events.
  • The processing resource pool handles events such as read -> Business Logic -> send;

The Reactor schema is flexible and can respond to different business scenarios. Flexibility lies in:

  • There can be only one or more reactors.
  • The processing resource pool can be a single process/thread or multiple processes/threads;

By arranging the above two factors, there are theoretically four options:

  • Single Reactor Single process/thread;
  • Single Reactor multiple processes/threads;
  • Multiple Reactor single process/thread;
  • Multiple Reactor multiple processes/threads;

Among them, “multiple Reactor single process/thread” implementation scheme compared with “single Reactor single process/thread” scheme is not only complex and has no performance advantages, so it is not applied in practice.

The remaining three schemes are all classic and have been applied in actual projects:

  • Single Reactor Single process/thread;
  • Single Reactor multithreading/process;
  • Multiple Reactor multiple processes/threads;

The solution uses processes or threads, depending on the programming language and platform used:

  • The Java language generally uses threads, such as Netty;
  • C uses both processes and threads. For example, Nginx uses processes and Memcache uses threads.

Next, three classic Reactor schemes are introduced respectively.

Reactor

Single Reactor Single process/thread

Generally speaking, C language implements the “single Reactor and single process” scheme, because the program written in C language is an independent process after running, and there is no need to create threads in the process.

The Java language implements a single-reactor single-thread scheme, because the Java program runs on the Process of the Java VIRTUAL machine. There are many threads in the virtual machine, and the Java program we write is just one thread.

Let’s look at a schematic of a single Reactor and process:

Reactor (Acceptor); Handler (Reactor);

  • The Reactor object listens for and distributes events.
  • Acceptor objects are used to obtain connections.
  • The Handler object is used to process business;

Select, Accept, Read, and Send are system call functions. Dispatch and “business processing” are operations that need to be completed. Dispatch is an operation that distributes events.

Next, I will introduce the single Reactor and single process:

  • The Reactor object listens for events through SELECT (IO multiplexing interface) and dispatches events to Acceptors or To Handlers, depending on the type of events received.
  • If it is a connection establishment event, an Acceptor receives the connection through the Accept method and creates a Handler object to handle subsequent response events.
  • If it is not a connection establishment event, the Handler object corresponding to the current connection will respond.
  • The Handler object completes the complete business process through the read -> Business Process -> Send process.

Because all the work is completed in the same process, the single-reactor single-process scheme is relatively simple to implement, and there is no need to consider inter-process communication and multi-process competition.

However, this scheme has two disadvantages:

  • The first disadvantage is that there is only one process, which cannot take full advantage of the performance of a multi-core CPU.
  • The second disadvantage is that the Handler object cannot process the events of other connections during business processing. If the business processing takes a long time, the response will be delayed.

Therefore, the single-reactor single-process solution is not suitable for computer intensive scenarios, but only for scenarios where business processes are very fast.

Redis is implemented by C language, which adopts the program of “single Reactor and single process”. Because the business processing of Redis is mainly completed in memory, the operation speed is very fast, and the performance bottleneck is not on CPU, so the processing of Redis command is a single process program.

Single Reactor Multithreaded/multiprocess

To overcome the disadvantages of the single-reactor single-thread/process solution, we need to introduce multi-threading/multi-process solution, which gives rise to the single-reactor multi-threading/multi-process solution.

Let’s take a look at the schematic diagram of a single Reactor with multiple threads:

Tell me more about the plan:

  • The Reactor object listens for events through SELECT (IO multiplexing interface) and dispatches events to Acceptors or To Handlers, depending on the type of events received.
  • If it is a connection establishment event, an Acceptor receives the connection through the Accept method and creates a Handler object to handle subsequent response events.
  • If it is not a connection establishment event, the Handler object corresponding to the current connection will respond.

The above three steps are the same for a single-reactor single-thread scenario, but the next steps are different:

  • The Handler is only responsible for receiving and sending data. After reading the data, the Handler sends the data to the Processor in the child thread for service processing.
  • The Processor in the child thread processes services. After the processing is complete, the Processor sends the result to the Handler in the main thread. The Handler then sends the response result to the client through the send method.

The advantage of single-reator multi-threading scheme is that it can make full use of multi-core CPU. Since multi-threading is introduced, it naturally brings the problem of multi-threading competing for resources.

For example, when a child thread completes a business process, it passes the result to the Reactor for the main thread to send, which involves competing to share data.

In order to avoid data disorder caused by multiple threads competing for shared resources, mutex should be added before the operation of shared resources to ensure that only one thread is operating the shared resources at any time. After the thread finishes the operation and releases the mutex, other threads have the opportunity to operate the shared data.

Following the single-reactor multithreaded solution, let’s look at the single-reactor multiprocess solution.

In fact, a single Reactor with multiple processes is more difficult to implement than a single Reactor with multiple threads, mainly because of the bidirectional communication between the child process and the parent process, and the parent process knows which client the child process is sending data to.

However, data can be shared between multiple threads. Although concurrency needs additional consideration, it is much lower than the complexity of inter-process communication, so there is no pattern of single Reactor and multiple processes in practical application.

Another problem with the single-reactor model is that a single Reactor object listens to and responds to all events and runs only on the main thread, which can be a performance bottleneck in the face of instantaneous high concurrency scenarios.

Multiple Reactor Multiple processes/threads

The solution to the “single Reactor” problem is to implement the “single Reactor” into “multiple reactors”, resulting in the multi-reactor multi-process/threading scheme.

As always, the picture is better than the name. A multi-reactor multi-process/thread scheme is shown below (using threads as an example) :

The scheme is described as follows:

  • The MainReactor object in the main thread monitors the connection establishment event by SELECT. After receiving the event, the connection is obtained by Accept in the Acceptor object, and the new connection is allocated to a child thread.
  • The SubReactor object in the child thread adds the connection assigned by the MainReactor object to select to continue listening and creates a Handler to handle the connection’s response events.
  • If a new event occurs, the SubReactor object responds by calling the Handler object corresponding to the current connection.
  • The Handler object completes the complete business process through the read -> Business Process -> Send process.

Although the multi-reactor and multi-threaded program seems complicated, it is much simpler than the single-reactor and multi-threaded program in practice, for the following reasons:

  • The main thread and child threads have a clear division of labor. The main thread is only responsible for receiving new connections, and the child thread is responsible for completing subsequent business processing.
  • The interaction between the main thread and the child thread is very simple, the main thread only needs to pass the new connection to the child thread, the child thread does not need to return data, directly send the result of the child thread to the client.

Netty and Memcache, two well-known open source software, both adopt the “multiple Reactor and multiple threads” scheme.

The open source software that uses the “multiple Reactor multiple process” scheme is Nginx, although there are some differences from the standard multiple Reactor multiple process scheme.

The specific difference is that the main process is only used to initialize the socket. Instead of creating a mainReactor to accept connections, the Reactor of the child process accepts connections. Only one child process accepts connections through the lock (to prevent the shock phenomenon). The child process accepts a new connection and stores it in its own Reactor for processing, not assigning it to other children.

Proactor

The previously mentioned Reactor is a non-blocking synchronous network pattern, while Proactor is an asynchronous network pattern.

This is to review the concepts of blocking, non-blocking, synchronous, and asynchronous I/O.

Let’s start with blocking I/O. When a user program executes read, the thread blocks until kernel data is ready and copied from the kernel buffer to the application buffer. When the copying process is complete, read returns.

Note that blocking waits for kernel data to be ready and data to be copied from kernel to user state. The process is shown as follows:

Now that you know about blocking I/O, look at non-blocking I/O. A non-blocking READ request returns immediately if the data is not ready, and you can proceed. The application keeps polling the kernel until the data is ready, and the kernel copies the data to the application buffer, and the read call doesn’t get a result. The process is shown as follows:

Notice that the last read call here, the process of getting the data, is a synchronous process, a waiting process. Synchronization refers to the process of copying kernel-state data into the user program’s cache.

For example, if the socket is set with the O_NONBLOCK flag, it is using non-blocking I/O access, which is blocked by default if nothing is set.

Therefore, whether read and send are blocking I/O or non-blocking I/O are synchronous calls. This is because the kernel waits to copy data from kernel space to user space, meaning that the process is synchronous. If the kernel does not copy efficiently, the read call will wait a long time during the synchronization process.

True asynchronous I/O is “kernel data ready” and “data copied from kernel state to user state” without waiting.

When we initiate AIO_READ (asynchronous I/O), we immediately return, and the kernel automatically copies data from kernel space to user space. The copying process is also asynchronous, and the kernel does this automatically. Unlike the previous synchronization, the application does not actively initiate the copy. The process is shown as follows:

For example, if you go to a canteen, you are like an app, and the canteen is like an operating system.

Blocking I/O is like, you go to the dining hall meal, but the canteen’s food is not ready, then you have wait and wait there, waiting for a long time, finally wait until the dining hall aunt took out of the dishes () the process of data preparation, but you still have to continue to wait for aunt beat food (kernel space) to your lunch box (user space), experience the process of these two, So you can leave.

For example, if you go to the canteen and ask your aunt if the dishes are ready, and your aunt tells you if they are ready, you leave. After a few minutes, you come to the canteen and ask your aunt, and your aunt says that the dishes are ready, so your aunt helps you to put the dishes into your lunch box, and you have to wait for this process.

Asynchronous I/O is like asking your lunch lady to cook the food and bring it to you without waiting for anything.

Obviously, asynchronous I/O performs better than synchronous I/O because asynchronous I/O does not have to wait for either kernel data to be ready or data to be copied from kernel space to user space.

Proactor uses asynchronous I/O technology, so it is called the asynchronous network model.

Now let’s understand the difference between Reactor and Proactor.

  • Reactor is a non-blocking synchronous network that senses ready-to-read events. In each perceive events (such as readable ready events), you need to take the initiative to call the read method to complete the application process of data read, also is the process to apply active socket to receive the data in the cache memory, read the application process of this process are synchronous, after reading the data process of application can process the data.
  • Proactor is an asynchronous network mode that is aware of completed read and write events. In the asynchronous read and write request, need to pass in the address of the data buffer (used to store the result data) and other information, so that the system kernel can automatically help us to complete the data read and write work, here the reading and write work is done by the operating system. Unlike Reactor, the application process is not required to initiate read/write to read and write data. Once the operating system finishes reading and writing, the application process is notified to process the data directly.

Therefore, Reactor can be understood as “event operating system notifying application process, and application process to deal with”, while Proactor can be understood as “event operating system to deal with, and then notify application process after processing”. In this case, “events” are I/O events with new connections, data to read, and data to write. In this case, “processing” includes reads from the driver to the kernel and reads from the kernel to user space.

For a real-life example, the Reactor model is a Courier downstairs who calls you and says it’s coming to your neighborhood and you need to come downstairs to get it. In Proactor, the Courier delivers the package directly to your door and notifies you.

Both Reactor and Proactor are network programming patterns based on “event distribution”. The difference lies in that Reactor pattern is based on “to be completed” I/O events, while Proactor pattern is based on “completed” I/O events.

Next, take a look at a schematic of the Proactor schema:

Here’s how the Proactor mode works:

  • Proactor Initiator is responsible for creating Proactor and Handler objects and passing them through

Asynchronous Operation Processor is registered with the kernel;

  • Asynchronous Operation Processor is responsible for processing registration requests and I/O operations;
  • Asynchronous Operation Processor notifies Proactor when the I/O Operation is complete.
  • Proactor calls back different handlers according to different event types for business processing;
  • Handler Completes service processing.

Unfortunately, asynchronous I/O in Linux is not perfect. Aio functions are asynchronous operation interfaces defined by POSIX, which are not supported at the operating system level. Instead, they simulate asynchronous operations in user space, and only support asynchronous AIO operations based on local files. Sockets are not supported in network programming, which leads to high performance Linux-based network applications using the Reactor scheme.

Windows implements a complete set of asynchronous programming interfaces that support sockets. This interface is called IOCP, which is asynchronous I/O implemented at the operating system level. In a real sense, asynchronous I/O. Therefore, the implementation of high performance network programs in Windows can use a more efficient Proactor solution.

conclusion

There are three common Reactor implementations.

The first scheme single Reactor process/thread, need not consider interprocess communication and data synchronization problems, therefore to implement is simpler, the defect of this scheme is unable to make full use of multi-core CPU, and deal with the business logic can’t be too long, otherwise it will delay the response, so does not apply to computer intensive scenario, It is suitable for fast business processing scenarios. For example, Redis uses a single Reactor and a single process.

Number two single Reactor multi-threading, multithreaded way to solve the defects of the scheme one but it is high concurrency is nearly distance, difference in only one Reactor object surveillance and response to bear all the events, and only in the main thread, in the face of instantaneous high concurrency scenario, easy to become the bottleneck of the performance.

The third scheme is multi-reactor and multi-process/thread, which solves the defects of scheme 2 by using multiple reactors. The main Reactor is only responsible for monitoring events, while the secondary reactors are responsible for responding to events. Netty and Memcache both adopt the “multiple reactors and multiple threads” scheme, while Nginx adopts the similar “multiple reactors and multiple processes” scheme.

Reactor can be understood as “the event operating system informs the application process, and the application process deals with it”, while Proactor can be understood as “the event operating system deals with it, and then notifying the application process”.

Therefore, the real killer is Proactor, which is an asynchronous network model based on asynchronous I/O that senses read and write events rather than the need for a Reactor to call read to retrieve data from the kernel.

However, both Reactor and Proactor are network programming patterns based on “event distribution”. The difference lies in that Reactor pattern is based on “to be completed” I/O events, while Proactor pattern is based on “completed” I/O events.

The resources

Cloud.tencent.com/developer/a…

Blog.csdn.net/qq_27788177…

Time.geekbang.org/column/arti…

www.cnblogs.com/crazymakerc…