Analysis of high performance IO model
Server-side programming often needs to construct high-performance IO models. There are four common IO models:
(1) Blocking IO: the traditional IO model.
(2) Synchronizing non-blocking IO: All created sockets are blocking by default. Non-blocking IO requires that the socket be set to NONBLOCK. Note that NIO is not the NIO (New IO) library for Java.
(3) IO Multiplexing: the classic Reactor design pattern, sometimes called asynchronous blocking IO, Selector in Java and Epoll in Linux are both this model.
Asynchronous IO is a classic Proactor design pattern, also known as Asynchronous non-blocking IO.
The concepts of synchronization and asynchronism describe the interaction between user threads and the kernel. Synchronization means that user threads need to wait or poll the kernel for the COMPLETION of THE I/O operation after initiating AN I/O request. Asynchronous means that the user thread continues to execute the I/O request. When the kernel completes the I/O operation, the user thread will be notified or the callback function registered by the user thread will be called.
The concepts of blocking and non-blocking describe how user threads invoke kernel I/O operations: blocking means that the I/O operation is not returned to user space until it has completely completed; Non-blocking means that the STATUS value is returned to the user immediately after the IO operation is invoked, without waiting for the IO operation to complete.
In addition, the Signal Driven IO (IO) model mentioned by Richard Stevens in Unix Network Programming Volume 1 is not covered in this article because it is not commonly used. Next, we analyze the implementation principles of four common IO models in detail. For ease of description, we use the read operation of IO as an example.
Block I/O synchronously
The synchronous blocking IO model is the simplest IO model in which user threads are blocked while the kernel is performing IO operations.
Figure 1 Synchronous blocking IO
As shown in Figure 1, the user thread initiates an IO read through the system call read, moving from user space to kernel space. The kernel waits until the packet arrives, then copies the received data into user space to complete the read operation.
Pseudocode for user threads using the synchronous blocking IO model is described as follows:
{
read(socket, buffer);
process(buffer);
}
Copy the codeThe user waits for read to read data from the socket into the buffer before processing the received data. During the I/O request process, the user thread is blocked. As a result, the user cannot do anything during the I/O request, resulting in insufficient CPU utilization.
2. Synchronize non-blocking IO
Synchronizing non-blocking I/OS sets the socket to NONBLOCK on the basis of synchronizing blocking I/OS. This allows the user thread to return immediately after making an IO request.
Figure 2 Synchronizing non-blocking IO
As shown in Figure 2, since the socket is non-blocking, the user thread immediately returns an I/O request. However, no data is read. The user thread continuously initiates I/O requests until the data arrives and then reads the data to continue the execution.
Pseudocode for user threads using the synchronous non-blocking IO model is described as follows:
{
while(read(socket, buffer) ! = SUCCESS) ; process(buffer); }Copy the codeThat is, the user repeatedly calls read, attempting to read data from the socket, and does not continue processing the received data until the read succeeds. In the whole I/O request process, although the user thread can immediately return after each I/O request, it still needs to continuously poll and repeat the request to wait for data, consuming a large amount of CPU resources. This model is rarely used directly, but the non-blocking IO feature is used in other IO models.
IO multiplexing
IO multiplexing model is based on the kernel-provided multiplexing function SELECT, which can avoid the problem of polling wait in synchronous non-blocking IO model.
Figure 3. Multiway separation function SELECT
As shown in Figure 3, the user first adds the socket for IO operations to the SELECT, then blocks and waits for the SELECT system call to return. When the data arrives, the socket is activated and the select function returns. The user thread formally initiates a read request, reads the data, and continues execution.
In terms of flow, the USE of SELECT for IO requests is not much different from the synchronous blocking model, and even more inefficient with the additional operations of adding monitoring sockets and calling select. However, the biggest advantage of using SELECT is that the user can process I/O requests from multiple sockets simultaneously in a single thread. The user can register multiple sockets, and then continuously call select to read the activated socket, to achieve the purpose of processing multiple I/O requests in the same thread. In the synchronous blocking model, multithreading is necessary to achieve this goal.
The pseudocode for the user thread using the select function is described as follows:
{
select(socket);
while(1)
{
sockets = select();
for(socket in sockets)
{
if(can_read(socket)) { read(socket, buffer); process(buffer); }}}}Copy the codeThe while loop adds the socket to the SELECT monitor, and then calls select all the way through the while to get the activated socket. Once the socket is readable, the read function is called to read the data from the socket.
However, the advantages of using the SELECT function do not end there. Although the above approach allows multiple I/O requests to be processed in a single thread, the process of each I/O request is still blocked (on the SELECT function), and the average time is even longer than the synchronous blocking IO model. CPU utilization can be improved if the user thread registers only the socket or IO requests it is interested in, then goes about its business and waits until the data arrives to process them.
The IO multiplexing model uses the Reactor design pattern to implement this mechanism.
Figure 4. Reactor Design pattern
As shown in figure 4, the EventHandler abstract class represents an IO EventHandler, which has the IO file Handle Handle (obtained via get_handle) and the operations on Handle handle_event (read/write, etc.). Subclasses that inherit from EventHandler can customize the behavior of event handlers. The Reactor class is used to manage EventHandler (register, delete, etc.) and implement an event loop using HANDLE_Events, which continuously calls the select function of the synchronous event multiplexer (usually the kernel) whenever a file handle is activated (read/write, etc.). Select returns (blocks), and HANDLE_EVENTS calls handLE_EVENT of the event handler associated with the file handle.
Figure 5 IO multiplexing
As shown in Figure 5, Reactor can uniformly transfer the polling of user threads for IO operation status to the HANDLE_Events event loop for processing. The user thread registers the event handler and then proceeds to do other work (asynchronously), while the Reactor thread calls the kernel’s SELECT function to check the socket status. When a socket is activated, the corresponding user thread is notified (or the callback function of the user thread is executed) and handLE_EVENT is executed to read and process data. Because the SELECT function blocks, the multiplex IO multiplexing model is also known as the asynchronous blocking IO model. Note that blocking refers to the thread being blocked when the select function is executed, not the socket. In the IO multiplexing model, sockets are set to NONBLOCK, but this does not matter because when the user initiates an I/O request, the data has already arrived and the user thread will not be blocked.
The pseudo-code for user threads using the IO multiplexing model is described as follows:
void UserEventHandler::handle_event()
{
if(can_read(socket))
{
read(socket, buffer);
process(buffer);
}
}
{
Reactor.register(new UserEventHandler(socket));
}
Copy the codeThe user needs to rewrite the HANDLE_EVENT function of EventHandler to read and process data. The user thread only needs to register its EventHandler with Reactor. The pseudo-code for the HANDLE_Events event loop in Reactor is roughly as follows.
Reactor::handle_events()
{
while(1)
{
sockets = select();
for(socket in sockets) { get_event_handler(socket).handle_event(); }}}Copy the codeThe event loop continuously calls SELECT to retrieve the activated socket, and then executes handle_event according to the EventHandler corresponding to the socket.
IO multiplexing is the most commonly used IO model, but it is not asynchronous enough because it uses a select system call that blocks threads. Therefore, IO multiplexing can only be called asynchronous blocking IO, not true asynchronous IO.
4. Asynchronous I/O
“True” asynchronous IO requires stronger support from the operating system. In the IO multiplexing model, the event loop notifies the user thread of the status event of the file handle, and the user thread reads and processes the data by itself. In the asynchronous IO model, when the user thread receives the notification, the data has been read by the kernel and placed in the buffer specified by the user thread. The kernel notifies the user thread to use the data directly after I/O completion.
The asynchronous IO model implements this mechanism using the Proactor design pattern.
Figure 6. Proactor design pattern
As shown in FIG. 6, Proactor mode and Reactor mode are similar in structure, but differ greatly in the use mode of Client. In the Reactor pattern, the user thread listens by registering an event of interest with the Reactor object, and then calls an event handler when the event is triggered. But Proactor pattern, user threads will AsynchronousOperation (read/write, etc.), Proactor and CompletionHandler registered to AsynchronousOperationProcessor operation is complete. AsynchronousOperationProcessor using the Facade pattern provides a set of asynchronous operations API (read/write, etc.) for the use of the user, when a user thread calls asynchronous API, then continue to perform their tasks. AsynchronousOperationProcessor opens independent kernel threads execute asynchronous operations, true asynchronous. When the asynchronous I/o operation is complete, AsynchronousOperationProcessor will registered user threads with AsynchronousOperation Proactor and CompletionHandler, The CompletionHandler is then forwarded along with the result data of the IO operation to Proactor, which is responsible for calling back the event CompletionHandler handle_event for each asynchronous operation. While each asynchronous operation in the Proactor pattern can be bound to a Proactor object, proActors are generally implemented in the Singleton pattern in operating systems to facilitate centralized distribution of operation completion events.
Figure 7 Asynchronous IO
As shown in Figure 7, in the asynchronous IO model, the user thread initiates a READ request directly using the asynchronous IO API provided by the kernel and immediately returns to continue executing the user thread code. At this point, however, the user thread has registered the invocation AsynchronousOperation and CompletionHandler into the kernel, and the operating system starts a separate kernel thread to handle the I/O operations. When the read request arrives, the kernel reads the data from the socket and writes it to a user-specified buffer. Finally, the kernel distributes the read data and the user thread’s registered CompletionHandler to the internal Proactor, which notifts the user thread of the I/O completion (usually by calling the user thread’s registered completion event handler) to complete the asynchronous I/O.
The pseudocode for user threads using the asynchronous IO model is described as follows:
void UserCompletionHandler::handle_event(buffer)
{
process(buffer);
}
{
aio_read(socket, new UserCompletionHandler);
}
Copy the codeThe user needs to rewrite the handLE_EVENT function of CompletionHandler to process the data. The parameter buffer represents the data prepared by Proactor. The user thread directly calls the asynchronous IO API provided by the kernel. And register the overwritten CompletionHandler.
Compared with the IO multiplexing model, asynchronous IO is not very common, many high-performance concurrent services using IO multiplexing model + multithreaded task processing architecture can basically meet the needs. In addition, the current operating system for asynchronous IO support is not particularly perfect, more is to use IO multiplexing model to simulate asynchronous IO (IO event trigger does not directly notify the user thread, but after reading and writing data into the user specified buffer). Asynchronous IO has been supported since Java7, and interested readers can try it out. Welcome to pay attention to the public number: Java treasure to receive more tutorial welfare