I. Explanation of relevant concepts

1. Synchronous and asynchronous

Synchronization is a reliable task sequence in which the completion of one task depends on another task. The dependent task can only be completed after the dependent task is completed. Either success succeeds, failure fails, and the states of the two tasks remain the same.

Asynchrony means that you do not need to wait for the dependent task to complete, but only notify the dependent task of what to complete, and the dependent task will be executed immediately. As long as you complete the whole task, it is complete. As to whether the dependent task is actually completed in the end, the dependent task cannot be determined, so it is an unreliable task sequence.

2. Plugging and non-plugging

The concepts of blocking and non-blocking are related to the state of a program (thread) waiting for a message to be notified (whether synchronous or asynchronous). That is to say, blocking and non-blocking are mainly in terms of the state of the program (thread) waiting for a message notification.

A blocked call means that the current thread is suspended until the result of the call is returned, waiting for message notification and unable to perform other services. The function returns only after the result is obtained.

Nonblocking, as opposed to blocking, means that the function does not block the current thread but returns immediately until the result is not immediately available. While the non-blocking approach apparently improves CPU utilization, it also has the added consequence of increased thread switching on the system. Whether the increased CPU execution time will compensate for the system switching costs needs to be evaluated.

(a) If the thread is still performing other message processing while waiting for the current function to return, this situation is called synchronous non-blocking;

(b) If the thread is in a suspended wait state while waiting for the current function to return and no other message processing is performed, this is called synchronous blocking;

Synchronous/asynchronous focuses on the mechanism of message notification, while blocking/non-blocking focuses on the state of the program (thread) waiting for message notification.

User space and kernel space

Today’s operating systems use virtual storage, so for 32-bit operating systems, the addressing space (virtual storage) is 4G (2 ^ 32). The core of the operating system is the kernel, which is independent of ordinary applications and has access to the protected memory space as well as all permissions to access the underlying hardware devices. In order to ensure that user processes cannot directly operate the kernel and ensure kernel security, the operating system divides the virtual space into two parts, one is the kernel space and the other is the user space. For Linux, the highest 1G byte (from virtual addresses 0xC0000000 to 0xFFFFFFFF) is used by the kernel and is called kernel space, while the lower 3G byte (from virtual addresses 0x00000000 to 0xBfffff) is used by various processes and is called user space.

4. Process switching

To control process execution, the kernel must have the ability to suspend a process running on the CPU and resume execution of a previously suspended process. This behavior is called process switching. Therefore, it can be said that any process runs under the support of the operating system kernel and is closely related to the kernel. Moving from a running process to a running process goes through the following changes:

Save the processor context, including program counters and other registers. 2. Update PCB information. 3. Move the PCB of the process to the corresponding queue, such as ready, blocking in an event queue, etc. 4. Select another process to execute and update its PCB. Update memory management data structures. Restore the processor context. Note: All in all, it’s a drain on resources

5. Process blockage

If some expected events do not occur in the executing process, such as system resource request failure, waiting for the completion of an operation, new data has not arrived, or no new work is done, the system automatically executes Block primitives to change the running state into the blocked state. It can be seen that the blocking of a process is an active behavior of the process itself, so only the running process (to obtain CPU) can be put into the blocking state. When a process is blocked, it consumes no CPU resources.

6. File descriptor

A File descriptor is a computer science term, an abstract concept used to describe a reference to a File.

The file descriptor is formally a non-negative integer. In fact, it is an index value that points to the record table of open files that the kernel maintains for each process. When a program opens an existing file or creates a new file, the kernel returns a file descriptor to the process. In programming, some low-level programming tends to revolve around file descriptors. However, the concept of file descriptors is usually only applicable to operating systems such as UNIX and Linux.

7, caching,

Cache I/O is also known as standard I/O, and the default I/O operation for most file systems is cache I/O. In Linux’s caching IO mechanism, the operating system caches IO data in the file system’s page cache. That is, data is copied to the operating system kernel buffer before it is copied from the operating system kernel buffer to the application’s address space.

Disadvantages of caching IO:

In the process of data transmission, multiple data copy operations are required in the application address space and the kernel. These data copy operations cost a lot of CPU and memory.

IO model

The essence of network IO is to read socket. Socket is abstracted as stream in Linux system, and IO can be understood as the operation of convection. As mentioned earlier, for an IO access (such as read), data is copied to the operating system kernel buffer before being copied from the operating system kernel buffer to the application address space.

So, when a read operation occurs, it goes through two phases:

Phase 1: Waiting for the data to be ready. Stage 2: Copying the data from the kernel to the process.

For socket flows,

The first step: usually involves waiting for packets of data on the network to arrive and then being copied to some buffer in the kernel. Step 2: Copy data from the kernel buffer to the application process buffer.

Network applications need to deal with nothing more than two types of problems, network IO, data computing. Compared with the latter, network I/O delay brings more performance bottlenecks to applications than the latter.

Network IO models are as follows: · Synchronous IO · Bloking IO · Non-blocking IO · multiplexing IO · Signal-driven IO Note: Since signal Driven IO is not commonly used in practice, I will only mention the remaining four IO models.

1. Block IO model

The application calls an IO function, causing the application to block waiting for data to be ready. If the data is not ready, keep waiting… When the data is ready and copied from the kernel to user space, the IO function returns a success indicator.

When the recv() function is called, the system first checks to see if it has ready data. If the data is not ready, the system is in a wait state. When the data is ready, it is copied from the system buffer to user space, and the function returns. In a socket application, when the recv() function is called, data may not already exist in user space, so the recv() function is in a wait state.

2. Non-blocking IO model

By setting a SOCKET interface to non-blocking, we tell the kernel not to put the process to sleep when the requested I/O operation cannot be completed, but to return an error. So our I/O operation functions will constantly test if the data is ready, and if not, continue testing until the data is ready. During this continuous testing process, a lot of CPU time will be consumed. The above model is never recommended.

3. IO reuse model

Because synchronous non-blocking mode requires continuous active polling, polling takes up a large part of the process, polling will consume a lot of CPU time, and “background” may have multiple tasks going on at the same time, people think of the circular query of the completion status of multiple tasks, as long as any task is completed, to deal with it. It would be nice if polling were not the user state of the process, but someone to help. So this is called IO multiplexing

IO multiplexing has two special system calls select, poll, and epoll functions. Select polling is kernel level. The difference between select polling and non-blocking polling is that the former can wait for more than one socket. It can listen on more than one I/O port at the same time. Copying data from the kernel to the user process is, of course, blocked. A poll or select call blocks the socket process as opposed to blocking IO blocking. A poll call does not wait until all the socket data arrives. How do you know if some data arrived? Monitoring is left to the kernel, which handles the data arrival. You can also think of it as non-blocking.

The I/O reuse model uses the SELECT, poll, and epoll functions, which also block processes, but unlike blocking I/O, these two functions can block multiple I/O operations simultaneously. In addition, the I/O function of multiple read and write operations can be detected at the same time. The I/O operation function is not called until there is data readable or writable (note that not all data is readable or writable).

For multiplexing, that is, polling multiple sockets. Since multiplexing can handle multiple IO, it also introduces new problems. The order between multiple IO becomes uncertain, and of course it can be for different numbering.

In the PROCESS of I/O programming, multithreading or I/O multiplexing can be used to process multiple client access requests at the same time. I/O multiplexing allows the system to process multiple client requests simultaneously in a single thread by reusing multiple I/O blocks onto the same SELECT block. Than with traditional multi-threading/process model, the I/O multiplexing system overhead is the biggest advantage is small, system does not need to create a new additional process or thread, also do not need to maintain these processes and threads running, decreasing the workload of system maintenance, saves system resources, the I/O multiplexing main application scenario is as follows:

The server needs to process multiple listening or connected sockets at the same time. 2. The server needs to handle sockets for multiple network protocols simultaneously.

At this point, you may have thought about how Redis does it. Redis uses multiplexing.

3. Signal drives IO

Summary: Two calls, two returns;

First we allow the socket to do signal-driven I/O and install a signal handler. The process continues to run without blocking. When the data is ready, the process receives a SIGIO signal and can process the data by calling the I/O operation function in the signal handler function.

4. Asynchronous IO model

Unlike synchronous I/OS, asynchronous I/OS are not executed sequentially. After the aiO_read system call is made by the user process, the kernel data is returned directly to the user process, whether it is ready or not, and the user process can do something else. Once the socket data is ready, the kernel copies the data directly to the process and sends notifications from the kernel to the process. In both the IO and IO phases, processes are non-blocking.

Linux provides AIO library functions to implement asynchrony, but they are rarely used. There are many open source asynchronous IO libraries, such as Libevent, Libev, and libuv.

5. Comparison of 5 I/O models

The difference between different I/O models is mainly in the time period between data waiting and data replication, which is clearly shown in the figure.

The difference between non-blocking IO and asynchronous IO is clear from the above picture. In non-blocking IO, although the process is not blocked most of the time, it still requires the process to actively check, and when the data is ready, it also requires the process to actively call recvfrom again to copy the data to user memory. Asynchronous IO is completely different. It is as if the user process hands off the entire IO operation to someone else (the kernel), who then sends a signal to notify when it is finished. During this time, the user process does not need to check the status of IO operations or actively copy data.

Synchronous non-blocking mode compared to synchronous blocking mode:

Advantages: The ability to do other work while waiting for the task to complete (including submitting other tasks, i.e. multiple tasks running in the “background”).

Disadvantages: The response latency for task completion is increased because read operations are polled every once in a while, and tasks can be completed at any time between polls. This results in a decrease in overall data throughput.

Select, poll, epoll

To sum up, select select, poll, and epoll should be selected according to the specific application occasions and the characteristics of these three methods. 1. On the surface, epoll performs best, but select and poll may perform better than epoll when the number of connections is small and the connections are very active. After all, epoll’s notification mechanism requires many function callbacks. Select is inefficient because it requires polling every time. But inefficiencies are relative and can be improved by good design, depending on the situation

Supplementary knowledge:

Level_triggered: Epoll_wait () tells the handler to read or write when a read or write event occurs on a monitored file descriptor. If the buffer is too small, epoll_wait() will tell you to continue reading and writing from the file descriptor when it is called again. If you do not read or write at all, epoll_wait() will notify you! If you have a large number of ready file descriptors on your system that you don’t need to read or write, and they return every time, it makes it much less efficient for the processor to retrieve the ready file descriptors it cares about!

Edge_triggered: Epoll_wait () notifies the handler to read or write when a read or write event occurs on a monitored file descriptor. If the read/write buffer is too small, epoll_wait() will not notify you until a second read/write event occurs on the file descriptor. This mode is more efficient than horizontal triggering, and the system won’t be flooded with ready file descriptors that you don’t care about!

The SELECT () and Poll () models are both horizontal trigger mode, the signal-driven IO is edge trigger mode, and the Epoll () model supports both horizontal trigger and edge trigger. The default is horizontal trigger.