Coroutines thread and the process is not open around in daily development concurrent programming content, but also want to learn how this piece of knowledge, must also be on the underlying computer knowledge has certain master, including the principle, operating system, network, etc., so I wrote a column thread process coroutines, plans to knowledge made a comb, is the summary of the self, I also hope to help readers.

This fourth article in the series focuses on the IO model of Unix, which will deepen our understanding of the underlying concepts and help us gain a deeper understanding of the tools available at the programming language level.

Front knowledge

Before introducing IO model, comb out a few important knowledge points first.

Unix Architecture

  • The kernel is mainly used to control the hardware and provide the operating environment, which is located in the core part of the operating system.
  • The interface provided by the kernel is called a system call.
  • There are shells and public libraries on top of system calls, respectively.
  • Applications can use public libraries and, in some cases, system calls.

File descriptor

Everything in Linux is a file. When a file is opened, the kernel returns a non-negative integer that identifies the file, becoming a file descriptor, or FD for short. During subsequent reads, writes, and other processing, the application can access the file through this descriptor without having to record additional information about the file.

The socket

The client and server essentially communicate using sockets.

Socket is an abstract concept, indicating one end of the TCP connection, through which data can be sent or received. We regard IP:Port as a set of sockets. A TCP connection consists of two sockets.

The process for a server to process a socket: Create a socket — bind a socket — listen a socket — receive and process information

The client processes the socket by creating a socket — connecting a socket — sending a message

A socket is also a file type, and a socket is an open file with a corresponding descriptor.

User space and kernel space

For the sake of kernel security and reliability, user programs cannot directly run kernel code or manipulate kernel data, so the operating system has a distinction between kernel space and user space. Applications running in user space (such as graphics and text editors, music players, and so on) need a specific mechanism to tell the kernel when they want to make certain system calls.

Unix IO model

In Unix, there are currently 5 IO models, divided into

  • Blocking IO
  • Nonblocking IO
  • IO multiplexing
  • Signal-driven IO
  • Asynchronous IO

Block type IO

Blocking IO is the most common and most commonly used IO model. Blocking IO can be suspended because an operation cannot be completed immediately.

As shown in the figure below, when an application calls recvFROM, its corresponding system call blocks and does not return until the datagram arrives and is copied to the application’s corresponding buffer. Blocking IO blocks in both of the above two processes: waiting for data and copying data.

At the kernel implementation level, this IO model is simple to implement and can return data without delay after the datagram is ready for subsequent processing, but it is often time-consuming for user processes to wait for the operation to complete.

Non-blocking IO

Non-blocking IO-related system calls always return immediately whether the operation has completed or not.

In non-blocking IO, an application process can polling the kernel in this manner, cycling through recvFROM to see if the datagram is ready, and then polling again after each time the polling kernel returns with the option of doing some other task.

Non-blocking IO type can be in the process of waiting for the data ready not blocked (but in the data from the kernel is copied to the user space is still in the process of blocking), in order to perform other tasks in the process of waiting for the data, but at the same time, due to the application process according to certain frequency polling, data ready time points may be located in between two polling, As a result, the data cannot be processed by subsequent processes after arrival, resulting in a certain delay. At the same time, the application process actively polls the kernel for data readiness. In many cases, data is not ready when polling for several times, which also causes excessive CPU resource consumption (usually, non-blocking I/OS need to be combined with other I/O notification mechanisms, such as I/O reuse).

IO multiplexing

The user process registers a set of file descriptors and corresponding events to the kernel through the IO multiplexing function. If any one or more data from multiple descriptors is ready, the IO multiplexing function returns ready data. In these descriptors, if there is no data-ready descriptor, the process is asleep, freeing CPU resources, and wakes up when a descriptor is ready, so that the user can work directly with the data-ready descriptor.

In Unix, select and poll are the two main I/O reuse functions. In Linux, epoll is a more advanced I/O reuse function. The following figure shows the basic flow of the IO reuse function. The application process blocks when calling SELECT, which listens for multiple socket descriptors and waits for any descriptor to be ready. When either descriptor is ready, the select function returns and the application process can copy the prepared datagrams to user space for further processing by calling recvFROM.

As shown in the figure below, blocking in IO reuse occurs when data is copied from the kernel to user space during a call to SELECT and recvFROM, respectively. In contrast to blocking and non-blocking IO, IO reuse at the kernel level supports blocking at multiple descriptors and returns when any descriptor is ready.

The strength of IO multiplexing is not that it can process a single descriptor faster, but that it can process more descriptors simultaneously (a common pattern is the non-blocking IO combined with epoll model). At present, IO multiplexing technology has been very mature, and its application scenarios may be more extensive than we imagined. For example, in network programming, in addition to processing multiple sockets at the same time, IO multiplexing will also be used in scenarios such as processing TCP and UDP sockets at the same time. In addition, many well-known application-level software products (such as Nginx) also make extensive use of IO reuse technology at the bottom.

IO multiplexing can process multiple descriptors in a single thread, avoiding the creation of threads in multithreading, switching between threads, etc., and the system overhead will be significantly reduced.

For each scheme, the advantages and the disadvantages are symbiotic, need according to the specific scene to select the optimal scheme, for example, when you need to deal with the number of connections is low, often multi-thread IO the implementation scheme of combining block type is better, so in each thread can timely processing data, are generally lower overall delay; When the number of connections to be processed is high and the number of threads in multiple threads can lead to performance degradation, it is better to use the related implementation of I/O multiplexing.

Signal-driven IO

The main idea of signal-driven IO is for the kernel to signal the process when the descriptor is ready, so it doesn’t block while waiting for the datagram to be ready. As shown in the figure below, using signal-driven IO on a socket first requires the SIGIO signal handler to be set up. The system call to set up the handler returns immediately and the process does not block. When the data is ready, the kernel generates SIGIO signals, and recvFROM can be called either in the signal handler function or in the main loop to copy the data from the kernel to user space.

Asynchronous I/o

A simple comparison of the following figures with the above diagrams shows that one significant difference is that in asynchronous IO, the application process does not need to call recvFROM to complete the data replication process. This is the main mechanism for asynchronous I/O: Asynchronous I/O-related functions notify the kernel to perform I/O operations, and then notify us after the kernel has completed all I/O operations, including waiting for data and copying data to user space. As you can see in the figure below, for example, when we call aio_read, we pass the descriptor and the associated data to the kernel and notify the kernel when the operation is complete. The call returns immediately and the process does not block.

As you can see, both asynchronous IO and signal-driven IO above include kernel notification procedures, but there are significant differences between the two IO models. First of all, for signal-driven IO, the kernel generates signals telling us when to perform the I/O operation, which we need to do ourselves, whereas for asynchronous IO, the kernel tells us when to complete the I/O operation, which is done by the kernel. Second, the former signals when the data is ready, while the latter signals when the data is ready and copied (in addition, the implementation of asynchronous IO under Linux is not yet mature).

IO model comparison

For the first four IO types, their first stage (waiting for data) is different, but the second stage processing is the same (recvFROM blocking wait), all belong to synchronous I/O, because in these four IO models, there are BLOCKING PROCESS I/O operations, while asynchronous I/O model is asynchronous I/O operations. The entire process does not block while waiting for data and data replication.

Here’s a metaphor for the image of the five IO models:

  • IO: Go to a restaurant and wait at the door
  • Synchronous non-blocking IO: If you go to a restaurant, you can move on to other things and come back every now and then
  • I went to the door of the restaurant, the restaurant has a waiter, pay the waiter, there is a let you in
  • Synchronous non-blocking signal drive: go to a restaurant for dinner, after going to get a number, you can go to do other things, to you, direct wechat to play the message for you to come
  • Asynchronous IO model: you sign in, go to do other, finished, directly ask you to send over

IO multiplexing

IO multiplexing is the most commonly used IO model, here to its specific implementation is introduced.

Select, poll, and epoll are all specific implementations of I/O multiplexing. Select was the first, followed by poll, and then epoll.

Select: passes polling to the kernel

Select allows an application to monitor a set of file descriptors and wait for one or more descriptors to become ready to complete IO operations.

Poll: Solves descriptor capping problems and improves efficiency

Poll () functions like SELECT () and is an implementation of IO multiplexing. Unlike select, poll() uses an array of variable-length (NFDS) pollFD structures that specify the file descriptors we expect to listen on and the corresponding events. Poll () solves the problem of upper limits on file descriptors in SELECT. At the same time, the problem of too large file descriptors affecting efficiency is avoided.

Epoll: Efficient event notification

When we call select() and poll(), we pass all the file descriptors we expect to check, whether based on the fd_set descriptor set or on a pollFD array of structures, and the kernel polls all file descriptors for each call. Performance problems arise when the number of file descriptors is large.

Epoll (Event Poll), introduced in Linux 2.6, effectively solves this problem. It is a unique I/O reuse function of Linux, but compared with SELECT and poll, Epoll uses red-black tree to manage data structures and has better performance.

Comparison of Application Scenarios

It’s easy to think that just using epoll is ok. Select and poll are obsolete, but they have their own usage scenarios.

The timeout parameter of SELECT has an accuracy of microseconds, while poll and epoll are milliseconds. Therefore, SELECT is more suitable for scenarios with high real-time requirements, such as nuclear reactor control. Select is more portable and is supported by almost all major platforms.

Poll has no maximum number of descriptors and should be used instead of SELECT if the platform supports it and is not demanding in real time.

Epoll only needs to run on a Linux platform, there are a large number of descriptors to poll for at the same time, and these connections should preferably be long. If less than 1000 descriptors need to be monitored at the same time, there is no need to use epoll because the advantages of epoll are not reflected in this application scenario.

The descriptor states that need to be monitored are variable and transient, and epoll is not necessary. Because all descriptors in epoll are stored in the kernel, epoll_ctl() is used to make system calls every time the state of the descriptor is changed. Frequent system calls reduce efficiency. And epoll descriptors are stored in the kernel, which is not easy to debug.

Apache and Nginx

Apache and Nginx are both common web servers. Compare their differences.

Apache, through multi-threading to support the concurrent client, server resource utilization rate is less than NGINx, simple function.

Nginx, the use of IO multiplexing, reading static files is the process of disk IO and network IO, the use of IO multiplexing is a natural advantage, can use fewer resources, support large concurrency, can support ten thousand levels of concurrency.

The resources

Unix Network Programming

CS-Notes