Linux network IO model

The essence of network IO is to read and write sockets. Sockets are abstracted as streams in Linux, and IO can be understood as convection operations.

The classification and category of IO

IO itself can be divided into memory IO, network IO and disk IO and cache IO, etc. Generally, when discussing IO, more refers to the last (network IO and disk IO, because these two are the slowest), here special analysis and description of network IO.

Classification of operation processing

Blocked/non-blocked

In terms of how functions/methods are implemented, that is, whether data is returned immediately before it is ready, or whether it is blocked after an IO request is made.

Blocking IO mechanism

In the case of blocking IO, when a user calls read, the user thread blocks until the kernel is ready and the data is copied from the kernel buffer to the user-mode cache. You can see the two parts of the block.

  • The CPU reads data from disk into the kernel buffer.

  • The CPU copies data from the kernel buffer to the user buffer.

Non-blocking IO mechanism

  • The non-blocking I/O makes a read request and finds that the data is not ready and continues, at which point the application repeatedly polls the polling kernel to see if the data is ready. When the data is not ready, the kernel immediately returns an EWOULDBLOCK error.

  • The read request does not get the result until the data is copied to the application buffer. And pay attention! Here, the last read call to fetch the data is a synchronous process, a waiting process. Synchronization refers to the process of copying kernel-state data into the user program’s cache.

Synchronous/asynchronous

IO read data flows through network -> kernel buffer -> user memory

  • The main difference between synchronous and asynchronous is whether the user process has to wait for data to move from the kernel buffer to user memory.

  • After waiting for kernel-mode data to be prepared, the user thread is automatically notified to read the information data. The user thread does not need to wait and can perform other operations.

For a network IO, there are two system objects, one is the process (or thread) that calls the IO, and the other is the kernel.

When a user-mode read operation occurs, it goes through two phases:

  • Phase 1: The user-mode thread is Waiting for the data to be ready.

  • Stage 2: User-mode threads that will copy the data from the kernel to the process.

    • The first step: usually involves waiting for packets of data on the network to arrive and then being copied to some buffer in the kernel.

    • Step 2: Copy data from the kernel buffer to the (user-mode) application process buffer. Network applications deal with two broad categories of problems: network IO and data computing. The former brings greater performance bottlenecks to applications.

Network IO models are as follows:

  • Synchronous IO model
  • Blocking IO model
  • Non-blocking IO model
  • Multiplexing IO model
  • Signal-driven IO Model
  • Asynchronous IO

Blocking IO model

In Linux, all sockets are blocking by default. A typical read operation flow is as follows:

When the user process invokes the recvFROM system call, as described above, there are two phases

Data preparation:

  • Most of the time, data does not arrive at the beginning, and the kernel has to wait for enough data to arrive. The user process will always block.

  • When the kernel waits for data to be ready, it copies the data from the kernel to user memory. The kernel then returns, and the user process terminates the block state and restarts.

A Blocking IO feature is that both phases of IO execution are blocked.

Non-blocking IO model [poll]

In Linux, you can make it non-blocking by setting the socket as follows:

  • When a user process calls recvfrom, if the data in the kernel is not ready, the user process does not block but immediately returns an error, which, from the user’s point of view, does not need to wait.

  • As can be seen from the figure, after judging that the result is an error, the user process realizes that the data is not ready, so it repeats the above operation until the data in the kernel is ready, and then it immediately copies the data to the user memory and returns.

Multiplexing IO model

Select /epoll/evpoll, also known as event-driven IO. The advantage is that a single process can process IO for multiple network connections simultaneously.

  • The rationale can be seen in “IO reuse Technology” below. Also called multiplex IO ready notification.

  • This is the ability for a process to foretell the kernel that one or more IO conditions specified by the process are ready and notify the user process.

  • Enables a process to wait on a sequence of events.

  • This process is not really that different from the Blocking IO process, in fact it looks worse just from the diagram because it requires two system calls. However, the advantage of Select is that it can handle multiple connections at the same time.

  • If the number of connections to be processed is not very high, using “Select/Epoll Web Server” does not necessarily perform better than using “Multithreaded + BIO Web Server”, and the latency is greater.

The advantage of Select/Epoll is not that it can process individual connections faster, but that it can process more connections.

In the IO multiplexing model, in practice, the socket is typically set to non-blocking for each socket, but as shown in the figure above, the entire user process is always blocked. The process is blocked by the select function rather than by the socket IO to the block.

Asynchronous IO

As soon as the user process initiates the read operation, it can start doing other things.

  1. From the kernel’s perspective, when it receives an asynchronous read, it first returns immediately, so no blocks are generated for the user process.

  2. The kernel waits for the data to be ready, then copies the data to the user’s memory. When this is done, the kernel sends a signal to the user process telling it that the read operation is complete.

To compare

The difference between non-blocking and asynchronous

The difference between non-blocking IO and asynchronous IO is clear.

  • In non-blocking IO, although the process is not blocked most of the time, it still requires the process to actively check, and when the data is ready, it also requires the process to actively call recvfrom again to copy the data to user memory.
  • Asynchronous IO is completely different. It is as if the user process hands off the entire IO operation to someone else (the kernel), who then sends a signal to notify when it is finished. During this time, the user process does not need to check the status of IO operations or actively copy data.

IO multiplexing technology

In the IO programming process, when you need to deal with multiple requests, you can use multithreading and I/O complex processing.

What is IO multiplexing?

Multiplexing the blocking of multiple IO to a block such as a SELECT allows the system to handle multiple requests simultaneously in a single thread.

Common APPLICATION scenarios of I/O multiplexing:

  • The server needs to process multiple listening and connected sockets simultaneously.
  • The server needs to handle sockets for multiple network protocols
  • The I/O multiplexing methods include SELECT, poll, and epoll/ EVpoll.

Select and poll work in much the same way:

  1. Register the FD to listen (it is best to create fd with non-blocking here)

  2. Each call checks the status of these FD’s and returns when one or more fd’s are ready

  3. The return result includes both ready and unready FDS

Select and poll compared to epoll mechanisms

Epoll is an obvious alternative to SELECT /poll in Linux network programming.

  • There is no limit to the number of socket descriptors that can be opened by a process (only limited by the operating system’s unlimit maximum number of file handles).

    • Limitations of Select: the number of FD opened by a process is limited, default is 2048; Although the value can be changed, the network efficiency may also decrease. A multi-process solution is an option, but the process creation itself is expensive, and inter-process data synchronization is far less efficient than inter-thread synchronization.

    • The maximum number of FD supported by epoll is the maximum number of files that can be opened: /proc/sys/fs/file-max

IO efficiency may decrease linearly as the number of file descriptors increases.

  • The epoll scanning system has different mechanisms

    • Select /poll is a collection of linear scan FDS;

    • Epoll is implemented based on a callback function on a FD. Active sockets actively call this callback function, but other sockets do not. It is an AIO, but the driving force is in the OS kernel.

  • Accelerate kernel-user space messaging using Mmap, a version of Zero-Copy.

  • Epoll’s API is simpler.

  • IO reuse also has the concept of horizontal triggering and edge triggering:

    • Horizontal trigger: when a ready FD has not been processed by the user process, the next query still returns. This is how select and poll trigger.
    • Edge trigger: No return next time regardless of whether a ready FD is processed or not. Theoretically, the performance is higher, but the implementation is quite complex, and any unexpected missing event can cause request processing errors. Epoll uses horizontal triggering by default and edge triggering can be used with the corresponding option.

Summary of IO model

A, B, C and D are fishing.

  • A was using the oldest fishing rod, so he had to wait until the fish hooked and then pull the rod. (Synchronous blocking)

  • B’s fishing rod has a function, can show whether there is a fish hooked, so, B and the MM next to chat, every once in a while to see if there is a fish hooked, then quickly pull rod; (Non-blocking)

  • C used a fishing rod similar to B’s, but he had the good idea of placing several fishing rods at the same time and then standing by and pulling up the corresponding rod when it was shown that a fish was hooked. (IO multiplexing mechanism)

  • D is a rich man, so he hires a man to fish for him and sends him a text message as soon as the man catches a fish. (Asynchronous mechanism)