1 Block I/OS synchronously

By default, all I/O operations on socket connections in Java application processes are synchronous blocking I/O.

In the blocking IO model, the Java process (or thread) that initiates the IO request blocks from the time the Java application initiates the IO system call until the system call returns. The application process can’t start processing buffer data in user space until it returns a success.

The specific flow of synchronous blocking I/O is as follows:

For example, issuing a system call to read on a socket in Java looks like this:

  1. The user thread (or thread) is blocked from the time Java initiates a read system call after an IO read.
  2. When the system kernel receives the READ system call, it prepares the data. At first, the data may not have reached the kernel buffer (for example, a complete socket packet has not been received), and the kernel waits.
  3. The kernel waits until the full data arrives, copies the data from the kernel buffer to the user buffer (memory in user space), and then returns the result (for example, the number of bytes copied into the user buffer).
  4. The user thread will not be unblocked and run again until the kernel returns.

Blocking IO is characterized by the fact that the user process (or thread) that initiated the IO request is blocked during both phases of the KERNEL’s IO operation.

The advantages of blocking IO are that application development is very simple; The user thread hangs while blocking and waiting for data, with little CPU usage.

The disadvantage of blocking IO is that there is typically a separate thread for each connection, and one thread maintains IO operations for each connection. In the case of small concurrency, this is fine. In high concurrency application scenarios, the blocking IO model requires a large number of threads to maintain a large number of network connections, resulting in huge overhead of memory and thread switching, low performance and basically unavailability.

2 Synchronize non-blocking I/OS

In Linux, the socket connection mode is blocking by default. You can set the socket to non-blocking mode. In the NIO model, once an application starts IO system calls, two things happen:

  1. In the absence of data in the kernel buffer, the system call immediately returns a call failure message.
  2. In the case of data in the kernel buffer, the system call blocks during the replication of the data until the data is copied from the kernel buffer to the user buffer. After the replication is complete, the system call returns a success and the user process (or thread) can begin processing buffer data in user space.

For example, issuing a system call to read on a non-blocking socket would look like this:

  1. In the stage where the kernel data is not ready, the user thread immediately returns when it makes an IO request. So, in order to read the final data, the user process (or thread) needs to make IO system calls over and over again.
  2. When kernel data arrives, the user process (or thread) makes a system call, and the user process (or thread) blocks. The kernel starts copying data, which it copies from the kernel buffer to the user buffer, and the kernel returns the result (for example, the number of bytes copied to the user buffer).
  3. After the user process (or thread) reads the data, it unblocks and starts running again. In other words, user space needs several attempts to ensure that the data is actually read and then executed.

Synchronous non-blocking IO is characterized by the application’s threads constantly making IO system calls, polling to see if the data is ready, and continuing polling if not until the IO system call is complete.

The advantage of synchronous non-blocking I/O is that each IO system call can be returned immediately while the kernel is waiting for data, and the user thread will not be blocked, resulting in good real-time performance.

The downside of synchronous non-blocking IO is constantly polling the kernel, which takes up a lot of CPU time and is inefficient.

In general, in high concurrency application scenarios, synchronous non-blocking IO is low performance and basically unavailable. Web servers do not use this IO model. This IO model will not be involved in the actual development of Java, but it is valuable because other IO models can use the non-blocking IO model as a basis for high performance.

3 I/O multiplexing

Currently, system calls that support I/O multiplexing include SELECT and epoll. Select system calls are supported by almost all operating systems and have a good cross-platform nature. Epoll was introduced in the Linux 2.6 kernel as a Linux enhanced version of the SELECT system call.

With select/epoll system calls in the IO multiplexing model, a single application thread can continuously poll hundreds of socket connections for ready states and return those ready states (or ready events) when one or more socket network connections have IO ready states.

Make a system call to the read operation of the multiplexed IO as follows:

  1. Selector registration. First, the target file descriptor (socket connection) that requires a read operation is pre-registered with the Linux SELECT /epoll Selector, which in Java corresponds to the Selector class. Then, start the polling process for the entire IO multiplexing model.

  2. Polling for ready states. Query the IO ready status of all pre-registered target file descriptors (socket connections) through the selector query method. Through the system call to the query, the kernel returns a list of ready sockets. The kernel buffer has data when any registered socket is ready or ready, the kernel adds the socket to the ready list, and returns a ready event.

  3. The user thread gets a list of ready states and makes a read system call based on the socket connection. The user thread blocks. The kernel starts copying the data, copying it from the kernel buffer to the user buffer.

  4. After the replication is complete, the kernel returns the result, and the user thread unblocks, reads the data, and continues to execute.

When the user process polls for IO ready events, the select query method of the selector needs to be called. The user process or thread that initiates the query is blocked. Of course, if a non-blocking overloaded version of the query method is used, the user process or thread that initiated the query will not block and the overloaded version will return immediately

The characteristics of THE IO multiplexing model are as follows: The IO of the IO multiplexing model involves two kinds of system calls, one is the system call of IO operation, the other is the SELECT /epoll ready query system call. The IO multiplexing model is built on the infrastructure of the operating system, that is, the kernel of the operating system must be able to provide multiway separated system calls select/epoll.

Like the synchronous non-blocking model, multiplexing IO also requires polling. The thread responsible for the select/epoll status query calls needs to continuously poll the SELECT /epoll to find the socket connections that are IO ready.

The IO multiplexing model is closely related to the synchronous non-blocking IO model. Specifically, every queriable socket connection registered with the selector is generally set to the synchronous non-blocking model, but this is not perceived by the user program.

The advantage of the IO multiplexing model is that a single selector query thread can handle thousands of network connections at the same time, so the user program does not have to create a large number of threads and do not have to maintain them, thus greatly reducing the system overhead. This is the biggest advantage of the IO multiplexing model over the blocking IO mode where one thread maintains one connection.

The NIO component of the Java language is implemented on Linux systems using ePoll system calls. So, the NIO component of the Java language uses the IO multiplexing model.

The disadvantage of the IO multiplexing model is that, in essence, the SELECT /epoll system call is blocking and belongs to synchronous IO. The system call itself is responsible for the read and write after the read and write event is ready, that is, the read and write process is blocked. To completely unblock threads, you must use the asynchronous IO model.

4 Asynchronous I/O(Signal-driven I/O)

The basic flow of the asynchronous IO model is that the user thread registers an IO operation with the kernel through a system call. After the entire I/O operation (including data preparation and data replication) is complete, the kernel notifies the user program, and the user performs subsequent service operations.

In the asynchronous IO model, the user program does not need to block during the entire kernel data processing process, including the kernel reading data from the network physical device (nic) into the kernel buffer and copying the data from the kernel buffer to the user buffer.

Make a system call to the read operation of an asynchronous IO as follows:

  1. When a user thread makes a read system call, it can do something else immediately. The user thread does not block.
  2. The kernel begins the first stage of IO: preparing data. When the data is ready, the kernel copies the data from the kernel buffer to the user buffer.
  3. The kernel sends a Signal to the user thread, or calls back the callback method registered by the user thread to tell the user thread that the read system call has completed and that data has been read into the user buffer.
  4. The user thread reads data from the user buffer to complete subsequent service operations.

The asynchronous IO model is characterized by the fact that the user thread is not blocked during both phases of the kernel waiting for data and copying data. The user thread needs to receive the kernel’s I/O completion event, or the user thread needs to register an I/O completion callback function. Because of this, asynchronous IO is sometimes called signal-driven IO.

The disadvantage of the asynchronous IO model is that the application only needs to register and receive events, leaving the rest to the operating system, which requires support from the underlying kernel.

In theory, asynchronous IO is truly asynchronous I/O, and its throughput is higher than that of the IO multiplexing model. At present, Windows systems implement true asynchronous IO through IOCP. On Linux, the asynchronous IO model was only introduced in version 2.6, and JDK support for it is currently incomplete, so there is no significant performance advantage for asynchronous IO.

Most applications with high concurrency servers are based on Linux systems. Therefore, IO multiplexing model is adopted in most of the development of such high concurrent network applications. The well-known Netty framework uses the IO multiplexing model rather than the asynchronous IO model.