The I/O model for UNIX systems

Synchronous blocking I/O, synchronous non-blocking I/O, I/O multiplexing, signal-driven I/O, and asynchronous I/O.

What is the I/O

It is the process of copying data between computer memory and external devices.

Why I/O

The CPU accesses the memory faster than the external devices. Therefore, the CPU reads the data from the external devices into the memory before processing the data. When your program sends a read instruction to an external device through the CPU, it takes some time for the data to be copied from the external device to memory, and the CPU has nothing to do. Your program is:

  • Give up your CPU to someone else
  • Or let the CPU keep checking: has the data arrived? Have the data arrived yet? .

That’s what the I/O model is all about.

Java I/O model

A network I/O communication process, such as network data reading, involves two objects:

  • The user thread that invokes this I/O operation
  • Operating system kernel

The address space of a process is divided into user space and kernel space. User threads cannot directly access the kernel space. After the user thread initiates an I/O operation (the Selector call is an I/O operation), the network data read operation goes through two steps:

  1. The user thread waits for the kernel to copy data from the nic into the kernel space
  2. The kernel copies data from kernel space to user space

One wonders if copying kernel data from kernel space to user space is a bit wasteful. After all, there’s really only one piece of memory, so can I just point it to user space and read it? Linux has a system call called Mmap that maps disk files to memory, eliminating copying in kernel and user space, but does not support network communication scenarios!

The difference between the VARIOUS I/O models is the way these two steps work.

Synchronously block I/O

The user thread blocks after making a read call, freeing the CPU. The kernel waits for the nic data to arrive, copies the data from the NIC to kernel space, then copies the data to user space, and wakes up the user thread.

Synchronize non-blocking I/ OS

The user process initiates a read call, which is a system call where the CPU switches from user to kernel mode and executes kernel code. The kernel finds that the socket is in kernel space, suspends the user thread, copies the data from kernel space to user space, wakes up the user thread, and returns with the read call.

The user thread continues to make read calls that fail until the data reaches the kernel. After this call, the thread blocks while waiting for the data to be copied from the kernel to user space, and then wakes up when the data reaches user space.

I/O multiplexing

The user thread reads in two steps:

  • The program initiates a SELECT call, asking the kernel: Is the data ready?
  • Once the kernel has the data ready, the user thread makes the read call

The thread is still blocked while waiting for data to be copied from kernel space to user space

Why is it called I/O multiplexing? Because a single SELECT call can check the status of multiple data channels inward.

The NIO API can do without Selector, which is synchronous non-blocking. So if you use Selector, that’s IO multiplexing.

Asynchronous I/O

When the user thread makes a read call, it registers a callback function. Read returns immediately, and after the kernel has prepared the data, it calls the specified callback function to complete processing. During this process, the user thread remains unblocked.

Signal drives I/O

Signal driven I/O can be understood as “half an asynchronous, non-blocking mode is applied constantly initiate read call query data to the kernel, and make this process the asynchronous signal driver, applications read call register a signal handler, actually is a callback function, after the data into the kernel, the kernel triggered the callback function, The application makes another read call in the callback function to read the kernel data. So it’s semi-asynchronous.

NioEndpoint components

Tomcat’s NioEndpoint implements the I/O multiplexing model.

The working process

Java multiplexer use:

  1. Create a Selector, register the event of interest on it, then call the Select method and wait for the event of interest to happen
  2. When something of interest happens, such as being readable, a new thread is created to read data from the Channel

NioEndpoint consists of LimitLatch, Acceptor, Poller, SocketProcessor, and Executor.

LimitLatch

The connection controller controls the maximum number of connections, which defaults to 8192 in NIO mode.The thread blocks when the number of connections reaches its maximum until subsequent components decrement the number of connections by one after processing a connection. When the maximum number of connections is reached, the bottom layer of the OS still receives client connections, but the user layer no longer does. Core code:

public class LimitLatch {
    private class Sync extends AbstractQueuedSynchronizer {
     
        @Override
        protected int tryAcquireShared(a) {
            long newCount = count.incrementAndGet();
            if (newCount > limit) {
                count.decrementAndGet();
                return -1;
            } else {
                return 1; }}@Override
        protected boolean tryReleaseShared(int arg) {
            count.decrementAndGet();
            return true; }}private final Sync sync;
    private final AtomicLong count;
    private volatile long limit;
    
    // The thread calls this method and gets permission to receive a new connection. The thread may block
    public void countUpOrAwait(a) throws InterruptedException {
      sync.acquireSharedInterruptibly(1);
    }

    // Call this method to release a connection permission, and the previously blocked thread may be awakened
    public long countDown(a) {
      sync.releaseShared(0);
      long result = getCount();
      returnresult; }}Copy the code

The user thread calls LimitLatch#countUpOrAwait to get the lock. If it cannot get the lock, the thread will be blocked in the AQS queue. How does AQS know whether to block or not block the user thread? AQS#tryAcquireShared() : if the current connection count < limit, the thread can acquire the lock and return 1, otherwise return -1.

Sync overridden AQS#tryReleaseShared() to accept new connections when a connection request is processed, so that the previously blocked thread will be woken up.

LimitLatch is used to limit the number of connections that applications can receive, and acceptors are used to limit the number of connections that applications can receive at the system level. First, LimitLatch is used to limit the number of connections that applications cannot handle, so connections pile up in the OS Queue, which is controlled by acceptCount.

Acceptor

Acceptors implement the Runnable interface, so they can run in a separate thread and accept new connections in an infinite loop. As soon as a new connection request arrives, the Accept method returns a Channel object, which is then handed over to the Poller for processing.

A port number can correspond to only one ServerSocketChannel. Therefore, the ServerSocketChannel is shared between multiple Acceptor threads. The ServerSocketChannel is a property of the Endpoint, which performs initialization and port binding. Accept is thread-safe if an Acceptor calls accept at the same time.

Initialize the

protected void initServerSocket(a) throws Exception {
    if(! getUseInheritedChannel()) { serverSock = ServerSocketChannel.open(); socketProperties.setProperties(serverSock.socket()); InetSocketAddress addr =new InetSocketAddress(getAddress(), getPortWithOffset());

        serverSock.socket().bind(addr,getAcceptCount());
    } else {
        // Retrieve the channel provided by the OS
        Channel ic = System.inheritedChannel();
        if (ic instanceof ServerSocketChannel) {
            serverSock = (ServerSocketChannel) ic;
        }
        if (serverSock == null) {
            throw new IllegalArgumentException(sm.getString("endpoint.init.bind.inherited")); }}// Block mode
    serverSock.configureBlocking(true); //mimic APR behavior
}
Copy the code
  • The getAcceptCount() parameter of the bind method represents the length of the OS wait queue. When the number of connections reaches the maximum, the OS can continue receiving connections. The maximum number of connections the OS can continue receiving is the queue length, which can be configured using the acceptCount parameter. The default value is 100

ServerSocketChannel accepts new connections via Accept (), which returns a SocketChannel object, which is then wrapped in a PollerEvent object, The PollerEvent object is pressed into the Poller Queue. This is a typical producer-consumer pattern, where acceptors communicate with Poller threads through a Queue.

Poller

It’s essentially a Selector, which also runs in a separate thread.

Poller internally maintains an array of channels, which in an infinite loop continuously detects the ready state of a Channel, generating a SocketProcessor task object that is thrown to Executor for processing once a Channel is readable.

The accept method generates a scoketChannel for each connection. The Poller uses selector#select to check whether the kernel is ready. To know which channel to listen to in the kernel.

Maintains a Queue:

SynchronizedQueue methods such as offer, poll, size, and clear use the synchronized modifier, which means that only one Acceptor thread reads or writes to the Queue at a time. Multiple Poller threads are running at the same time, and each Poller thread has its own Queue. Each Poller thread may be called by multiple Acceptor threads simultaneously to register a PollerEvent. The number of pollers can be configured using the Pollers parameter.

Duties and responsibilities

  • Poller continuously queries the kernel for Channel status through its internal Selector object. Once readable, Poller generates a SocketProcessor task class for the Executor to handle

  • Poller loops to check whether the managed SocketChannel times out. If timeout occurs, close the SocketChannel

SocketProcessor

Poller creates the SocketProcessor task class for the thread pool to handle. The SocketProcessor implements the Runnable interface that defines the tasks executed by Executor threads. The Http11Processor reads the Channel’s data to generate the ServletRequest object.

The Http11Processor does not read channels directly. Because Tomcat support synchronous non-blocking I/O and asynchronous I/O model, in the Java API, corresponding to different Channel class, such as a AsynchronousSocketChannel and SocketChannel to Http11Processor shielding the differences, Tomcat designed a wrapper class called SocketWrapper. Http11Processor only calls SocketWrapper to read and write data.

Executor

The thread pool that runs the SocketProcessor task class. SocketProcessor’s run method calls Http11Processor to read and parse the request data. As we know, Http11Processor is the encapsulation of the application layer protocol. It will call the container to get the response, and then write the response through the Channel.

Tomcat’s custom thread pool, which creates the worker threads that actually do the work. Is to execute socketProcess # Run, which parses the request and processes it through the container, eventually calling the Servlet.

Tomcat’s high concurrency design

High concurrency means that a large number of requests can be processed quickly. It is necessary to design a reasonable thread model to keep THE CPU busy, and try not to let the thread block, because once blocked, the CPU will be idle. As many tasks as there are, the corresponding number of threads are used to process them. For example, NioEndpoint does three things: receiving connections, detecting I/O events, and processing requests. The key is to customize the number of threads for each of these three things:

  • Special thread groups are used to run acceptors, and the number of acceptors can be configured
  • Special thread groups are used to run pollers. The number of pollers can be configured
  • Specific task execution is handled by a dedicated thread pool, and the size of the thread pool can also be configured

conclusion

The I/O model is designed to address memory and peripheral speed differences.

  • Blocking or non-blocking refers to whether an application initiates an I/O operation and returns immediately or waits
  • Synchronous and asynchronous refers to the copying of data from the kernel space to the application space when the application program communicates with the kernel. Whether the copy is initiated by the kernel or triggered by the application program.

The Tomcat#Endpoint component’s main job is to handle I/O, and NioEndpoint implements a multiplexed I/O model using the Java NIO API. Instead of blocking on the I/O wait itself, the thread reading or writing data passes the job off to the Selector.

When a client initiates an HTTP request, it is initiated by an Acceptor#run

socket = endpoint.serverSocketAccept();
Copy the code

It receives the connection and passes it to a thread named Poller to detect I/O events. The Poller thread will always select the channel where the kernel copies the data from the nic to the kernel space (that is, the kernel is ready for the data) and hand it to a thread named Catalina-exec. This process also involves the kernel copying data from kernel space to user space, so it blocks for the exec thread, at which point the user space (i.e., the exec thread) receives the data, parses it, and does business.

reference

  • Blog.csdn.net/historyasam…