Java development essentials! I/O and Netty

I/O technology is becoming more and more important in system design, performance optimization, and middleware research and development. Learning and mastering I/ O-related technology is not only an additional skill for Java siege lion, but a necessary skill. This article will take you through the history and implementation of BIO/NIO/AIO, and introduce the basic principles of Netty, the current popular framework.

Java I/O model

1 BIO(Blocking IO)

BIO is the synchronous blocking model, where each client connection corresponds to one processing thread. In BIO, both the Accept and read methods block. If there is no connection request, the Accept method blocks. The read method blocks if there is no data to read.

2 NIO(Non Blocking IO)

NIO is a synchronous non-blocking model. A thread on the server can process multiple requests. The connection requests sent by the client are registered with the Selector of the multiplexer.

NIO’s three core components:

Buffer: Used to store data. The underlying implementation is based on arrays and provides Buffer classes for eight basic types.

Channel: Used for data transmission. The operation is buffer-oriented and supports bidirectional transmission. Data can be read from a Channel to a Buffer or written from a Buffer to a Channel.

The Selector: When a Selector registers a Channel with a Selector, the Selector mechanism automatically and continuously checks whether the registered Channel has any I/O events ready (such as readable, writable, network connection completed, etc.). This makes it easy for a program to efficiently manage multiple channels, or network connections, with a single thread. For this reason, selectors are also called multiplexers. When a read or write occurs on a Channel, the Channel is in a ready state, listened for by the Selector, and then retrieved by the SelectionKeys for subsequent I/O operations.

Epoll is an enhanced version of the Linux multiplex IO interface SELECT/Poll. It can significantly improve system CPU utilization in the case of a small number of active applications in a large number of concurrent connections. It does not need to go through the whole set of monitored descriptors to fetch events. You simply iterate over the set of descriptors that are asynchronously awakened by kernel IO events and added to the Ready queue.

3 AIO (NIO) 2.0

AIO is an asynchronous non-blocking model, which is generally used in applications with a large number of connections and a long connection time. After the completion of read/write events, the callback service notifies the program to start the thread for processing. Unlike NIO, when reading or writing, you simply call the read or write methods directly. Both methods are asynchronous. In the case of reads, when there is a stream to read, the operating system passes the readable stream into the buffer of the read method and notifies the application. For write operations, the operating system actively notifies the application when the stream passed by the write method has been written. Read /write methods are asynchronous and call the callback function after completion.

Two-i /O model evolution

1 Traditional I/O model

In the traditional I/O communication mode, the client connects to the server, and the server receives and responds to the request from the client in the following process: reading > decoding > application processing > encoding > sending the result. The server creates a new thread for each client connection, establishing a channel to process subsequent requests, the BIO way.

In this way, with the increasing number of clients, the response to connections and requests will drop sharply, and too many threads will be used to waste resources. The number of threads is not unlimited, and various bottlenecks will be encountered. Although this can be optimized using thread pools, there are still many problems. For example, when all threads in the thread pool are processing requests, they cannot respond to other client connections. Each client still needs a dedicated server thread to serve, even if the client has no requests, it is blocked and cannot be released. Based on this, an event-driven Reactor model is proposed.

2 Reactor model

The Reactor model is based on event-driven development. The server program processes incoming multiplex requests and synchronously dispatches them to the corresponding processing thread. The Reactor model is also called the Dispatcher model, which monitors events for I/O multiplexing and dispatches them to a process after receiving them. This is one of the essential techniques for writing high-performance web servers.

The Reactor model is supported by NIO, and its core components include Reactor and Handler:

• Reactor: The Reactor runs in a separate thread that listens for and dispatches events to the appropriate handlers to react to I/O events. It acts like a corporate telephone operator, answering calls from customers and redirecting the line to the appropriate contact.
• Handlers: The actual event that the Reactor responds to the I/O event by scheduling appropriate Handlers that perform non-blocking actions. Similar to actual employees of the company the customer wants to talk to.

According to the number of reactors and the number of Handler threads, reactors can be divided into three models:

• Single-thread model (Single-reactor single-thread)
• Multithreading model (SINGLE Reactor Multithreading)
• Principal/Slave Multithreading model (Reactor Multithreading)

Single threaded model

Connection events are monitored internally by a Selector and dispatched via Dispatch. If a connection is established, an Acceptor accepts a connection and creates a Handler to handle subsequent connection events. If it is a read/write event, the Handler corresponding to the connection is directly called to handle it.

Handler completes the read -> (decode -> compute -> encode) ->send business process.

The advantages of this model are simple, but the disadvantages are obvious. When a Handler blocks, both the Handler and Accpetor of other clients cannot be executed, thus achieving high performance. It is only applicable to scenarios where business processing is very fast, such as Redis read and write operations.

Multithreaded model

In the main thread, the Reactor object monitors connection events using a Selector, dispatches the event through Dispatch, and if it is a connection establishment event, it is processed by an Acceptor, which accepts the connection and creates a Handler to handle subsequent events. The Handler is only responsible for responding to events, but does not carry out service operations, that is, only reads data and writes data. The service processing is handed over to a thread pool for processing.

The thread pool allocates a thread to do the actual business processing, and then passes the response to the main process’s Handler, which sends the result to the client.

A single Reactor is responsible for monitoring and responding to all events. However, when our server encounters a large number of clients making simultaneous connections, or performs some time-consuming operations, such as identity authentication and permission checking, when requesting connections, such instantaneous high concurrency easily becomes a performance bottleneck.

Master-slave multithreaded model

There are multiple reactors, each with its own Selector Selector, thread, and dispatch.

The mainReactor in the main thread monitors connection establishment events through its Selector, receives the event through Accpetor, and assigns a new connection to a child thread.

The subReactor in the child thread adds the connection assigned by the mainReactor to the connection queue to listen through its Selector and creates a Handler to handle subsequent events.

Handler Completes the complete business process of read -> Service processing -> Send.

The most authoritative source for Reactor is Scalable IO in Java by Doug Lea for those interested.

Three Netty thread model

The Netty threading model is an implementation of the Reactor pattern, as shown in the following figure:

1 thread group

Netty abstracts two thread pools, BossGroup and WorkerGroup. Both of the types are NioEventLoopGroup. The BossGroup receives connections from clients, and the WorkerGroup processes connections that complete the TCP three-way handshake.

NioEventLoopGroup contains multiple NIoEventLoops and manages the life cycle of NioEventLoop. Each NioEventLoop contains a NIO Selector, a queue, and a thread; Threads are used to poll for read and write events for channels registered to the Selector and to handle events posted to the queue.

Boss NioEventLoop thread execution steps:

• Process the accept event, establish a connection with the client, and generate a NioSocketChannel.
• Register NioSocketChannel with a selector on a worker NIOEventLoop.
• Tasks that process the task queue, namely runAllTasks.

Worker NioEventLoop thread execution steps:

• Polls read and write events for all NIoSocketChannels registered with its Selector.
• Handle read and write events and handle business on the corresponding NioSocketChannel.
• RunAllTasks process tasks in the TaskQueue. Some time-consuming business processes can be placed in the TaskQueue to be processed slowly without affecting the flow of data in the pipeline.

The Worker NIOEventLoop processes the NioSocketChannel service using a pipeline, which maintains a linked list of handler handlers to process the data in the channel.

2 ChannelPipeline

Netty abstracts a Channel’s data pipeline into a Channel pipeline in which messages flow and are delivered. ChannelPipeline has a two-way linked list of ChannelHandler, an I/O event interceptor. ChannelHandler intercepts and processes I/O events. It is convenient to add and delete ChannelHandler to achieve different business logic customization. There is no need to modify the existing ChannelHandler to achieve modification closure and extension support.

A ChannelPipeline is a series of Instances of ChannelHandlers that can intercept inbound and outbound events flowing through a Channel. Whenever a new Channel is created, a new Channel pipeline is created and bound to the Channel. The association is permanent. A Channel can neither attach to another Channel nor detach from the current one. All of this is done by Netty without special intervention from the developer.

Depending on the origin, an event will be handled by ChannelInboundHandler or ChannelOutboundHandler, and the ChannelHandlerContext implementation will forward or propagate to the next ChannelHandler. A ChannelHandler can tell the next ChannelHandler in the ChannelPipeline to execute. Read events (inbound events) and write events (outbound events) use the same pipeline. Inbound events are passed from the head of the list back to the last inbound handler, and outbound events are passed from tail to the last outbound handler. The two types of handlers do not interfere with each other.

ChannelInboundHandler callback method:

ChannelOutboundHandler callback method:

3 Asynchronous non-blocking

Write operations: Writing data to a connection via NioSocketChannel’s write method is non-blocking and returns immediately, even if the thread calling the write is our business thread. Netty determines whether the thread calling NioSocketChannel’s write is a thread in its corresponding NioEventLoop in the ChannelPipeline. If not, the write request will be encapsulated as a WriteTask and posted to the queue in the corresponding NioEventLoop. Then, when the thread in the corresponding NioEventLoop polls for read and write events, it will be taken out of the queue for execution.

Read operation: When reading data from a NioSocketChannel, the business thread does not block and wait. Instead, it waits for the I/O polling thread in a NioEventLoop to notice that the data is ready for reading and processing.

The read and write events of each NioSocketChannel are executed in a single thread managed by the corresponding NioEventLoop. There is no concurrent read and write to the same NioSocketChannel, so locking is not required.

When we use Netty framework for network communication, when we initiate AN I/O request, we will immediately return, without blocking our business call thread. If you want to get the response result of the request, the business calling thread does not need to wait by blocking. Instead, when the response result comes out, the business calling thread uses the I/O thread asynchronously notifies the business, so the business thread does not have to do other things because of blocking during the whole request -> response process.