Official account: Java Xiaokaxiu, website: Javaxks.com

Author: SessionBest, link: cnblogs.com/sessionbest…

Netty is a high-performance, asynchronous event-driven NIO framework based on JAVA NIO API implementation. It provides support for TCP, UDP, and file transfer as an asynchronous NIO framework for all IO operations of Netty

1. Introduction of Netty

Netty is a high-performance, asynchronous event-driven NIO framework based on API implementations provided by JAVA NIO. It provides support for TCP, UDP and file transfer. As an asynchronous NIO framework, all Netty IO operations are asynchronous and non-blocking. Through the Future-Listener mechanism, users can easily obtain the I/O operation results or through notification mechanism. As the most popular NIO framework, Netty has been widely used in the field of Internet, big data distributed computing, game industry, communication industry, etc. Some well-known open source components in the industry are also built based on Netty NIO framework.

2. Netty thread model

In terms of JAVA NIO, Selector provides the foundation for Reactor pattern, and Netty designs an efficient threading model combining Selector and Reactor pattern. Let’s start with the Reactor model:

2.1 Reactor model

Wikipedia explains the Reactor model this way: “The reactor design pattern is an event handling pattern for handling service requests delivered concurrently by one or more inputs. The service handler then demultiplexes the incoming requests and dispatches them synchronously to Associated request handlers. “. First of all, the Reactor model is event-driven, with one or more concurrent input sources, a Server Handler and Request Handlers. The Service Handler synchronously multiplexes incoming requests to the corresponding Request Handler. It can be shown as follows:

The structure is similar to the producer and consumer model, that is, one or more producers put events into a Queue, and one or more consumers poll events from the Queue to process them. The Reactor model has no Queue to buffer it, and every time an event is entered into the Service Handler, The Service Handler will proactively distribute the Evnent type to the corresponding Request Handler for processing.

2.2 Realization of Reator mode

Scalable In Java NIO Scalable in Java is well explained by Doug Lea in Scalable IO in Java, and the implementation of the Reator pattern is illustrated in a PPT excerpt

\1. The first implementation model is as follows:

This is the simplest single-thread Reactor model. Because the Reactor model uses asynchronous non-blocking I/O, none of the I/O operations are blocked, and theoretically one thread can process all the I/O operations independently. The Reactor thread is a generalist, multiplexing sockets, accepting new connections, and distributing requests to the processing chain.

For some low-volume applications, a single-threaded model can be used. However, it is not suitable for high-load, high-concurrency applications for the following reasons:

  1. When a NIO thread processes hundreds or thousands of links at the same time, the performance is not supported, even if the NIO thread’s CPU load reaches 100%, it cannot fully process messages
  2. When the NIO thread is overloaded, the processing speed slows down, resulting in a large number of client connections timeout. After the timeout, the connection is resended, which further increases the NIO thread load.
  3. The reliability is low. If a thread unexpectedly loops, the entire communication system becomes unavailable

To solve these problems, Reactor multithreading model is developed.

2.Reactor Multi-threaded model:

Compared with the previous model, this model uses multithreading (thread pool) in the processing chain part.

In most scenarios, this model meets the performance requirements. However, in some special application scenarios, for example, the server authenticates client handshake messages. In such scenarios, a single Acceptor thread may have performance problems. To solve these problems, a third Reactor thread model is developed

3.Reactor master-slave model

Compared with the second model, the Reactor is divided into two parts. The mainReactor is responsible for monitoring the server socket and accepting new connections. Assign the established sockets to the subReactor. The subReactor is responsible for multi-channel separation of connected sockets, reading and writing network data, and business processing, which is thrown to worker thread pool for completion. In general, the number of subreactors can be equal to the number of cpus.

2.3 Netty model

According to Reactor’s three models in 2.2, which one is Netty? In fact, Netty’s threading model is a variation of the Reactor model, that is, a variation of the third form without thread pooling, which is the default of Netty NIO. The participants of the Reactor model in Netty mainly include the following components:

  1. Selector
  2. EventLoopGroup/EventLoop
  3. ChannelPipeline

Selector is the Selector multiplexer provided in NIO, acting as demultiplexer, which will not be described here; The other two features and their roles in Netty’s Reactor schema are described below.

3.EventLoopGroup/EventLoop

When the system is running, if the thread context is switched frequently, it will bring additional performance loss. Multi-threaded concurrent execution of a business process, business developers also need to always be vigilant about thread safety, which data may be modified concurrently, how to protect? This not only reduces development efficiency, but also incurs additional performance costs.

In order to solve the above problems, Netty adopts the serialization design concept. The IO thread EventLoop is always responsible for the message reading, encoding, and subsequent Handler execution. Therefore, the whole process does not change the thread context, and the data does not face the risk of concurrent modification. This also explains why the Reactor master-slave model has removed thread pools from the Netty threading model.

An EventLoopGroup is an abstraction of a group of Eventloops. An EventLoopGroup provides a next interface, which can obtain one of the eventloops in a group according to certain rules to process tasks. The thing to know about EventLoopGroup is that in Netty, BossEventLoopGroup and WorkerEventLoopGroup are required to work in Netty server programming. Typically a server port is a ServerSocketChannel corresponding to a Selector and an EventLoop thread, meaning that the number of threads for BossEventLoopGroup is 1. BossEventLoop receives the client connection and delivers the SocketChannel to the WorkerEventLoopGroup for IO processing.

The implementation of EventLoop acts as a Dispatcher in the Reactor pattern.

4.ChannelPipeline

ChannelPipeline acts as a request handler in the Reactor pattern.

The default implementation of ChannelPipeline is DefaultChannelPipeline. The DefaultChannelPipeline itself maintains a tail and head ChannelHandler that is not visible to the user. They are at the head and tail of the list queue. Tail is in the upper part, while head is near the network layer. There are two important interfaces for ChannelHandler in Netty, ChannelInBoundHandler and ChannelOutBoundHandler. Inbound refers to the flow of network data from the external to the system, and outbound refers to the flow of network data from the internal to the system. The user-implemented ChannelHandler can implement one or more of these interfaces as needed and put them into a linked list queue in Pipeline. ChannelPipeline will find corresponding handlers to handle them according to different IO event types. At the same time, the linked list queue is a variant of the chain of responsibility mode. All the handlers that meet the event association from top to bottom will process the event.

ChannelInBoundHandler The ChannelInBoundHandler processes the packets sent from the client to the server. It is generally used to perform half-packet/pasting, decode, read data, and process services. ChannelOutBoundHandler Processes the packets sent from the server to the client. The ChannelOutBoundHandler is used to encode the packets and send the packets to the client.

Below is an illustration of ChannelPipeline execution:

More knowledge about Pipeline can refer to: [Pipeline model](blog.csdn.net/yanghua_kob… Discussion on Pipeline Model

5.Buffer

The extended Buffer provided by Netty has a number of advantages over NIO. As a very important piece of data access, let’s take a look at the features of Netty’s Buffer.

1.ByteBuf reads and writes Pointers

  • In ByteBuffer, the read and write Pointers are position, while in ByteBuf, the read and write Pointers are readerIndex and writerIndex respectively. Intuitively, ByteBuffer uses only one pointer to realize the function of two Pointers, saving variables. However, flip must be called when the read/write state of the ByteBuffer is switched, and clear must be called when the Buffe is read before the next write. Flip is called before each read and clear is called before each write, which undoubtedly brings tedious steps to development, and the content cannot be written until it is finished, which is very inflexible. ByteBuf, by contrast, relies only on the readerIndex pointer for reading and only on the writerIndex pointer for writing. There is no need to call the corresponding method before each read or write.

2 \ zero copy

  • Netty receives and sends bytebuffers using DIRECT BUFFERS, which use out-of-heap DIRECT memory for Socket reading and writing without the need for secondary copy of byte BUFFERS. If traditional HEAP BUFFERS are used for Socket reads and writes, the JVM copies the HEAP Buffer into direct memory before writing it to the Socket. The message is sent with an extra memory copy of the buffer compared to direct out-of-heap memory.
  • Netty provides the combined Buffer object, which can aggregate multiple ByteBuffer objects. Users can operate the combined Buffer as easily as operating a single Buffer. The traditional memory copy method of combining several small buffers into one large Buffer is avoided.
  • Netty adopts the transferTo method to transfer files, which can directly send the data in the file buffer to the target Channel, avoiding the memory copy problem caused by the traditional write method.

\3. Reference counting and pooling techniques

  • In Netty, each applied Buffer may be a valuable resource for Netty. Therefore, in order to gain more control over memory application and reclamation, Netty implements memory management according to reference counting method. Netty’s use of Buffer is based on DirectBuffer, which greatly improves the efficiency of I/O operations. However, compared with DirectBuffer and HeapBuffer, DirectBuffer has a natural disadvantage in addition to the high EFFICIENCY of I/O operations. That is, the application of DirectBuffer is less efficient than HeapBuffer, so Netty implements PolledBuffer with reference counting, that is, pooling. When the reference count is 0, Netty collects the Buffer into the pool. The Buffer will not be reused at any point in the next request.

conclusion

Netty is essentially an implementation of the Reactor pattern, with Selector as a multiplexer, EventLoop as a repeater, and Pipeline as an event handler. However, different from the general Reactor, Netty uses serialization and chain of responsibility pattern in Pipeline.

The buffer in Netty has been optimized compared to the buffer in NIO, which greatly improves performance.