preface

As an asynchronous communication framework, Netty has almost become a must for high-performance RPC framework. For example, Netty is used for communication components in the Dubbo framework, and for producer and consumer communication in RocketMQ. Today, we’ll look at the basic architecture and principles of Netty.

Spring Boot

Netty features with NIO

Netty is an asynchronous, event-driven network application framework that can be used to develop high-performance servers and clients.

When writing network callers, we used to create a Socket on the client side and use that Socket to connect to the server side.

The server creates a Thread based on the Socket and sends the request. After making the call, the client needs to wait for the server to complete the processing before continuing with the subsequent operations. The thread is in a waiting state.

If the number of client requests increases, the server creates more processing threads, which is not an easy task for the JVM.

To solve the above problem, the concept of NIO was introduced, that is, non-blocking I/O. The Selector mechanism is at the heart of NIO.

Each time a client requests it, a Socket Channel is created and registered with a Selector (a multiplexer).

Then, the Selector looks at the server I/O read/write event, and instead of waiting for the I/O event to complete, the client can continue to do the following work.

Once the server has finished reading or writing the IO, the Selector gets a notification and tells the client that the IO is complete.

After receiving the notification, the client can obtain the required data through SocketChannel.

The procedure described above is somewhat asynchronous, but the Selector implementation is not truly asynchronous.

Because a Selector listens for an IO change through thread blocking, it doesn’t have the client waiting, it waits for the IO to return, and it tells the client to fetch the data. The real “asynchronous IO” (AIO) is not introduced here. If you are interested, you can find it by yourself.

Let’s talk about Netty. As NIO’s implementation, Netty is suitable for server/client communication scenarios, as well as high concurrency applications under TCP protocol.

For developers, it has the following features:

• By encapsulating NIO, developers don’t need to pay attention to the underlying principles of NIO and just call Netty components to get the job done.
• Transparent to network calls, from Socket to establish TCP connections to network exceptions are wrapped.
• Netty supports multiple serialization frameworks. You can customize codecs and codecs through the ChannelHandler mechanism.
• Performance-friendly, Netty provides a thread-pool pattern and reuse mechanism for Buffers (object pooling) that eliminates the need to build complex multithreaded models and queues of operations.

Let’s start with a simple example

In order to meet high concurrency network requests, the concept of NIO was introduced. Netty is the implementation of NIO, NIO encapsulation, network call, data processing and performance optimization have good performance.

The easiest way to learn about architecture is to see how Netty works by example, accessing server-side code from the client side. Again, the components that are invoked in the code and how they work.

Suppose there is a client to call a server, suppose the server is called EchoServer, the client is called EchoClient, Netty architecture implementation code is as follows.

Server code

Build the server side, assuming the server accepts the message from the client and prints it on the console. First, generate an EchoServer, passing in the constructor the port number you want to listen on.

Here’s how to start the service:

The Server startup method involves calling components such as EventLoopGroup and Channel. These will be explained in more detail later.

Here’s a general idea:

• Create EventLoopGroup.
• Create ServerBootstrap.
• Specify the NIO transport Channel to use.
• Sets the socket address with the specified port.
• Add a ServerHandler to a Channel’s ChannelPipeline.
• Bind the server asynchronously; Calling the sync() method blocks and waits until the binding is complete.
• Gets the CloseFuture of the Channel and blocks the current thread until it completes.
• Close the EventLoopGroup to release all resources.

After starting NettyServer will listen for a port request, when received the request needs to process. In Netty, a client requests a server, which is called an “inbound” operation.

Can be achieved by ChannelInboundHandlerAdapter, specific content as follows:

As you can see from the above code, the server-side code contains three methods. All three methods are triggered based on events.

They are:

• The operation when a message is received, channelRead.
• Method when the message is finished reading, channelReadComplete.
• Method when an exception occurs, exceptionCaught.

Client code

The client and server code is similar in that the IP and Port of the server are entered during initialization.

Also include the following in the client startup function:

The order in which the client starts the program:

• Create the Bootstrap.
• Specify an EventLoopGroup to listen for events.
• Define the transmission mode of a Channel as NIO (non-block output output).
• Set the InetSocketAddress of the server.
• When creating a Channel, add an EchoClientHandler instance to the ChannelPipeline.
• Connect to the remote node, block and wait until the connection is complete.
• Blocks until the Channel is closed.
• Close the thread pool and release all resources.

After completing the above operations, the client establishes a connection with the server to transfer data. Also, when receiving an event triggered in a Channel, the client triggers the corresponding event.

Such as Channel activation, client receiving messages to the server, or exception catching.

The code structure is relatively simple. The server and client initializes the listener and connection, respectively. Then each defines its own Handler to handle each other’s requests.

Netty core components

From the simple example above, some Netty components are used for service initialization and communication. Here are some of their uses and relationships.

(1) the Channel

As you can see from the above example, a Channel is established when the client and server connect.

This Channel is known as a Socket connection and is responsible for basic IO operations such as bind (), connect (), read (), write (), and so on.

To put it simply, Channel stands for connection, connection between entities, connection between programs, connection between files, and connection between devices. It is also the carrier of inbound and outbound data.

(2) the EventLoop and EventLoopGroup

Now that there is a Channel connection service, information can flow between each other. If the message sent by the service is called an “outbound” message, the message received by the service is called an “inbound” message. Then the “outbound”/” inbound “of the message will generate events.

For example, the connection is active. Data reading; User events; Abnormal events; Open links; Close links and so on.

Following this line of thinking, given data, the flow of data produces events, then there is a mechanism to monitor and coordinate events.

That mechanism (component) is EventLoop. In Netty, each Channel is assigned an EventLoop. An EventLoop can serve multiple channels.

Each EventLoop occupies one Thread, and this Thread handles all IO operations and events that occur on the EventLoop (Netty 4.0).

An EventLoopGroup is the object that generates the EventLoop. The first line of the example code contains a new EventLoopGroup object.

An EventLoopGroup contains multiple EventLoop objects.

All the EventLoopGroup does is create a new Channel and assign it an EventLoop.

In the case of asynchronous transmission, an EventLoop can handle events generated in multiple channels. Its main job is event discovery and notification.

This is compared to the previous situation where one Channel occupies one Thread. Netty’s approach is much more reasonable.

The client sends the message to the server, and When EventLoop finds it, it tells the server, “You go get the message,” while the client does other work.

When EventLoop detects a message returned by the server, it also notifies the client: “Message returned, go get it.” The client then retrieves the message. The EventLoop is the monitor and the speaker.

③ChannelHandler, ChannelPipeline and ChannelHandlerContext

If an EventLoop is the notifier of an event, ChannelHandler is the handler of the event.

In ChannelHandler you can add business code, such as data conversion, logic operations, and so on.

As shown in the example above, the Server and Client each have a ChannelHandler to handle, read information, network availability, network exceptions, etc.

There are different channelhandlers for outbound and inbound events:

• ChannelInBoundHandler (inbound event handler)
• ChannelOutBoundHandler (outbound event handler)

It is assumed that each request raises an event, which is handled by ChannelHandler. The order in which these events are handled is determined by ChannelPipeline.

The ChannelPipeline provides containers for the ChannelHandler chain. When a Channel is created, it is automatically assigned to the ChannelPipeline by the Netty framework.

ChannelPipeline guarantees that the ChannelHandler processes events in a certain order. When an event is triggered, it sends data through ChannelPipeline to the ChannelHandler in a certain order.

To put it bluntly, ChannelPipeline is “queuing”. The “queue” here is the order in which events are processed.

ChannelPipeline can also add or remove channelHandlers to manage the entire queue.

As shown in the figure above, the ChannelPipeline makes channelHandlers sort in order, with messages flowing in the direction indicated by the arrow and processed by the ChannelHandler.

ChannelPipeline and ChannelHandler, the former manages the ordering of the latter. Then the association between them is represented by the ChannelHandlerContext.

Whenever a ChannelHandler is added to the ChannelPipeline, the ChannelHandlerContext is also created.

The main function of the ChannelHandlerContext is to manage the interaction between ChannelHandler and ChannelPipeline.

I don’t know if you noticed, but in the original example where ChannelHandler was handling the event function, the parameter that was passed in was ChannelHandlerContext.

The ChannelHandlerContext parameter runs through the ChannelPipeline, passing information to each ChannelHandler, and is a qualified “correspondent.”

The core components mentioned above are summarized in the following figure to help you remember their relationship.

Netty data container

This section describes the core components of Netty. The server generates events during data transmission and monitors and processes the events.

Let’s take a look at how the data is stored and read and write. Netty uses ByteBuf as a data container to store data.

How ByteBuf works

Structurally, ByteBuf consists of an array of bytes. Each byte in the array holds information.

ByteBuf provides two indexes, one for reading data and one for writing data. These indexes move through byte arrays to locate where information needs to be read or written.

When read from ByteBuf, its readerIndex is incremented by the number of bytes read.

Similarly, when writing to ByteBuf, its writerIndex is incremented by the number of bytes written.

Note that the limit is when readerIndex reads exactly where writerIndex writes.

If readerIndex exceeds writerIndex, Netty throws an IndexOutOf-BoundsException exception.

ByteBuf Usage mode

Now that we’ve talked about how ByteBuf works, let’s look at its usage patterns.

According to the different storage buffer can be divided into three types:

• The ** heap buffer, **ByteBuf stores data in the JVM’s heap, which is implemented in arrays for fast allocation. Because it is managed by the JVM on the heap, it can be quickly released when not in use. Byte [] data can be obtained by bytebuf.array ().
• ** Direct buffer, ** allocates memory directly outside of the JVM’s heap for storing data. It does not take up heap space, and memory capacity needs to be considered when using it. It performs better with Socket delivery because the data is sent indirectly from the buffer, and the JVM copies the data to the direct buffer before sending. Because direct buffer data is allocated out of the heap, garbage is collected by the JVM, and allocation also requires copying, it is expensive to use.
• The ** composite buffer, as its name implies, aggregates the above two types of buffers together. Netty provides a CompsiteByteBuf that puts heap buffer and direct buffer data together to make it easier to use.

The distribution of the ByteBuf

After talking about structure and usage patterns, let’s look at how ByteBuf allocates buffer data.

Netty provides two implementations of Byte bug Locator, which are:

• **PooledByteBufAllocator, ** implements pooling of ByteBuf objects to improve performance and reduce memory fragmentation.
• **Unpooled -bytebufallocator, ** does not implement object pooling and generates new object instances each time.

Object pooling is a similar technique to thread pooling, with the main goal of increasing memory utilization. A simple way to implement pooling is to build a layer of memory pool on top of the JVM heap memory, use the ALLOCATE method to obtain space in the pool, and use the release method to return space to the pool.

When an object is created and destroyed, the allocate and release methods are called in large numbers. Therefore, the memory pool faces the problem of defragmentation reclamation. After space is frequently applied for and released, the memory pool needs to ensure continuous memory space for object allocation.

Based on this requirement, there are two algorithms for optimizing memory allocation in this area: the partner system and the slab system.

Partner system, with complete binary tree management memory area, left and right nodes are partners, each node represents a memory block. Memory allocation divides large chunks of memory until the smallest fragment is found.

Memory release determines whether the partners (left and right nodes) that release the memory fragment are free. If so, the left and right nodes are combined into a larger chunk of memory.

Slab system mainly solves the problem of memory fragmentation by dividing large chunks of memory according to a certain size to form a memory set composed of the same size memory slices.

Apply for a small piece of memory or an integer multiple of the required memory space. When releasing the memory, the memory fragments are returned to the memory set.

Netty memory pool management occurs in the form of Allocate objects. An Allocate object consists of multiple Arenas, each of which can Allocate and reclaim chunks of memory.

There are three types of memory block management units in an Arena:

• TinySubPage
• SmallSubPage
• ChunkList

Tiny and Small comply with the management policies of Slab systems, and ChunkList comply with the management policies of partner systems.

When the requested memory is between tinySize and smallSize, the memory block is obtained from tinySubPage.

If the memory size is between smallSize and pageSize, the memory block is obtained from smallSubPage. Between pageSize and chunkSize, get memory from ChunkList; Memory blocks larger than ChunkSize (the size of allocated memory is not known) are not allocated through pooling.

The Bootstrap of Netty

The core components of Netty and the data store. Back to the original example, a new Bootstrap object will be created at the beginning of the program, and all subsequent configuration will be based on this object.

Bootstrap is used to configure Netty core components into an application and get them running.

In terms of the inheritance structure of Bootstrap, it can be divided into two types: Bootstrap and ServerBootstrap, one corresponding to the Bootstrap of the client and the other corresponding to the Bootstrap of the server.

The client Bootstrap has two main methods bind () and connect (). Bootstrap creates a Channel through the bind () method.

After bind (), create a Channel connection by calling the connect () method.

The ServerBootstrap, unlike the client, creates a ServerChannel after the Bind () method, which not only creates new channels but also manages existing channels.

From the above description, there are two differences between server and client boot:

• ServerBootstrap Binds a port to listen for connection requests from clients. Bootstrap (client boot) only needs to know the server IP and Port to establish a connection.
• Bootstrap (client boot) requires one EventLoopGroup, but ServerBootstrap (server boot) requires two EventLoopGroups. Because the server requires two different sets of channels. The first set of ServerChannels itself listens for sockets on local ports. The second set of sockets listens for client requests.

conclusion

We started with NIO, and we talked about the core mechanism of Selector. Then through the introduction of Netty client and server source code running process, so that we have a basic understanding of Netty code, if you have mastered, then hurry to practice the factory interview real questions!

In Netty’s core component, a Channel provides a Socket connection Channel, and an EventLoop listens for events generated by the Channel and notifies the executor. A container for the EventloopGroup that generates and manages the EventLoop.