Review:

What is Netty?

Netty is an asynchronous event-driven network programming framework. Its high performance comes from its I/O model and thread processing model. The I/O model determines how to send and receive data, and the thread processing model determines how to process data.

Netty supports three I/O models that can be switched at will. We generally use the NIO thread model, where a single thread can monitor multiple non-blocking channels through Selector, thus avoiding threads hanging due to frequent I/O blocking. In addition, the concept of Channel and Buffer is introduced, and the data interaction between Channel and Buffer can be arbitrarily operated, avoiding the sequential operation of IO stream directly.

Event-driven model: Reactor event distribution model + Handlers asynchronous model

Event Distribution Model: Netty is based on the multithreaded model of the Master and slave Reactors. The master Reactor is responsible for client connection requests and the slave Reactor for channel read and write requests. In Netty, the primary Reactor is a thread pool that is shared by the Reactor and worker threads. Each thread in the primary Reactor pool acts as a primary Reactor, corresponding to an Accept event, i.e., a port. Threads from the slave thread pool are responsible for processing and executing read and write events, making the slave thread pool fully utilized.

The I/O operations in Netty are all asynchronous, based on the observer model. The main thread puts the event into the event queue, and the other thread circulates through the event list, returning a ChannelFuture. The main thread then uses the Listener mechanism to determine whether the asynchronous thread has performed the I/O completion and fetch the result of the I/O operation.

The event-driven model consists of four basic components:

Event Queue: An entry to receive events and store events to be processed

Event Mediators: Distribute different events to different business logic units

Event Channels: Communication channels between dispensers and processors

Event Processor: Implements the service logic. After the processing is complete, an event is emitted to trigger the next operation

Netty component

ServerBootStrap and BootStrap are bootstraps that are used to configure information about Netty programs to connect components together. NioEventLoop maintains a thread and a task queue, which are responsible for listening and processing all I/O operations and asynchronously submitting tasks. ServerSocketChannel and SocketChannel, which provide asynchronous I/O operations and read data in the network layer to the memory buffer through ByteBuf. The Selector polls the Channel and retrieves these events, sending them to the Channel Pipeline. ChannelPineLine can choose to listen for and process the events it is interested in. It can intercept and process the events through ChannelHandler and propagate them backwards and forwards. Because I/O operations in Netty are asynchronous, events return ChannelFuture execution results. We can use its addListener() method to register listeners and fetch the execution results immediately after the event is triggered.

Why use Netty?

①NIO requires manual control of Selector, Channel, and ByteBuffer.

(2) Using NIO for network programming, can not avoid opening up multithreading, using Reactor model, so it involves

③ Using NIO network programming directly, it is difficult to solve all kinds of abnormal situations in network transmission. For example, sticking and unpacking packages, client disconnection and reconnection, and network congestion.

④NIO Selector empty polling, Netty perfect solution.

⑤Netty is based on extensible and customizable event model and thread model; Netty reduces resource consumption and zero copy minimizes unnecessary memory replication.

EventLoopGroup and EventLoop

EventLoop implements the JUC ExecutorService interface, provides thread management and task queues, and can execute tasks directly. An EventLoopGroup can contain multiple Eventloops. An EventLoop binds a thread, and any I/O events handled by an EventLoop are handled by the thread bound to it. Only one EventLoop can be registered for a Channel. However, an EventLoop can be registered with multiple channels at the same time.

The creation of NioEventloop

Initializing NioEventLoopGroup:

ThreadPerTaskExecutor = ThreadPerTaskExecutor = ThreadPerTaskExecutor

Create CPU*2 threads using DefaultThreadFactory and name each thread individually. Then start each thread.

Loop create NioEventLoop (equivalent to master/slave thread) :

Execute the constructor of parent SingleThreadEventLoop to create the MpscQueue task queue and bind it to ThreadPerTaskExecutor! Initialize the SelectorProvider in the NioEventLoop, and use the SelectorProvider to create and retrieve the native Selector. Then through the optimization option of flag to determine whether to optimize the Selector [fantasy linkage, I am directly using the native Selector], if not optimized, directly return; If optimized, reflect the optimized Selector provided by Netty, initialize SelectedSelectionKeySet instead of the native SelectionKeySet. It can automatically expand capacity and eliminate complicated remove.

(3) Create an Executor selector EventExecutorChooser (used to select master and slave threads for tasks) :

Netty uses factory mode to choose to create a power Chooser or a regular Chooser by determining whether the number of TaskExecutors currently required is a power of 2. The regular Chooser is the normal mod selection (increment % length). The power Chooser is selected by the & operation (incrementing with & Length-1). The Netty server uses a single Selector and a single Executor.

NioEventLoop start

Netty encapsulates the channel binding NioEventLoop as a task and starts the NioEventLoop (boss Thread) to execute the task asynchronously (NioEventLoop is itself an Executor). Because channel.bind() is asynchronous and takes time, NioEventLoop determines whether the bind was successful first. If not already bound, it will be put into the future. listener listener and notified when the Future is complete. When the task arrives, NioEventLoop places the task in its own MpscQueue and binds the currentThread currentThread to NioEventLoop, The nioEventloop.run () method is then executed (NioEventLoop is itself an ExecuteService).

Execute NioEventLoop execution

Select (oldWakenUp, oldWakenUp, oldWakenUp, oldWakenUp)

When the select() polling time exceeds the specified timeout and the selector is unawakened (oldWakenUp==false to prevent missing tasks in the task queue), Execute selectNow() non-blocking wake up and execute (subsequent executions pass oldWakenUp=false again. This also reduces the number of times after select.wakeup(), improving efficiency), and then resets the counter. Continue with the next poll.

Otherwise, if the time difference exceeds the execution time of a select, then selectNow has been executed and the counter is reset. Continue with the next poll

Otherwise, judge selectKeys! WakeuP =true means that the user wakes up actively, there is a task in the task queue, and the scheduling time of the first task is up. The next poll continues.

Otherwise, the current time >=select() + select() block time, indicating that empty polling occurred! Counter +1. RebulidSelector () reproduces the selector if the counter exceeds 512 times.

RebuildSelector () : Creates a new Selector, transfers the events of interest to the new Selector, and then closes the old Selector. (Empty trap BUG: When Selector. Select () blocks polling ready events, it can be woke up in some cases even if no event is ready, and the selectedKeys are empty and cannot access the if block, so in the source code it will jump straight to the bottom of the method, causing an infinite loop.)

ProcessSelectedKeys () : iterating through the selectedKeys and getting their attachment ), then retrieve the ready set of OPS operations, & determine the type of operation and perform the unsafe operation on the data in the channel.

③ Process the task queue and eject the task execution in the queue

Selector Operation type

OP_ACCEPT operation type:

Attachment gets ServerSocketChannel and calls Accept () to get socketChannel. Bind SocketChannel to a selected NioEventLoop in the Worker Group and register OP_READ to the Selector of the NioEventLoop.

OP_READ Operation type:

If the attachment receives more than zero data, the attachment receives data. A 1024-byte ByteBuf is assigned to the Channel to receive data and propagate it to the Pipeline. If ByteBuf is full, the read/write pointer moves and continues receiving. After the data is received, the data size is recorded and the capacity of ByteBuf is adjusted for the next OP_READ.

If the attachment receives less than zero data, it closes the attachment. Clears SelectorKeys on the Selector of the current NioEventLoop and discards all waiting tasks in the task queue. Close the channel.

OP_WRITE Operation type:

Write data from ByteBuf to a channel, stopping when it reaches a threshold. Write data for 16 consecutive times. If the write operation is not complete after 16 times, a write task is scheduled to perform the write operation. By default, the Handler node writes data using writeAndFlush(). Write once triggers a Flush system call with low throughput. So we can turn on asynchronous write enhancement. Netty will specify how many Flush operations are performed per Write, and schedule Flush operations until the number of Write operations reaches the specified number.

Channel

Channel In order to support arbitrary switching between different protocols and DIFFERENT I/O models (I don’t support switching protocols and I/O models, but I do support transformation models based on Nio-socket programming), Netty abstractions a top-level parent class, AbstractChannel, Different subclasses channel correspond to connections of different protocols and I/O models.

The Channel to create:

Explicitly incoming NioServerSocketChannle. Class, reflection created by ChannelFactory NioServerSocketChannel:

① the SelectorProvider opens the ServerSocketChannel and returns a ServerSocketChannel object

(2) Create ChannelID, Unsafe object, ChannelPipeline, ChannelConfig object, set readInterestOp to selectionkey. OP_ACCEPT, and set channel to non-blocking state.

Channel initialization:

Configure the option parameters (mainly TCP attributes) and user-defined parameters set by BootStrap into the ChannelConfig object. Get the ChannelPipeline object in the ServerSocketChannel and add the ChannelInitializer initializer to the end of the ChannelPipeline. After the initializer is added successfully, the callback is triggered to iterate over all ChannelHanderContext for the following handlerAdded, Then delete the ChannelInitializer initializer (this adds all handler nodes to the pipeline responsibility chain!). .

The Channel is registered

Register NioServerSocketChannel with the event poller Selector in EventLoop (underlying call channel.register(Selector,Ops)). HandlerAdded events and channelRegistered events are then fired on the ChannelPipeline, triggering the Handler Handler to add events and channel binding events.

The binding

Use channel.bind(port), then fire the channelActive event in the ChannelPipeline, the event propagated to a node will be executed, Set selectionKey interest set to selectionKey. OP_ACCEPT said can receive a new connection (interestOps | selectionKey. OP_ACCEPT, my side is directly binding, Netty is through events + cor).

API

Read () : Reads data from the current channel into the first inbound buffer in the channelPipeline

Write () : Writes data from the channelPipeline to the message sending loop array, and calls Flush () to write data from the loop array to the destination channel. ]

ChannelPipeline and ChannelHandlerContext

Each Channel created will be assigned a new Channel pipeline, which has the idea of local serial and overall parallelism, better than one queue + multiple threads. ChannelPipeline is a bidirectional linked list composed of ChannelHandlerContext parent class nodes. The first and last of this list are the two default subclasses HeadContext and TailContext. ChannelHandler (ChannelInboundHandler) the inbound data handler handles the reads, namely socket.read (); ChannelOutboundHandler The outbound data handler receives the I/O request, which is processed by the chain of responsibility and written out through the socket.write () method. . HeadContext is an outbound type, which is mainly used to propagate read data streams down. TailContext is an inbound type that handles the end of the read data stream.

Each state change (registration, binding, or cancellation) of a Channel generates an event, which is processed by the channelInboundHandler and then forwarded to the ChannelHandler that follows, returning a ChannelFuture asynchronous result.

Pipeline node add:

The first check is whether the user has been added repeatedly (Netty checks whether the user has been added by @sharable annotation and added attribute on handler). Then put the Handler encapsulated into a new node AbstractChannelHandlerContext and join in the Pipeline. If the Channel is registered with the EventLoop and the Executor is registered with the EventLoop, then a Task callback Task is added and the node callback method is invoked after the Channel is registered. If both are registered, the callback method after adding the node is directly called. Different Contexthandlers have different implementations of the callback method.

Pipeline node deletion

Logical add with node.

Pipeline event propagation:

A read event for a channel is emitted, which is propagated from the tail node of the ChannelPipeline (a write event is propagated from the head node). The ChannelHandlerContext gets the event message and EventExecutor, and then executes its channelRead() method. So our custom ChannelHandler to inherit SimpleChannelInboundHandler, rewrite channelRead () method.

ByteBuf:

Blog.csdn.net/zs319428/ar…

ByteBuf controls byte arrays by maintaining read and write indexes, which control only read access and write indexes control only write access, eliminating the need for flip() read-write mode conversion of native ByteBuffer.

ByteBuf has three modes:

(1) Heap Buffer mode, which puts data into the JVM’s Heap memory, can provide fast access to arrays; System calls to the socket buffer(unpooled.buffer ()) are required to perform IO operations.

The Direct Buffer mode is cleaner().clean(). The Direct Buffer mode is cleaner().clean().

Composite Buffer: A Composite view that combines multiple BytebuFs to avoid copying data between bytebuFs

Zero copy at the Netty application layer

1. Use a Composite Buffer:

1) Unpooled. WrappedBuffer. (ByteBuf… Bufs/byte[] array) : Wraps the byte array into an UnpooledHeapByteBuf object, returning a new instance of ByteBuf with its own read/write index, but sharing the same memory space as the original ByteBuf object.

② bytebuf.slice () : Splits a byteBuf into multiple BytebuFs that share the same storage area.

Filechannel.tranferto () : Sends the file buffer data directly to the target channel, avoiding the memory copy caused by the traditional write loop

If you forget to release the requested ByteBuf, Direct Buffer OOM will be generated. Therefore, ByteBuf needs to be released manually

Outside a heap: freeDictBuffer (buffer)

In pile: recyclerHandler. Recycle (this)

Why use Netty instead of NIO

Supports common application layer protocols

Netty encapsulates ByteBuf as a data accumulator to temporarily store data (ByteBuf appends data and automatically expands data). During data encoding, Netty determines the length of the data accumulator and the known message length. The length will be coded only if the conditions are met, which solves the problem of sticky and unpacked packages. (This data codec is byte specific operation, Netty also supports Protobuf secondary codec, for byte to object conversion)

Traffic integer is supported

Perfect disconnection, idle, etc

Avoid empty polling Bug

Encapsulates ByteBuf to avoid resetting Pointers that cannot be extended.

  1. Locks in Netty:
  • Use CountDownLatch instead of wait/notify. The latter need a monitor?
  • NioEventLoop is multi-producer single-consumer. For taskQueue, LinkedBlockingQeueu is inefficient. So Netty uses MpscQueue.
  • Netty’s pipeline processing is local serial + overall parallel mode is better than one queue + multiple threads mode
  • Using threadLocal
  1. Netty reduce memory usage tips (/ reduce full GC)
  • Use primitive types instead of wrapper types
  • Class variables are superior to instance variables
  • Estimated when allocating memory
  • Dynamically adjust the size of the next buffer to be allocated based on the data received

heartbeat

IdelStateHandler

Source: blog.csdn.net/u013967175/…

Schedule scheduled tasks for the task queue corresponding to the EventLoopExecutor to determine whether the read and write of a channel is idle and times out. Timeout triggers a timeout event to the ChannelPipeline, which executes the custom userEventTriggered() method to send a custom heartbeat packet to maintain the connection.

Disconnect and reconnect:

Source: blog.csdn.net/u010889990/…

If the client triggers userEventTriggered(), the client will send a customized heartbeat packet to prevent the server from triggering read timeout. (Of course, the client must check whether the heartbeat packet can be sent successfully. If the heartbeat packet fails, the connection will be closed.)

If the client does not send customized heartbeat packets, the server read times out. The server then compares the client’s read time with the server’s write timeout. If the read time difference is small, it indicates that the connection is normal. The client may be processing large files in GC and has no time to send heartbeat packets. The channel connection will not be closed if the client read times out

(2) The channelInactive() method in Netty is triggered by waving packets when the Socket connection is closed. Therefore, the channelInactive() method can sense normal offline status, but cannot sense abnormal offline status due to network exceptions. Therefore, we also need to wait for three times the heartbeat time of the heartbeat mechanism to receive no heartbeat (in case the service is in GC, etc.). If there is a heartbeat, we need to reconnect() the client.

Netty tuning

Parameter tuning

  • If too many file handles are opened, too many open files are reported. In Linux, run the ulimit -n [XXX] command.
  • SokcetChannel. ChildOption (TCP_NODELAY, true) : open the Nagle algorithm
  • Serversocket. bind(ADDR,SO_BACKLOG) : Specifies the maximum number of waiting connections for the server
  • Shared: SO_ReuseAddr Address reuse: If the NIC is bound to the same port, the Address is already in use. This parameter allows the port to be used earlier for closed connection release. The idea is to shorten the TCP four-wave timewait, allowing the port to close the connection completely earlier.

Using the optimization

  • Sharable: Indicates whether the Handler can be shared. If so, the Hnadler cannot be added to the Pipeline repeatedly.
  • Complete the thread name: Pass new DefaultThreadFactory(” XXX “) in the EventLoopGroup constructor parameter
  • Improved Handler name: Handler construct parameters can be passed names.