Netty is introduced

Problems with native NIO

  1. NIO class library and API complex, use more trouble, need to master the use of Selector, ServerSocketChannel, SocketChannel, ByteBuffer and other API.
  2. Additional skills are required: Java multithreading, Reactor patterns, and a strong familiarity with multithreading and network programming to write high quality NIO programs.
  1. The workload and difficulty of development are very large: for example, clients face reconnection, intermittent network disconnection, half-packet read and write, failure cache, network congestion and abnormal flow processing, etc.
  2. JDK NIO BUG: Epoll BUG, which causes Selector to poll empty, eventually causing the CPU to 100%. Until JDK1.7 this problem still exists and has not been fundamentally resolved.
According to the JDK NIO API, the Selector select() method blocks until the IO event completes or times out. However, on Linux, this can sometimes be problematic. In some cases, the select() method returns directly, even if there is no timeout and no I/O events arrive. This is known as the Epoll bug. This is a serious bug that causes threads to go into an infinite loop, which can cause the CPU to spike up to 100%, greatly affecting system reliability. None of the JDK has fully addressed this issue.Copy the code

An overview of the

Netty is a Java open source framework provided by JBoss. Provides an asynchronous, event-driven network application framework for rapid development of high performance, high reliability network IO applications. Netty is a network programming framework based on NIO. Using Netty can help you develop a network application quickly and easily, which is equivalent to simplifying and streamlining NIO development process. As the most popular NIO framework, Netty has been widely used in the Field of Internet, big data distributed computing, game industry, communication industry and so on. Well-known Elasticsearch and Dubbo frameworks all use Netty inside.

Netty’s strengths: zero-copy, extensible event model; Supports protocols such as TCP, UDP, HTTP, and WebSocket. Provides secure transmission, compression, large file transfer, codec support and so on.

Has the following advantages:
  1. Elegant design, providing blocking and non-blocking sockets;
  2. Provide flexible and extensible event models;
  3. Provide a highly customizable threading model;
  4. With higher performance and throughput, zero-copy technology is used to minimize unnecessary memory replication and reduce resource consumption.
  1. Provide secure transmission features;
  2. Support multiple mainstream protocols; Preset a variety of codec functions, support user development of private protocol.

Threading model

An overview of the

Different thread mode, the performance of the program has a great impact, before learning Netty thread mode, first understand each thread mode, finally look at the Netty thread model has what advantages.

The current thread models are:

  • The traditional blocking I/O service model
  • Reactor model: There are three typical implementations, depending on the number of reactors and the number of resource pool threads to process
    • Single Reactor Single thread
    • Single Reactor multithreading
    • Master slave Reator multithreading

Traditional blocked I/O service model

Using blocking I/O mode to obtain input data, each connection requires a separate thread to complete the data input, business processing and data return work.

There are problems with this model:

  • When the number of concurrent requests is large, a large number of threads are created, consuming a large amount of system resources.
  • After the connection is created, if the current thread has no data to read temporarily, the thread will block in the read operation, resulting in a waste of system resources.

Reactor model

Reactor pattern: A pattern in which one or more inputs are sent to the server processor at the same time. The server program processes incoming requests and dispatches them simultaneously to the appropriate processing thread. Reactor pattern is also called Dispatch pattern.

The Reactor model uses IO to reuse listening events and distribute them to a thread (process). This is the key to high concurrency in the network server.

Here’s a look at Reactor’s three classic models:

Single Reactor Single thread

  1. Selector enables an application to listen for multiple connection requests through a blocking object.
  2. The Reactor object monitors client connection events through Selector and dispatches them when received.
  1. A connection is established, an event is requested, the Acceptor processes the connection request, and a Handler object is created to handle subsequent business processing after completion.
  2. Handler completes the Read-> business process ->Send to complete the business process.
Advantages:

Simple model, no multithreading, process communication, competition and other problems, all in a thread to complete

Disadvantages:
  1. Performance problems: Only one thread can not fully use the performance of a multi-core CPU. When a Handler processes services on a connection, the entire process cannot process other connection events, which leads to performance bottlenecks.
  2. Reliability problems: The thread terminates unexpectedly or enters an infinite loop. As a result, the communication module of the entire system is unavailable and cannot receive and process external messages, resulting in node faults.

Single Reactor multithreading

  1. The Reactor object monitors client request events through Selector and dispatches events when received.
  2. If a connection request is established, the right Acceptor processes it by Accept.
  1. If it is not a connection request, the Reactor distribution calls the connection to the due Handler.
  2. Handler only responds to events and does not perform specific service processing (only read and send operations). After reading data through read, it is distributed to a thread in the subsequent Worker thread pool to process services.
  1. The Worker thread pool allocates independent threads to complete the real business and returns the results to the Handler.
  2. After receiving the response, the Handler sends the result to the client.
Advantages:

Can take full advantage of multi-core CPU processing power

Disadvantages:

Multithreaded data sharing and access are complicated, Reactor processes all the monitoring and response of events, and performance bottlenecks are easy to occur in the case of single thread operation and high concurrency.

Reactor is multithreaded

  1. Reactor Mainline The MainReactor listens for client connection events through select(). Upon receiving the events, the MainReactor processes the client connection events through acceptors.
  2. When an Acceptor processes a client connection event (establishing a Socket connection with the client), the MainReactor assigns the connection to the SubReactor. (That is, MainReactor only listens for client connection requests, and sends the connection to SubReactor to listen for SUBSEQUENT I/O events.)
  1. The SubReactor adds connections to its connection queue for listening and creates handlers to handle events.
  2. When a new event occurs on the connection, the SubReactor calls the corresponding Handler.
  1. Handler uses Read to Read request data from the connection and distribute the request data to the Worker thread pool for business processing.
  2. The Worker thread pool allocates separate threads to do the actual business processing and returns the results to the Handler. The Handler sends response data to the client using Send.
  1. A MainReactor can correspond to multiple subreactors, that is, a MainReactor thread can correspond to multiple subreactors threads.
Advantages:
  • The data interaction between the MainReactor thread and the SubReactor thread is simple and has clear responsibilities. The MainReactor thread only needs to receive new connections, and the SubReactor thread completes the subsequent business processing.
  • The data interaction between the MainReactor thread and the SubReactor thread is simple, the MainReactor thread only needs to transfer the new connection to the SubReactor thread, and the SubReactor thread does not need to return data.
  • Multiple SubReactor threads can handle higher concurrent requests
Disadvantages:
  • The disadvantage of this pattern is high programming complexity. However, due to its obvious advantages, it is widely used in many projects, including Nginx, Memcached, Netty and so on.
  • This pattern is also known as the 1+M+N threading pattern for servers, which means that a server developed using this pattern has one (or more) connection establishment threads, +M IO threads, and N business processing threads. This is a mature server programming pattern in the industry.