preface

When we use Netty for server development, we typically define two NioEventLoopGroup thread pools, a “bossGroup” thread pool for client connections and a “workGroup” thread pool for read and write operations. So why do we do this? What are the benefits of this? Can you use just one NioEventLoopGroup? That’s the subject of today’s discussion — Netty’s threading model

Reactor thread model

In fact, the Netty threading model is an implementation of the Reactor model, and what is the Reactor model?

The Reactor model is based on event-driven development. Its core components include Reactor and thread pool, wherein Reactor is responsible for monitoring and allocating events, and thread pool is responsible for processing events. According to the number of Reactor and thread pool, Reactor is divided into three models:

  • Single-thread model (Single-reactor single-thread)
  • Multithreading model (SINGLE Reactor Multithreading)
  • Principal/Slave Multithreading model (Reactor Multithreading)

Single threaded model

Pictures from: gee.cs.oswego.edu/dl/cpjslide…

  • Connection events are monitored internally by selector, and dispatched via Dispatch. If a connection is established, it is processed by Acceptor, which accepts the connection through Accept. And create a Handler to handle the subsequent events of the connection. If it is a read/write event, call the Handler corresponding to the connection to handle it
  • Handler completes the read->(decode->compute->encode)->send business process
  • The advantage of this model is simplicity, but the disadvantage is obvious. When a Handler blocks, both the Handler and Accpetor of other clients cannot be executed, thus achieving high performance. This model is only applicable to scenarios where business processing is very fast

Multithreaded model

Pictures from: gee.cs.oswego.edu/dl/cpjslide…

  • In the main thread, the Reactor object monitors connection events using a Selector, dispatches the event through Dispatch, and if it is a connection establishment event, it is processed by an Acceptor, which accepts the connection and creates a Handler to handle subsequent events. The Handler is only responsible for responding to events, but does not carry out service operations, that is, only reads data and writes data. The service processing is handed over to a thread pool for processing
  • The thread pool allocates a thread to do the actual business processing, and then passes the response to the main process’s Handler, which sends the result to the client (here’s the core code).

A single Reactor is used to monitor and respond to all events. However, when a large number of clients connect at the same time, or perform some time-consuming operations, such as identity authentication, permission checking, etc. when requesting connections, the instantaneous high concurrency will easily become a performance bottleneck

Master-slave multithreaded model (most popular)

Pictures from: gee.cs.oswego.edu/dl/cpjslide…

  • There are multiple reactors, each with its own selector selector, thread, and dispatch
  • The mainReactor in the main thread monitors connection establishment events through its selector, receives the event through Accpetor, and assigns a new connection to a child thread
  • The subReactor in the child thread adds the connection assigned by the mainReactor to the connection queue to listen through its selector and creates a Handler to handle subsequent events
  • Handler Completes the complete business process of read-> Service processing -> Send

The connection between the threading model in Netty and Reactor

Netty mainly relies on NioEventLoopGroup thread pool to implement the specific thread model

Single threaded model

The single-thread model is to specify only one thread to perform client connection and read/write operations, which is completed in a Reactor. The corresponding implementation in Netty is to set the number of NioEventLoopGroup threads to 1. The core code is as follows:

 NioEventLoopGroup group = new NioEventLoopGroup(1);
        ServerBootstrap bootstrap = new ServerBootstrap();
        bootstrap.group(group)
                .channel(NioServerSocketChannel.class)
                .channel(NioServerSocketChannel.class)
                .option(ChannelOption.TCP_NODELAY, true)
                .option(ChannelOption.SO_BACKLOG, 1024)
                .childHandler(new ServerHandlerInitializer());
Copy the code

Its workflow is roughly as follows:

The above single-threaded model corresponds to Reactor’s single-threaded model

Multithreaded model

Multithreading model is to process client connection in a single Reactor, and then transfer the business process to the thread pool. The core code is as follows:

NioEventLoopGroup eventGroup = new NioEventLoopGroup();
ServerBootstrap bootstrap = new ServerBootstrap();
bootstrap.group(eventGroup)
        .channel(NioServerSocketChannel.class)
        .option(ChannelOption.TCP_NODELAY, true)
        .option(ChannelOption.SO_BACKLOG, 1024)
        .childHandler(new ServerHandlerInitializer());
Copy the code

BossGroup and workerGroup use the same group

@Override
public ServerBootstrap group(EventLoopGroup group) {
    return group(group, group);
}
Copy the code

The working process is as follows:

Master-slave multithreaded model (most commonly used)

The master-slave multithreading model has multiple reactors, that is, there are multiple selectors, so we define a bossGroup and a workGroup, and the core code is as follows:

NioEventLoopGroup bossGroup = new NioEventLoopGroup();
NioEventLoopGroup workerGroup = new NioEventLoopGroup();
ServerBootstrap bootstrap = new ServerBootstrap();
bootstrap.group(bossGroup,workerGroup)
        .channel(NioServerSocketChannel.class)
        .option(ChannelOption.TCP_NODELAY, true)
        .option(ChannelOption.SO_BACKLOG, 1024)
        .childHandler(new ServerHandlerInitializer());
Copy the code

The working process is as follows:

Note: In Netty, the bossGroup thread pool still only randomly selects one thread to handle client connections, and NioServerSocetChannel is bound to the bossGroup thread. NioSocketChannel is bound to the workGroup thread

summary

The above summarizes three models of Reactor and their corresponding implementations in Netty. In Netty, we mostly use the master-slave multithreading model. The most authoritative source for Reactor’s learning is Scalable IO in Java by Doug Lea for those interested

The resources

  • Gee.cs.oswego.edu/dl/cpjslide…
  • Time.geekbang.org/column/arti…
  • Segmentfault.com/a/119000000…
  • www.infoq.cn/article/net…