Prologue of the preface

The server model involves threading patterns and IO patterns, and knowing these can target a variety of scenarios. The series is divided into three parts:

  • Single-thread/multi-thread blocking I/O model
  • Single-threaded non-blocking I/O model
  • Multithreaded non-blocking I/O model, Reactor and improvements

preface

The server model discussed here mainly refers to the server-side I/O processing model. From different dimensions can have different classification, here from I/O blocking and non-blocking, I/O processing single-thread and multi-thread perspective to explore the server model.

I/ OS can be divided into blocking I/ OS and non-blocking I/ OS. Blocking I/O The current thread is blocked when performing I/O read and write operations, but non-blocking I/ OS are not blocked.

In the case of threads, one thread is responsible for all client connection I/O operations in the case of single threads, while in the case of multithreading, several threads are responsible for all client connection I/O operations.

Multithreaded non-blocking I/O model

The single-threaded non-blocking I/O model has greatly improved machine efficiency, and multi-threading can continue to improve machine efficiency on multi-core machines. The simplest and most natural thing to do is to assign client connections by component to several threads, with each thread handling connections within the corresponding group. As shown in the figure, there are four clients accessing the server. The server hands over socket 1 and socket 2 to thread 1, while thread 2 manages socket 3 and socket 4. Event detection and non-blocking reads and writes allow each thread to process efficiently.

The classic multithreaded non-blocking I/O model is the Reactor model. First, look at the single-thread Reactor, which divides the whole process of the server into several events, such as receive event, read event, write event, execute event, etc. Reactor uses event detection to distribute these events to different processors for processing. As shown in the figure, several client connections access the server side, and the Reactor is responsible for detecting and distributing events to handlers, including the Accept handler that receives the connection, the READ handler that reads the data, the write handler that writes the data, and the Process handler that executes the logic. As long as there are events waiting to be processed, the Reactor thread can continue to execute without blocking somewhere, so the processing is very efficient.

Based on the single-thread Reactor model, it is improved to multi-thread model according to the actual application scenario. There are two common approaches: one is to introduce multithreading into the time-consuming process processor, such as using thread pools; The other option is to use multiple Reactor instances directly, one thread for each Reactor instance.

An improvement of the Reactor pattern is shown in the figure. The overall structure is basically similar to that of a single-threaded Reactor, except that a thread pool is introduced. Since receiving connections, reading data, and writing data are generally less time-consuming, they are all processed in the Reactor thread. However, for work that might be time consuming for logical processing, a thread pool can be introduced into the Process processor, which does not perform the task itself, but hands it off to the thread pool, thereby eliminating time-consuming operations in the Reactor thread. By moving time-consuming operations into a thread pool, the Reactor keeps the Reactor efficient even though it has only one thread.

Another refinement of the Reactor schema is shown in the figure. There are multiple Reactor instances, each of which corresponds to a thread. Since receiving events are relative to the server side, the client connection reception is uniformly handled by one Accept processor. The ACCEPT processor evenly distributes the received client connections to all Reactor instances, and each Reactor instance processes the client connections assigned to that Reactor. This includes read data, write data, and logical processing of the connection. This is how a multiple Reactor instance works.

Multithreaded non-blocking I/O mode gives server-side processing power a big boost, making full use of the machine’s CPU and suitable for high-concurrency scenarios, but it also makes programs more complex and prone to problems.

============= advertising time ===============

The public menu has been divided into “distributed”, “machine learning”, “deep learning”, “NLP”, “Java depth”, “Java concurrent core”, “JDK source”, “Tomcat kernel” and so on, there may be a suitable for your appetite.

My new book “Analysis of Tomcat kernel Design” has been sold on JINGdong, friends in need can buy. Thank you all.

Why to write “Analysis of Tomcat Kernel Design”

= = = = = = = = = = = = = = = = = = = = = = = = =

Welcome to: