Redis, Nginx, Tomcat, JavaNio and Netty all use multiplexing to improve performance. However, this part involves kernel or hardware knowledge such as operating system nic, which leads to my understanding is always in the cloud. Therefore, I have to make sure to understand this thing. Otherwise, it is not impossible to realize my dream of squatting in the house! Select, poll, epoll.

I read this series of articles to clarify my own thinking:

  • zhuanlan.zhihu.com/p/63179839
  • zhuanlan.zhihu.com/p/64138532
  • zhuanlan.zhihu.com/p/64746509

1. Basic socket receiving flow

    1. After receiving the data from the network cable, the network adapter transmits the data through the hardware circuit and stores the data to an address in the memory for the operating system to read.
    1. After the data is written to the memory, the nic sends an interrupt signal to the CPU (the CPU will preferentially respond to the interrupt signal and run an interrupt program for the NIC to remove the corresponding interrupt signal of the NIC).
    1. When a request is received, a socket object is created, which is similar to a file descriptor and is handed over to the file system for management (so you can check the file descriptor occupied by the process to see if there are too many sockets). The socket contains a send buffer, a receive buffer, and a wait list. After the connection is established, the listening SocketServer process is moved from the work queue of the kernel space to the wait queue of the socket, blocking the listening process. Wait for the data to arrive.

2. recv

With recv, after the socket receives data, the operating system removes the socket from the wait queue and puts it back into the work queue of the kernel space. In this case, the recV method returns the received data.

The disadvantage of recV is that it is single-threaded. It can only monitor one socket and process a socket connection before processing the next one.

3. select

The select model can handle monitoring of multiple sockets, which is actually monitoring multiple sockets, but processing is also handled in the same thread.

His implementation idea is actually relatively simple, directly use an array to store all sockets, and then once any socket received data, notify the monitoring process, and then the process will traverse all sockets, read data processing.

Call recV and call SELECT corresponding interrupt program content is not the same, when there is data received, the network adapter will send different interrupt program, wake up the listener process to join the work queue of the kernel, and remove all waiting queue listener process in the socket.

The select model implements listening on multiple sockets, but there is a performance penalty:

  1. When a select call is made, the listener process is added to the wait queue of all sockets, which is traversed once.
  2. When there is data in any socket, the listener needs to be removed from all socket waiting queues. This is the second traversal.
  3. When the listener process is executed by the CPU, it needs to traverse all sockets to determine which socket received data. This is the third traverse.

Therefore, there are three times of traversal. Therefore, the maximum number of sockets monitored at the same time cannot be too large.

4. epoll

Epoll solves the continuous traversal of the select process, of course, there are corresponding interrupt procedures. It’s just that the interrupt program operates on different objects.

Here, epoll_create() tells the kernel to create an eventPoll object, epFD above, which is also managed by the file system and also contains a wait queue.

The eventPoll object maintains a wait queue for all the sockets that need to be monitored.

When the program runs to epoll_WAIT and needs to fetch data from the socket, the process will be blocked waiting for data to come, and the process reference will be added to the wait queue of EventPoll to remove the process from the work queue.

When a socket receives incoming data, it sends an interrupt to the CPU. The CPU processes the interrupt, which on the one hand adds the socket to the ready queue and on the other hand wakes up the eventPoll waiting process to execute in the work queue again. So the program can directly know which sockets are ready and can read data without having to iterate over all the sockets.

So ePoll separates the maintenance of the socket list from process blocking through the object of EventPoll.

  • By creating a watch list and a ready list of sockets, the process avoids traversing all sockets and can read data directly from the ready queue socket.
  • By creating a unified wait queue, it avoids the need for processes to be added to all socket wait queues.