IO multiplexing – Overview

In order to talk about multiplexing, of course, still want to follow the trend, using whip body ideas, first talk about the disadvantages of traditional network IO, with the way to hold up multiplexing IO advantages. For ease of understanding, all of the code below is pseudocode, just know what it means.

Blocking IO

The server writes the following code to process the connection and request data from the client.

listenfd = socket();   // Open a network communication port
bind(listenfd);        / / binding
listen(listenfd);      / / to monitor
while(1) {
  connfd = accept(listenfd);  // Block the connection
  int n = read(connfd, buf);  // Block reading data
  doSomeThing(buf);  // What do you do with the data you read
  close(connfd);     // Close the connection and loop for the next connection
}
Copy the code

The code will stumble, with the server thread blocking in two places, the Accept function and the read function.

If we expand out the details of the read function, we see that it blocks in two phases.

1. The nic copies the data sent by the client into the kernel buffer,
2. The kernel buffer sets the associated file descriptor to be readable, copying data from the kernel buffer to the user buffer.

The overall process is shown below.

Therefore, if the client of this connection does not send data, the server thread will block on the read function and cannot accept other client connections.

Non-blocking IO

To solve the above problem, the key is to modify the read function.

A smart approach is to create a new process or thread each time to call the read function and do the business.

while(1) {
  connfd = accept(listenfd);  // Block the connectionPthread_create (doWork);// Create a new thread
}
void doWork(a) {
  int n = read(connfd, buf);  // Block reading data
  doSomeThing(buf);  // What do you do with the data you read
  close(connfd);     // Close the connection and loop for the next connection
}
Copy the code

This way, once a connection is established to a client, it can immediately wait for a new client to connect, without blocking read requests from the original client. However, this is not called non-blocking IO, it’s just a multithreading trick to keep the host thread from getting stuck on the read function. The read function that the operating system provides for us is still blocked.

So real non-blocking IO, not through our user layer tricks, but by begging the operating system to provide us with a non-blocking read function.

The effect of this read function is to return an error value (-1) immediately if no data arrives (arrives on the nic and is copied to the kernel buffer), rather than blocking and waiting.

The operating system provides this functionality by simply setting the file descriptor to non-blocking before calling read.

fcntl(connfd, F_SETFL, O_NONBLOCK);
int n = read(connfd, buffer) ! = SUCCESS);Copy the code

This requires the user thread to loop through read until the return value is no more than -1 before processing the business.

Here we notice two things.

1. Non-blocking READ means that it is non-blocking until the data arrives, either before it reaches the nic or before it reaches the NIC but is not copied to the kernel buffer. When the data has reached the kernel buffer, calling read still blocks and waits for the data to be copied from the kernel buffer to the user buffer before returning.
2. By creating one thread per client, server-side thread resources can easily be consumed.

The overall process is shown below

IO multiplexing

Another clever way is to place the file descriptor (ConnFD) in an array after each client connection is accepted. Then a new thread iterates through the array, calling the non-blocking read method on each element.

fdlist.add(connfd);

while(1) {
  for(fd <-- fdlist) {
    if(read(fd) ! =- 1) {
      doSomeThing(a); }}}Copy the code

But just as we used multiple threads to make blocking IO look like non-blocking IO, this traversal was just a trick we users came up with, and it was still a wasteful system call every time read returned -1.

Making system calls in a while loop is just as uneconomical as making RPC requests in a while when you are doing distributed projects.

So, again, the operating system has to give us a function that does this, and we pass a bunch of file descriptors through a system call to the kernel, and the kernel layer walks through it, to really solve this problem.

select

Select is a system call function provided by the operating system. We can send an array of file descriptors to the operating system. The operating system can iterate over which file descriptors can be read or written, and then tell us what to do:

The function definition for the SELECT system call is as follows.

int select(
    int nfds,
    fd_set *readfds,
    fd_set *writefds,
    fd_set *exceptfds,
    struct timeval *timeout);
// NFDS: increases the maximum file descriptor in the file descriptor set to be monitored by 1
// readfds: monitors the arrival of read data to the file descriptor set, passing in and out parameters
// writefds: monitors the arrival of write data to the file descriptor set, passing in and out parameters
// ExceptfDS: Set of file descriptors to monitor exception occurrence
// timeout: indicates the monitoring time of periodic blocking
// 1.null, wait forever
// 2. Set timeval to wait for a fixed time
// 3. Set the time in timeval to 0, check the description, and return immediately for polling
Copy the code

First, a thread continually accepts client connections and places the socket file descriptor into a list.

while(1) {
  connfd = accept(listenfd);
  fcntl(connfd, F_SETFL, O_NONBLOCK);
  fdlist.add(connfd);
}
Copy the code

Then, instead of iterating itself, another thread calls SELECT, handing the list of file descriptors to the operating system to iterate over.

while(1) {
  // Pass a list of file descriptors to select
  // If there are any ready file descriptors, nready indicates how many are ready
  nready = select(list); . }Copy the code

However, when the select function returns, the user still needs to traverse the list that was just submitted to the operating system.

However, the operating system will identify the ready file descriptor and there will be no meaningless system call overhead for the user layer.

while(1) {
  nready = select(list);
  // The user layer is still traversed, but with fewer invalid system calls
  for(fd <-- fdlist) {
    if(fd ! =- 1) {
      // Read only the ready file descriptor
      read(fd, buf);
      // There are only nready ready descriptors
      if(--nready == 0) break; }}}Copy the code

Several details can be seen:

1. The SELECT call requires passing in the FD array and making a copy of it to the kernel, which can be a huge resource drain in high-concurrency scenarios. (Can be optimized to not copy)
2. Select at the kernel level still checks the ready state of the file descriptor by traversing it, a synchronous process, but without the overhead of system call context switching. (Kernel layer can be optimized for asynchronous event notification)
3. Select simply returns the number of file descriptors that can be read, which one the user must traverse. (Can be optimized to return only user-ready file descriptors without the user doing invalid traversal)

The flow chart of the entire select is as follows.

As you can see, this approach allows one thread to handle multiple client connections (file descriptors) while reducing the overhead of system calls (multiple file descriptors have only one SELECT system call + N ready-to-state file descriptors).

poll

Poll is also a system call function provided by the operating system.

int poll(struct pollfd *fds, nfds_tnfds, int timeout);

struct pollfd {
  intfd; /* File descriptor */
  shortevents; /* Monitor events */
  shortrevents; /* Monitor events that meet the criteria */
};
Copy the code

The main difference from SELECT is that the limit of 1024 file descriptors that select can listen on is removed.

epoll

Remember the three details of select from above?

1. The SELECT call requires passing in the FD array and making a copy of it to the kernel, which can be a huge resource drain in high-concurrency scenarios. (Can be optimized to not copy)
2. Select at the kernel level still checks the ready state of the file descriptor by traversing it, a synchronous process, but without the overhead of system call context switching. (Kernel layer can be optimized for asynchronous event notification)
3. Select simply returns the number of file descriptors that can be read, which one the user must traverse. (Can be optimized to return only user-ready file descriptors without the user doing invalid traversal)

So Epoll mainly improves on these three points.

1. A collection of file descriptors is kept in the kernel, and the user doesn’t need to re-pass it each time, just tell the kernel what has been changed.
2. Instead of polling to find ready file descriptors, the kernel wakes them up with asynchronous IO events.
3. The kernel only returns file descriptors with IO events to the user, and the user does not need to traverse the entire set of file descriptors.

Specifically, the operating system provides these three functions.

The first step is to create an epoll handle

int epoll_create(int size);
Copy the code

Second, add, modify, or delete file descriptors to the kernel to monitor.

int epoll_ctl(
  int epfd, int op, int fd, struct epoll_event *event);
Copy the code

In the third step, a similar select() call is made

int epoll_wait(
  int epfd, struct epoll_event *events, int max events, int timeout);
Copy the code

The specific process is as follows:

• This article reprinted from mp.weixin.qq.com/s/YdIdoZ_yu… Assault, delete