This article should take about 10 minutes to read.

“Recently, the author is studying the micro-service gateway, and is going to write a micro-service gateway of service forwarding for data proxy and forwarding, so as to better serve his own engineering services. Using nginx, Kafka, Redis network model and Linux, we found that we can’t get around the IO reuse model.”

The system can only use Linux sauce, in fact, there are many IO models.

Blocking I/O model Non-blocking I/O model I/O multiplexing model Signal driven I/O model Asynchronous I/O modelCopy the code

Let’s talk about it purely todayI/O multiplexing model

Take an example: consider a problem, we usually work with several people (threads), but if the rapid development of the current business, is it necessary to increase the number of people (threads)? We all know that hiring people costs a lot of money (consumes a lot of resources), which can result in outgoings exceeding costs (insufficient system resources).

Is there a way to have one person serve multiple upstream and downstream connections?

There is reuse (IO reuse technology), is the best answer to the above things.

To better understand IO reuse, let’s use an example – “Chasing girls” in college.

Here’s a joke:

Let’s say you’re chasing girls, your goddess lives in a room on the 24th floor, and the lady at the door doesn’t let boys into the building. You small deer disorderly bump, to see one side of your goddess more, below the circumstance of not breaking the law, can wait in the doorway all the time ah wait, to each girl all want careful observation 👀, be afraid to miss. But you also get sleepy, you get thirsty, you get tired. What to do? You can’t miss the goddess. As you struggle with your brain, you think that I am definitely not a person who likes it. Let’s stare at the goddess and tell each other that you agree to stare at the goddess at the door of the 24th floor. If you take the above boys as servers and girls as clients, then this is an example of a thread.

Then one day, you think – “I want to see my goddess efficiently”, and when she shows up at the door, you just have to sneak in (without telling anyone else) to accompany her. If you see your goddess, you will tell you through wechat. Then you will put down what you are doing and go to the 24th floor to find your goddess in high spirits. Now this is a case of IO reuse.

At present, there are three common I/O multiplexing models: SELECT, Poll, and epoll.

You chase the goddess mode -select

When you use this mode after the goddess, a girl (I/o events) in and out of the dormitory, you don’t know which is your goddess, you can only be indiscriminate each observation 👀 all girls, you are very clever, every girl will take photos (array storage cap) than carefully, and find out your goddess, you chat with her. So you have very poor time complexity, very low efficiency o (n), and the more people there are at this door, the longer you watch each one.

You chase the goddess mode -poll

When you use this mode when the goddess of the chase you, will you take pictures the transfer of data to do the position (real-time import from your phone to your computer – array copy into the kernel space From an array into a linked list, there is no limit to the storage), and then image using the artificial than you smart (query each fd corresponding state), You’re still essentially at o (n) efficiency as above.

Epoll (Event Poll)

Different from the select, poll two modes, dormitory aunt is important at this time, she helps you monitor your goddess “your goddess can have multiple” (events) action (what kind of I/O event happened which flow), aunt will give photo a photo of your goddess, sent to your computer (fd) on each event correlation. At this point you receive a message from the dormitory administrator aunt, immediately rushed to the dormitory on the 24th floor. So the time is down to order one.

summary

Select, poll, and epoll are all I/O multiplexing mechanisms. IO multiplexing is a mechanism that monitors multiple goddesses (descriptors), notifies you as soon as one of your goddesses appears (a resource descriptor is read/write ready), and then you follow up on a series of quests based on your goddesses’ preferences (notifies other programs to read/write). However, in essence, select poll epoll are synchronous I/O operations, which are relatively specialized (that is, they are only responsible for the read and write state that they care about). The whole process cannot be in parallel (each resource needs to be blocked synchronously)

Does that mean cheating on philandering men (an efficient procedure)? I’m glad. I told you there was. For example, the girl who messes with herself is handed over to a multi-personality to be processed through various compliance operations (asynchronous I/O) (the implementation of asynchronous I/O copies data from the kernel to user space).

Well that’s the end of the joke.

thinking

One question, how does an operating system operate to support IO multiplexing? Both Epoll and Select provide a solution for multiplexing multiple I/O. Epoll is unique to Linux, and SELECT is required by POSIX, which is implemented by common operating systems.

Synchronous blocking (BIO)

  • The server uses a single thread. When a request is accepted, the recV or SEND call is blocked, and other requests cannot be accepted (the recV or send call must wait for the last request to finish).Unable to handle concurrency
  • The server uses multithreading, when accept a request, open thread recV, can complete concurrent processing, but with the increase of the number of requests need to increase the system thread,A large number of threads take up a lot of memory space, and thread switching will bring a lot of overhead. The number of threads that actually have read and write events in 10,000 threads will not exceed 20%. It is also a waste of resources to open one thread for each accept

Synchronous non-blocking (NIO)

  • When the server accepts a request, it adds FDS (FD in full)file descriptor) collection, polling FDS collection recV (non-blocking) data every time, returning an error immediately if there is no data,Polling all FDS (including those with no read/write events) at a time can be a waste of CPU

IO multiplexing

  • The server uses a single thread to obtain the FD list through system calls such as SELECT /epoll, and traverse the FDS with events to accept, recv, and send to enable themSupport for more concurrent connection requests

In-depth technology:

Select:

Select was the first implementation (implemented in BSD around 1983), and when select was implemented, it quickly exposed a lot of problems

  • Select will modify the array of parameters passed in, which is very unfriendly for a function that needs to be called many times. In other words, it is not thread safe.
  • Select if I/O data, will only return, will not tell you who is the data returned above, this time you want to know who, you have to go through once, dozens of does not matter, hundreds of very trivial
  • Unfortunately, select can only listen on 1024/2048 links, which is the default.
  • Select is not thread safe, as mentioned above, sock is not used, congratulations, I refuse, I am not happy not allowed to recycle. You don’t have to use me, but I’m gonna make a mess for you.

“If a file descriptor being monitored by select() is closed in another thread, the result is Unspecified.”

Poll:

Later to do select improvements, made a poll out, how many links, you say, happy.

Storage has been changed from arrays to linked lists.

Horizontal triggering is also supported. If the fd is not processed, the next poll will report the FD again.

With the development of The Times and the progress of society, computer processing efficiency is getting higher and higher. The original design looks a little backward now. A thousand links was a good number in those days, but not today. Poll still retains the fatal drawback of select, which is thread unsafe.

Epoll:

Epoll was implemented by Davide Libenzi in 2002 (during the performance crisis years). Now I am safe, I can not only tell you today which SOCK has data, but also tell you which sock has data, you don’t have to look for it, how nice.

This is a pressure map that compares poll epoll performance in detail and shows that this happens when the number of connections sent increases with the number of requested links, resulting in a decrease in processing speed.

Performance reference

Epoll has two trigger modes: EPOLLLT and EPOLLET. LT is the default mode, and ET is the high speed mode.

By default, epoll_wait returns its event each time the fd has data to read, reminding the user to do so. In high speed (edge-triggered) mode: it only prompts once and will not prompt again until the next data influx, regardless of whether there is still data to read in the FD.

Therefore, in high-speed mode, when reading a fd, the buffer must be read until the return value of read is less than the requested value or an EAGAIN error occurs. Another feature is that epoll uses “event” ready notification, registering a FD with epoll_ctl. Once the FD is ready, the kernel uses a callback-like callback mechanism to activate the FD and epoll_wait is notified.

Epoll design advantage message passing (similar to go’s two-pole threading model – splitting the user and kernel states) Epoll is implemented by the kernel and user space sharing a single piece of memory. The storage overhead of sock is about 1GB, and the mmap() file is used to map the memory to accelerate the message passing with the kernel space

To sum up:

Select should be POSIX and generally implemented by operating systems.

    selectNot thread-safe,selectCan only monitor1024A linkselectIf any of the SOCK (I/O stream) data is present,selectIt will only return data, but it will not tell you which sock has the data, so you have to search for it one by oneselectModifies the array of parameters passed in, which for a function that needs to be called many timesCopy the code

Poll is similar to SELECT, but it does not solve the problem of efficiency (time, waste of resources), so it was replaced by the later evolution of the Epoll model.

Poll to remove1024Poll By design, poll does not modify the array passed in, but poll is still not thread-safeCopy the code

Epoll is unique to Linux, and macOS systems have a particularly interesting implementation, KQueue, that implements the epoll-like model

Epoll is now thread-safe. Epoll now not only tells you about the sock group, but also which sock has data, so you don't have to find it yourself.Copy the code

In fact, it is not necessary to use the epoll model when your request volume is really not very large, but you need to design a high-performance service forwarding gateway (a bit pursuit), just like Pan Jianfeng’s GNET.

The last

Do you know that Nginx, Kafka and Redis systems with high IO operations are IO reuse models? There is still a long way to go, take your time to watch and learn. I still vaguely remember that my colleagues used to chase girls with epoll model and marveled at his 🐮.

Reference: Operating system – IO multiplexing principle analysis – Zhuang Xiaoyan

Nginx

Reference: Industry Press – Computer operating system

Reference: the difference between SELECT, poll, and epoll in IO multiplexing