preface
When using Netty to transform the handwritten RPC framework, we need to introduce some relevant knowledge, so that a lot of things we can see, handwritten RPC is a side task, the follow-up focus is still Kubernetes related content.
Blocking vs. non-blocking synchronous vs. asynchronous
Blocking and non-blocking
Blocking and non-blocking are ways of dealing with whether the data is ready when the process accesses it. When the data is not prepared, the process is suspended until the result of the call is returned, and the function will not return until it gets the result. Nonblocking, as opposed to blocking, means that the function does not block the current thread but returns immediately until the result is not immediately available.
Synchronous and asynchronous
Synchronization means that when a function call is made, it does not return until the result is received. That means you have to do one thing at a time and wait until the first thing is done before you can do the next thing. Asynchrony is the opposite of synchronization. When an asynchronous procedure call is made, the caller does not get the result immediately. The part that actually handles the call notifies the caller through status, notifications, and callbacks after completion.
For example, for the B/S architecture that we often use, synchronous and asynchronous refer to requests from clients to access data, and blocked and non-blocked refer to server processes to access data, whether the process has to wait. There are essential differences between the two, and their modifiers are different.
Blocking and non-blocking refers to whether the process needs to wait for the data it accesses if it is not ready. In short, this is equivalent to the implementation difference within the function, that is, whether to return directly or wait for ready when it is not ready.
Synchronous and asynchronous refers to the mechanism, access to the data synchronization generally refers to the active request and wait for the I/O operations to complete the way, when the data is in place in reading and writing must be blocked, asynchronous means after active request data could continue to handle other tasks, then waiting for I/O operations to complete the notification, this can make the process data read and write are not blocked.
For example
Mother let me go to the kitchen to boil a pot of water, ready to dumplings.
Blocking: As long as the water is not boiling, I will stare at the pot, changing with each passing day, I am motionless, the kitchen is my home, boiling water is my destiny;
Non-blocking: I go to my house first dozen king, but every minute, I have to go to the kitchen take a look, for fear that time is long, the water burned dry on the bad, so lead to my game also idea dozen, if not, and dropped a paragraph;
Synchronization: whether every minute to look at the pot, or inch by inch has been looking at the pot, as long as I do not go to see, I do not know the water is not good, a waste of time ah, an inch of time an inch of gold;
Asynchronous: I bought an electric kettle on Taobao, as long as the water opened, it sounded, haiya, can feel at ease to play king, hit can eat dumplings;
Conclusion:
Blocking/non-blocking: What am I doing while I’m waiting for you to work?
4. To do nothing, to die, etc.
Non-blocking: Do something else, but always check in with your progress;
Synchronous/asynchronous: How will you let me know when you’re done?
Synchronization: I just don’t ask, you don’t tell me;
Asynchronous: you finish, directly call me to come over;
IO model
The essence of network IO is to read Socket. Socket is abstracted as stream in Linux system, and IO can be understood as the operation of convection. The standard Linux file access methods are as follows:
When a read operation is initiated, it goes through two phases:
- Wait for data to be prepared.
- Copy data from the kernel to the process;
Socket flows also go through two phases:
- Copy the information after the arrival of disk or other devices into the kernel cache;
- Copy kernel cache data to application process cache;
Network applications need to deal with nothing more than two types of problems, network IO, data computing. Compared with the latter, network IO delay brings more performance bottlenecks to applications than the latter. Next, we introduce the IO model:
Blocking IO
The synchronous blocking IO model is one of the most common and the simplest. In Linux, all sockets are blocking by default. Blocking is when a process rests and the CPU processes another process.
The user-space application makes a system call (recvForm), which causes the application to block until the data is ready and copied from the kernel to the user process, which then processes the data, blocking the entire process between waiting for the data and processing the data. Cannot process other network IO. The calling application is in a state where it is no longer consuming CPU but simply waiting for a response, so this is very efficient from a processing perspective. When recv()/recvfrom() is called, the process of waiting for and copying data occurs in the kernel, as shown below:
-
The application process sends recFROM to the kernel to read data.
-
Prepare datagrams (application process blocked);
-
Move data from the kernel to the application space;
-
After the replication is complete, a success message is displayed.
The characteristics of
Synchronous blocking THE entire IO process is blocked, the user can return data in time, no delay, simple and easy for the developer, the system can not cope with high concurrent access, and the user can not do any other operations during the waiting period.
Synchronizing nonblocking IO
Synchronous non-blocking is the use of polling to periodically check whether the data is ready to complete. In this model, processes are opened in a non-blocking form. IO does not complete immediately and if there is no data in the buffer an EWOULDBLOCK error is returned and the application is not kept waiting.
Non-blocking I/OS also make recvForm system calls to check that data is ready. Unlike blocking I/OS, non-blocking breaks up large blocks for a whole chunk of time into N smaller blocks, so the process constantly has a chance to be accessed by the CPU. That is, the non-blocking recvForm system call does not block, and the kernel immediately returns to the process with an error if the data is not ready.
-
The application process sends recvFROM to the kernel to read data.
-
Return the EWOULDBLOCK error immediately if no datagram is ready;
-
The application process sends recvFROM to the kernel to read data.
-
If the data package is ready, perform the following steps, otherwise return the error code;
-
Copying data from the kernel to user space
-
After completion, a success message is displayed.
The characteristics of
Synchronous non-blocking mode compared to synchronous blocking mode, the process can do other things while waiting for the task, but the disadvantage is that due to the scheduled polling mode, the overall system throughput decreases.
IO multiplexing
Polling takes up a large part of the process, which consumes a lot of CPU time. In concurrent cases, the server may receive tens of millions of requests at a moment. In this case, it needs to create tens of millions of threads to read data. Since application threads do not know when data will be read, they must send recvFROM requests to the kernel to read data. With so many threads constantly calling recvFROM to request data, details are a waste of thread resources.
So someone came up with the idea of a thread that loops to query the completion status of multiple tasks (FD file descriptors) and processes each task as soon as it completes. In this way, only one or several threads are required to complete the data state query operation, and when the data is ready, the corresponding threads can be allocated to read the data. This saves a lot of thread resources. This is IO multiplexing.
-
The application process makes a select call to the kernel.
-
The kernel listens for all the sockets that the select is responsible for;
-
When data is ready in any socket, select returns;
-
The application process then calls recvFROM to copy the data from the kernel to user space.
-
After completion, a success message is displayed.
The characteristics of
In contrast to synchronous non-blocking, an application thread blocks by calling a SELECT /poll poll, and the kernel thread polls the application thread to see if the buffers corresponding to all file descriptors are ready. If one of the buffers is ready, The data can be copied and then returned to the user thread, which reduces the need for the user thread to constantly poll and avoid the two context switches that occur with each poll.
In addition, the IO multiplexing model can block multiple I/O operations simultaneously. In addition, the I/O function of multiple reads and writes can be detected at the same time, and the I/O function is not actually called until data is readable or writable (not all of the data is readable or writable).
It is also important to note that since IO multiplexing can handle multiple IO, it introduces a new problem: the order between multiple IO becomes uncertain.
Signal-driven IO
IO multiplexing resolves the problem that one thread or multiple threads can monitor multiple file descriptors, but select uses polling to monitor multiple file descriptors, and polling the file descriptor’s readable state continuously to know whether there is readable data, so mineless polling seems a bit waste. Because polling is not effective in most cases, then someone thought, can I not always poll to see if the data is ready, can I send a request and let me know when the data is ready, so this is signal-driven IO.
Signal-driven IO is not used to monitor the data ready state in the way of circular request, but to establish a SIGIO signal connection when calling SIGAction. When the kernel data is ready, the thread is notified of the readability state through SIGIO signal. When the thread receives the readability state signal, A recvFROM request is issued to the kernel to read the data. In the signal-driven IO model, application threads can return after signaling without blocking, so one application thread can monitor multiple file descriptors at the same time.
-
The application process starts the socket signal drive IO function, executes a signal processing function through the system call SIGAction, and requests immediate return;
-
When the data is ready, the SIGIO signal of the corresponding process is generated, and the application process is notified by signal callback.
-
The application process then calls recvFROM to copy the data from the kernel to user space.
-
After completion, a success message is displayed.
The characteristics of
Signal-driven IO compared with IO multiplexing, in this way of establishing signal association, the request only needs to wait for the notification of ready data, so that a large number of invalid data state polling operations can be avoided.
Asynchronous non-blocking IO (Asynchronous IO)
Whether it’s IO multiplexing or signal-driven, when we want to read data, we always issue a two-phase request. The first is a SELECT request asking if the data state is ready, and the second is a RecevForm request asking to read the data. At this point, we wonder why there is always a data ready state before reading data. Can the application process simply send a read request to the kernel, telling the kernel to read data and return it immediately? When the kernel data is ready, the kernel actively copies the data from the kernel to user space, and when all operations are complete, the kernel issues a notification to tell the application, so the asynchronous non-blocking IO model occurs.
An asynchronous non-blocking IO model application can do something else as soon as it initiates an AIO_read operation. Subsequent operations have the kernel taking over, and when the kernel receives an Asynchronous read, it returns immediately, with no blocks to the user process. The kernel then waits for the data to be ready and copies it to user memory. When all is done, the kernel either sends a signal to the user process or executes a thread-based callback function to complete the IO processing.
-
The application process initiates the aio_read operation and returns immediately.
-
The kernel waits for the data to be ready and then copies the data to user memory.
-
The kernel sends a signal to the user process;
-
Received the signal, return a success prompt;
The characteristics of
Asynchronous non-blocking IO compared to signal-driven IO, where the kernel tells us when the next IO operation is ready to start, asynchronous non-blocking IO lets the kernel tell us when the operation is complete.
Summary of five IO models
The difference between blocking AND non-blocking I/OS
Calling blocking IO blocks the process until the operation is complete, while non-blocking IO returns immediately while the kernel is still preparing data.
The difference between synchronous I/O and asynchronous I/O
The difference between the two is that the process is blocked when performing an I/O operation. That is, the application process calls recvFROM. Recvfrom copies data from the kernel to the user’s memory.
For example
Xiao Wang went to buy a train ticket and got a refund three days later. Performers (Lao Li, scalpers, ticket sellers, couriers), it takes 1 hour to travel to and from the station.
Synchronous blocking IO
Xiao Wang went to the train station to buy a ticket and queued for three days to get a refund. Xiao Wang could do nothing but buy a ticket for the whole three days.
Synchronize non-blocking IO
Xiao Wang went to the train station to buy a ticket, went to the train station the next day to ask if there was a refund, and bought a ticket three days later. The whole process takes wang three round trips, which takes three hours, during which he can do other things.
IO multiplexing
select/poll
Wang went to the railway station to buy tickets, entrusted scalpers to buy them, and then called the scalpers every 12 hours. The scalpers bought tickets in three days, and then Wang went to the railway station to pay the money and get the tickets. The whole xiao Wang needs to go back and forth 2 times, which takes 2 hours to go back and forth. The scalper needs a handling fee of 100, and calls 6 times.
epoll
Xiao Wang went to the railway station to buy a ticket and entrusted the scalper to buy it. After the scalper bought it, he informed Xiao Wang to get it. Then Xiao Wang went to the railway station to pay the money and get the ticket. During the whole process, Xiao Wang needs to go back and forth twice, which takes 2 hours. The scalper needs a handling fee of 100, without making a phone call.
Signal driven IO
Xiao Wang went to the railway station to buy a ticket, the conductor left a phone number, after the ticket, the conductor called Xiao Wang, then Xiao Wang went to the railway station to pay the money to get the ticket. The whole process requires wang to make two round-trip trips, which takes 2 hours. There is no handling fee and no need to make a phone call.
Asynchronous non-blocking IO
Xiao Wang went to the train station to buy a ticket and left a phone number to the conductor. When he got the ticket, the conductor called xiao Wang and sent the ticket to his door by express. During the whole process, Xiao Wang needs to make a round trip for 1 hour. There is no handling fee and no need to make a phone call.
reference
What does IO multiplexing mean
100% understanding of 5 IO models
Let’s talk about five IO models for Linux
The end of the
Welcome everyone little attention, little praise!