This article is participating in “Java Theme Month – Java Development Practice”, for details: juejin.cn/post/696826…

This is the first day of my participation in Gwen Challenge

Ideological motivation

No matter how bitter, must remember: suffering is our life on the road indispensable experience, only alive, just have the possibility of happiness!

The daily work

Be sure to be confident. You are a scenery, there is no need to look up in the scenery of others.

Classification of network IO models

  • BIO
  • Pseudo asynchronous
  • NIO
  • AIO

BIO

BIO is a typical network programming model. It is the process by which we usually implement a server program.

Complete the following steps

  1. The main thread accept request is blocked

  2. When the request arrives, a new thread is created to process the socket and complete the response to the client.

  3. The main thread continues to accept the next request

  4. One big problem with this model is that as the number of client connections increases, the number of threads created by the server also skyrockets and system performance deteriorates dramatically.

Pseudo asynchronous (BIO in thread pool mode)

Based on the BIO model, the Tomcat BIO Connector uses a thread pool to avoid creating a thread for each client: requests are thrown into the thread pool to wait for processing asynchronously.

NIO (Non-blocking mechanism)

The NIO API consists of three main sections: Buffers, Channels, and Selector.

  • NIO is based on the event-driven concept. It adopts the Reactor model to solve the problem that a server cannot process a large number of client connections simultaneously in the BIO model.

  • NIO is based on a Selector. The operating system kernel fires (via JNI callback to the method stack frame in the Java thread stack) when the socket has an event that is readable, writable, completed, or new TCP request. Selector returns the ready set of Selectionkeys, which are read and written through the SelectableChannel. (Use SelectionKey as the flag field of the event)

    • Since the JDK’s Selector base is based on the epoll implementation, the theory can simultaneously handle the maximum number of file handles on the operating system.

    • All read and write operations of SelectableChannel are = non-blocking. When half a packet is read because the data is not ready (data preparation phase), the SelectableChannel returns immediately, and does not synchronously block waiting for the data to be ready. When the TCP buffer data is ready, a Selector read event is triggered to drive the next read operation.

Therefore, a Reactor thread can process N clients simultaneously, which greatly improves the concurrent read and write capability of the Java server. JDK1.4 introduced the NIO class library, where NIO stands for non-block IO, primarily implemented using Selector multiplexers. Selector is implemented via epoll on mainstream operating systems such as Linux.

NIO implementation process, similar to SELECT:

  1. Create a ServerSocketChannel listening client connection and bind the listening port, setting it to non-blocking mode.

  2. Create a Reactor thread, create a Selector, and start the thread.

  3. Registers the ServerSocketChannel with the Reactor thread’s Selector. Listen for accept events.

  4. Selector loops wirelessly through the thread’s run method to poll for ready keys.

  5. Selector listens for new client access, processes new requests, completes TCP three-way handshake, and establishes a physical connection.

  6. Registers a new client connection to the Selector and listens for reads. Read network messages sent by clients.

  7. When the data sent by the client is ready, the client request is read and processed.

AIO

JDK1.7 introduces NIO2.0, which provides an implementation of asynchronous file channels and asynchronous socket channels. The underlying layer is implemented via IOCP on Windows and epoll on Linux

LinuxAsynchronousChannelProvider.java,
UnixAsynchronousServerSocketChannelImpl.java
Copy the code

  1. Create AsynchronousServerSocketChannel, binding listener port

  2. Call AsynchronousServerSocketChannel accept method, was introduced into its own CompletionHandler. Including the previous step, all are non-blocking

  3. Incoming connections, callback CompletionHandler completed method, in it, call AsynchronousSocketChannel read method, the incoming CompletionHandler responsible for dealing with data.

  4. When the data is ready, the Completed method of the CompletionHandler responsible for processing the data is triggered. Proceed to the next step. The write operation is similar, passing in the CompletionHandler.

Synchronous and asynchronous

These two concepts relate to the notification mechanism for messages (synchronous communication/ Asynchronous communication).

synchronous

When the completion of a task needs to depend on another task, the dependent task can be completed only after the dependent task is completed, which is a reliable task sequence. Either they both succeed or they both fail, and the states of the two tasks remain the same.

  • When a call is made, it does not return until the result is returned. But once the call returns, you get the return value. In other words, the caller actively waits for the result of the call.

  • For synchronous calls, the application layer needs to consult the system kernel. If the data has not been read, the task of reading the file has not been completed.

The application layer either hangs up or does something else based on its blocking and non-blocking (so synchronous and asynchronous do not determine its state while waiting for data to return);

If the data has been read, the kernel returns the data to the application layer, which can then use it for other related purposes.

asynchronous

There is no need to wait for the dependent task to complete, just notify the dependent task to complete what work, the dependent task is executed immediately, as long as the completion of the entire task itself is considered complete. As to whether the dependent task is actually completed in the end, the dependent task cannot be determined, so it is an unreliable task sequence.

  • After the call is issued, the call returns directly, so no result is returned.

  • In other words, when an asynchronous procedure call is made, the caller does not get the result immediately.

  • Instead, after the call is made, the caller is notified by a state, notification, or callback function to handle the call.

For asynchronous invocation, the application layer does not need to actively inquire the system kernel. After the system kernel finishes reading the file data, it will actively notify the application layer that the data has been read. At this time, the application layer can receive the data returned by the system kernel and then do other things. That is, whether it is synchronous or asynchronous depends on how the message is notified when the task is complete. The method of blind and active inquiry by the caller is synchronous invocation, and the method of active notification of task completion by the called is asynchronous invocation.

There are three notification mechanisms for messages: status, notification, and callback.

The former is inefficient; the latter two are efficient and similar.

Blocking and non-blocking

These two concepts relate to the state of a program (thread) waiting for a message notification (whether synchronous or asynchronous).

Blocking calls

Until the result of the call is returned, the current thread is suspended, waiting for message notification and unable to perform other business. The function returns only after the result is obtained.

Non-blocking call

A function that does not block the current thread and immediately returns to complete another task until the result is not immediately available.

In summary, whether it is blocked or not is what the interface looks like when it calls (makes the request) and waits for data to return. Those that are suspended and unable to perform other actions are blocking, and those that can be immediately “pumped out” to complete other “tasks” are non-blocking.

Discussion of blocking and synchronization

One might equate blocking calls with synchronous calls, but they are not the same.

  1. For synchronous calls, many times the current thread may still be active, but logically the current function does not return, in which case the thread may also process other messages. One more thing, to expand here:

    • If the thread is still performing other message processing while waiting for the current function to return, this is called synchronous non-blocking;

    • If the thread does not perform other message processing while waiting for the current function to return, but is in a pending wait state, this is called synchronous blocking;

So synchronization can be implemented in two ways: synchronous blocking and synchronous non-blocking; Similarly, asynchrony has two implementations: asynchrony blocking and asynchrony non-blocking;

  1. For blocking calls, the current thread is suspended until the current function returns;

While the non-blocking approach apparently improves CPU utilization, it also has the added consequence of increased thread switching on the system. Whether the increased CPU execution time will compensate for the system switching costs needs to be evaluated.

Synchronous asynchronous/blocking Non-blocking

Before explaining the difference between synchronous IO and Asynchronous IO, we need to define both. Stevens’ definition (actually POSIX) looks like this:

 A synchronous I/O operation causes the requesting process to be blocked until that I/O operation completes; 
 An asynchronous I/O operation does not cause the requesting process to be blocked;
Copy the code

The difference between the two is that synchronous IO blocks the process during “IO operation”. According to this definition, the aforementioned blocking IO, non-blocking IO, IO multiplexing all belong to synchronous IO.

  • One might say that non-blocking IO is not blocked. I/O operation refers to actual I/O operations, such as recvfrom.

  • When performing the recvfrom system call with non-blocking IO, if the kernel data is not ready, the process will not be blocked.

However, when the data in the kernel is ready, recvFROM copies the data from the kernel to the user’s memory, at which point the process is blocked. During this time, the process is blocked.

  • Asynchronous IO, however, returns when an I/O is initiated and is ignored until the kernel sends a signal telling the process that the I/O is complete. During this entire process, the process is not blocked at all.

Synchronous asynchronous/blocking Non-blocking

“Synchronous asynchronous” and “blocking non-blocking” are two different concepts.

synchronous / asynchronous is to describe the relation between two modules.
blocking / non-blocking is to describe the situation of one module.

An example:
"I": a
"bookstore" : b

a asks b: do you have a book named "c++ primer"?

blocking: before b answers a, a keeps waiting there for the answer. Now a (one module) is blocking. a and b are two threads or two processes or one thread or one process? we DON'T know.

non-blocking: before b answers a, a just leaves there and every two minutes, a comes here for looking for the answer. Here a (one module) is non-blocking. a and b are two threads or two processes or one process? we DON'T know. BUT we are sure that a and b couldn't be one thread.

synchronous: before b answers a, a keeps waiting there for the answer. It means that a can't continue until b finishes its job. Now we say: a and b (two modules) is synchronous. a and b are two threads or two processes or one thread or one process? we DON'T know.

asynchronous: before b answers a, a leaves there and a can do other jobs. When b gets the answer, b will call a: hey! I have it! Then a will come to b to get the book when a is free. Now we say: a and b (two modules) is asynchronous. a and b are two threads or two processes or one process? we DON'T know. BUT we are sure that a and b couldn't be one thread.
Copy the code

Synchronous asynchronous/blocking Non-blocking

Synchronization and asynchrony are very broad concepts that focus on whether the occurrence or execution of an event causes a temporary wait for the entire process when multiple tasks and events occur.

The key difference between blocking and non-blocking is whether an operation is requested and if the condition is not met, whether it waits or returns a flag message.

When discussing IO (hard disk, network, peripherals), a complete IO read request operation consists of two phases:

  • 1) Check whether the data is ready;

  • 2) Copy data (kernel copies data to user thread)

The difference between blocking IO and non-blocking IO is in the first phase, if the data is not ready, whether to wait while checking whether the data is ready or just return a flag message.

The book Unix Network Programming defines synchronous IO and asynchronous IO as follows:

In fact, the synchronous and asynchronous IO models are for user thread/kernel interactions:

IO synchronization: After the user sends AN I/O request, if the data is not ready, the user thread or kernel constantly polls the data to see whether the data is ready. When the data is ready, the data is copied from the kernel to the user thread.

Asynchronous I/O: Only the sending of THE I/O request is done by the user thread. Both phases of the I/O operation are completed automatically by the kernel, which then sends a notification to the user thread that the I/O operation has completed. That is, in asynchronous IO, there is no blocking on the user thread.

This is the key difference between synchronous I/O and asynchronous I/O. The key difference between synchronous I/O and asynchronous I/O is whether the data copy is done by the user thread or the kernel. Therefore, asynchronous IO must be supported by the underlying operating system.

Note that synchronous AND asynchronous I/OS are different from blocking and non-blocking I/OS.

Blocking and non-blocking IO are reflected in whether the user thread is waiting for the data to be ready or receives a flag message when the user requests an I/O operation if the data is not ready. That is, blocking and non-blocking IO are reflected in how data is processed during the first phase of AN IO operation to see if it is ready.

Synchronous/asynchronous versus blocking/non-blocking

  • Synchronous blocking: the least efficient.

  • Asynchronous blocking: An asynchronous operation can be blocked, but it is blocked not while processing a message, but while waiting for a message notification.

  • Synchronous non-blocking: Actually inefficient, this program needs to switch back and forth between two different behaviors, which can be inefficient.

  • Asynchronous non-blocking: More efficient.

User space versus kernel space

  • Today’s operating systems use virtual storage, and for 32-bit operating systems, the addressing space (virtual storage) is 4G (2 ^ 32).

  • The core of the operating system is the kernel, which is independent of ordinary applications and has access to the protected memory space as well as all permissions to access the underlying hardware devices.

In order to ensure that user processes cannot directly operate the kernel and ensure kernel security, the operating system divides the virtual space into two parts, one is the kernel space and the other is the user space.

For Linux, the highest 1G byte (from virtual addresses 0xC0000000 to 0xFFFFFFFF) is used by the kernel and is called kernel space, while the lower 3G byte (from virtual addresses 0x00000000 to 0xBfffff) is used by various processes and is called user space.

Process switching

To control process execution, the kernel must have the ability to suspend a process running on the CPU and resume execution of a previously suspended process. This behavior is called process switching/task switching/context switching. Therefore, it can be said that any process runs under the support of the operating system kernel and is closely related to the kernel.

Moving from a running process to a running process goes through the following changes:

  • Holds processor context, including program counters and other registers.
  • Update PCB information.
  • Move the process PCB to the appropriate queue, such as ready, blocked at an event queue, etc.
  • Select another process to execute and update its PCB.
  • Update memory management data structures.
  • Restore processor context.
  • Process blocking
  • If some expected events do not occur in the executing process, such as system resource request failure, waiting for the completion of an operation, new data has not arrived, or no new work is done, the system automatically executes Block primitives to change the running state into the blocked state. It can be seen that the blocking of a process is an active behavior of the process itself, so only the running process (to obtain CPU) can be put into the blocking state. When a process is blocked, it consumes no CPU resources.

File descriptor fd

A File descriptor is a computer science term, an abstract concept used to describe a reference to a File.

The file descriptor is formally a non-negative integer. In fact, it is an index value that points to the record table of open files that the kernel maintains for each process. When a program opens an existing file or creates a new file, the kernel returns a file descriptor to the process. In programming, some low-level programming tends to revolve around file descriptors. However, the concept of file descriptors is usually only applicable to operating systems such as UNIX and Linux.

Cache IO

Cache I/O is also known as standard I/O, and the default I/O operation for most file systems is cache I/O. In Linux’s caching IO mechanism, the operating system caches IO data in the file system’s page cache. That is, data is copied to the operating system kernel buffer before it is copied from the operating system kernel buffer to the application’s address space.

Disadvantages of caching IO

In the process of data transmission, multiple data copy operations are required in the application address space and the kernel. These data copy operations cost a lot of CPU and memory.