Front knowledge
Several concepts of operating system
The Linux operating system architecture is divided into user mode and kernel mode. Kernel is a special software program in essence, which controls the hardware resources of the computer, such as coordinating CPU resources, allocating memory resources, and providing a stable running environment for the upper application programs.
User mode is the upper – layer application, its operation depends on the kernel. In order for a user program to access kernel-managed resources, the kernel must provide common access interfaces, called “system calls.”
2. System calls and Soft Interrupts System calls are a set of common interfaces provided by the operating system for applications to access. They enable processes running in user mode to access kernel-managed resources, such as the creation of new threads, memory requests, and so on.
When an application makes a system call, it causes a “soft interrupt” as follows:
- The CPU stops executing the current program stream and saves the value of the CPU register to the stack.
- Find the function address of the system call, execute the function, and get the result of execution.
- The CPU restores the value of the register and continues to run the application.
To sum up, system calls cause the application to switch from user to kernel state, with a soft interrupt, which requires additional overhead and should be minimized if the application performance is to be improved.
Bio
As the original IO model of Java network programming, the Bio is Blocking IO operations such as Accept, read, and write.
When a thread is processing an I/O operation on a Socket, it blocks, and if the service is running on a single thread, it will stall until the I/O operation is complete. To avoid this, only one thread can be assigned to each connection.
This is not too much of a problem if the number of connections is small, but the Bio is helpless when faced with hundreds of thousands or millions of client connections for the following reasons:
- Thread is a very valuable resource, and the cost of thread creation and destruction is very high. In Linux system, thread is essentially a process, and creating and destroying thread is a very heavy system function.
- Threads themselves take up memory resources. Creating a thread requires around 1MB of stack space. Creating a thousand threads is scary enough.
- The cost of switching between threads is high. The operating system needs to save the context of running threads, temporarily store the value of registers in the thread stack, and call system functions to switch threads. If there are too many threads, it is likely that the threads will spend more time switching than running.
// Bio version of Echo service
public class BioEchoServer {
public static void main(String[] args) throws IOException {
ServerSocket serverSocket = new ServerSocket(9999);
while (true) {
final Socket accept = serverSocket.accept();
new Thread(() -> {
try {
InputStream inputStream = accept.getInputStream();
OutputStream outputStream = accept.getOutputStream();
while (true) {
byte[] bytes = new byte[1024];
int size = inputStream.read(bytes);
if (size <= -1) {
accept.shutdownOutput();
accept.close();
break; } outputStream.write(bytes); outputStream.flush(); }}catch(Exception e) { e.printStackTrace(); } }).start(); }}}Copy the codeadvantages
- The code is simple
disadvantages
- A thread can only handle one client connection.
- If the number of threads is not controllable, the service may crash in the face of sudden traffic.
- The frequent creation and destruction of threads requires additional overhead.
- A large number of threads leads to frequent thread switching.
Nio
Nio stands for “non-blocking IO”, which is a new IO system introduced in JDK1.4.
Nio uses a Channel instead of a Stream, which is one-way and is either an input Stream or an output Stream. A Channel is two-way, allowing you to read and write data simultaneously.
Nio adds a data buffer called “ByteBuffer” that must read and write data to a Channel. A ByteBuffer is itself an array of bytes with Pointers that move around as data is read and written.
Another core component of Nio is the “Selector” multiplexer, which we’ll talk about in the next section.
Nio is characterized by “non-blocking”, such as call ServerSocketChannel. The accept (), the thread does not block until the client connection, it will get the results immediately. SocketChannel is returned if there is a client connection, otherwise NULL is returned. For socketchannel.read (), a return of 0 indicates that the Channel currently has no data to read, a return of -1 indicates that the client is disconnected, and data is read only if the value is greater than 0.
Call Channel. ConfigureBlocking (false) setting the Channel as a non-blocking is the key, or call or blocking, remember!!!!!!
IO operations do not block, so we can use CPU resources properly without opening new threads.
For example, instead of blocking the thread and waiting for the client to connect, we poll it every once in a while to see if there is any new client access. If there is, we add it to the container, and then poll the SocketChannel in the container to see if there is any data to read or write. If there is no data to read or write, we skip it. In this way, even if there is only one thread, it can handle a large number of connections, strictly controlling the number of threads.
public class NioEchoServer {
static List<SocketChannel> channels = new ArrayList<>();
public static void main(String[] args) throws IOException, InterruptedException {
ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
serverSocketChannel.bind(new InetSocketAddress(9999));
// Set Channel to non-blocking!!
serverSocketChannel.configureBlocking(false);
while (true) {
SocketChannel socketChannel = serverSocketChannel.accept();
if(socketChannel ! =null) {
socketChannel.configureBlocking(false);
channels.add(socketChannel);
}
// Process data read operations
Iterator<SocketChannel> iterator = channels.iterator();
while (iterator.hasNext()) {
SocketChannel channel = iterator.next();
ByteBuffer byteBuffer = ByteBuffer.allocate(1024);
int readSize = channel.read(byteBuffer);
if (readSize > 0) {
// flip, readable > writable
byteBuffer.flip();
channel.write(byteBuffer);
} else if (readSize < 0) { iterator.remove(); channel.close(); }}// Avoid CPU idling and sleep for a while
ThreadUtil.sleep(10); }}}Copy the codeadvantages
- A single thread can handle a large number of connections.
- The number of threads is controllable.
- No blocking and higher performance than Bio.
disadvantages
- A large number of Socketchannels are polled each time, 10,000 times for every 10,000 connections, and each poll is a system call that causes a “soft interrupt” that consumes performance.
- The time of polling interval is difficult to control, too long will lead to response delay, too short will consume CPU resources.
- In most cases where connections are inactive, invalid polling increases and consumes CPU meaninglessly.
IO multiplexing
The main problem with Nio is that we don’t know which connections are ready to be processed in the face of a large number of connections, so we have to iterate over all connections at a time. When only a few connections are active, the efficiency of each poll is too low.
To solve the Nio problem, the “IO multiplexing” mechanism was introduced. In Java, the “Selector” interface is an abstract representation of a multiplexer. It has different implementations on different platforms. In general, select is supported by almost all platforms.
Common IO multiplexing implementation
select
In Linux, the select function is defined as:
int select(int nfds, fd_set *readfds, fd_set *writefds,
fd_set *exceptfds, struct timeval *timeout);
Copy the codeParameter Description:
- NFDS: the maximum value of the following FD sets +1.
- Readfds: FD collection that listens for readable events.
- Writefds: FD collection that listens for writable events.
- Exceptfds: FD set that listens for exception events.
- Timeout: indicates the timeout period.
Calling the select function can be interpreted as: the application passes the collection of socketchannels to be listened on to the kernel, which polls through all socketchannels and tells the application which socketchannels are ready.
If you have 10,000 connections, your application will need to make 10,000 system calls to traverse by itself, but with SELECT, you only need one system call. As mentioned earlier, to improve program performance, minimize the number of system calls.
Disadvantages of SELECT:
- Each call requires copying the FD collection from user space to kernel space.
- The kernel needs to traverse all FDS.
- The maximum number of FD’s supported is 1024, which is too small.
poll
In Linux, the poll function is defined as follows:
int poll(struct pollfd *fds, nfds_t nfds, int timeout);
Copy the codeIts implementation is very similar to select, except that it describes FD collections differently, with SELECT being an Fd_set structure and poll being a PollFD structure. Poll supports no maximum number of FDS up to 1024.
Disadvantages of poll:
- Each call requires copying the FD collection from user space to kernel space.
- The kernel needs to traverse all FDS.
epoll
In Linux, epoll consists of three functions: epoll_create
int epoll_create(int size);
int epoll_create1(int flags);
Copy the codeControl of epoll FD: epoll_ctl
int epoll_ctl(int epfd, int op, int fd, struct epoll_event *event);
Copy the codeWait for IO events on epoll: epoll_wait
int epoll_wait(int epfd, struct epoll_event *events,
int maxevents, int timeout);
int epoll_pwait(int epfd, struct epoll_event *events,
int maxevents, int timeout,
const sigset_t *sigmask);
Copy the codeWith the select and poll functions, the application needs to manage the Socket FD to listen on itself, and each call needs to copy the FD collection from user space to kernel space.
With epoll, create an epoll instance using epoll_create, call epoll_ctl to add the Socket FD you want to listen on, and let the kernel manage the FD collection without copying the FDS again when calling epoll_wait.
Another improvement of epoll is that it does not need to iterate over all Socket FDS every time. When you call epoll_ctl to add a FD, you specify a callback function for each FD. This callback function is triggered when the FD is awakened and adds the current FD to a ready list.
Two trigger modes of epoll
- Horizontal trigger (LT) :
epoll_waitThe application is notified when an event is detected, and the application does not have to process it and will continue to be notified next time. - Edge trigger (ET) :
epoll_waitThe application is notified when an event is detected, which the application must handle and will not be notified again.
As a Java programmer, you can ignore the three implementations described above and just care about the Selector interface, which in Java is an abstract representation of a multiplexer.
Here is a multiplexer version of EchoServer:
public class MultiplexingIOEchoServer {
public static void main(String[] args) throws IOException {
ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
serverSocketChannel.bind(new InetSocketAddress(9999)).configureBlocking(false);
Selector selector = Selector.open();
// Subscribe to ACCEPT
serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);
while (true) {
// Wait for channels to be ready
if (selector.select() > 0) {
Iterator<SelectionKey> iterator = selector.selectedKeys().iterator();
while (iterator.hasNext()) {
SelectionKey key = iterator.next();
iterator.remove();
if (key.isAcceptable()) {// Process new connections
SocketChannel sc = ((ServerSocketChannel) key.channel()).accept();
sc.configureBlocking(false);
sc.register(selector, SelectionKey.OP_READ);
} else if (key.isReadable()) {// There is data to read
SocketChannel channel = (SocketChannel) key.channel();
ByteBuffer buffer = ByteBuffer.allocate(1024);
if (channel.read(buffer) > -1) {
buffer.flip();
channel.write(buffer);
} else {
channel.close();
}
}
}
}
}
}
}
Copy the codeCalling Selector. Select () can be thought of as calling select(), epoll(), and epoll_wait(), which blocks without a Channel ready, so you can safely call it from while(true).
When you have a ready Channel, the Selector encapsulates the Channel as a SelectionKey, and the selection.selectedKeys () method returns a HashSet of keys, iterating over those keys to iterate over all the ready channels, Get the Channel’s event type via selectionKey.readyops ().
The Selector multiplexer solves Nio’s problem of knowing the connection ready in a single system call, even with ten thousand client connections. It also solves the problem of poorly set Nio polling intervals, processing events when there are events, and blocking on system calls waiting for events when there are no events. In the case of epoll, it also avoids the extra overhead of copying the FD collection from user space to kernel space for each poll, further improving system performance.