This article introduces the operating system I/O working principle, Java I/O design, basic use, common methods and implementation of high performance I/O in open source projects, thoroughly understand the way of high performance I/O

Basic concept

Before introducing I/O principles, let’s review a few basic concepts:

(1) Operating system and kernel

Operating system: The system software that manages computer hardware and software resources. Kernel: The core software of an operating system, which manages system processes, memory, device drivers, files, and network systems, and provides applications with secure access to computer hardware

2 Kernel space and user space

To prevent user processes from directly manipulating the kernel and ensure kernel security, the operating system divides the memory addressing space into two parts: Kernel-space, user-space for use by Kernel programs, user-space for use by User processes For security purposes, Kernel space and user-space are isolated, even if the User’s program crashes, the Kernel is not affected

3 the data flow

The data in the computer is based on the transmission of high and low voltage signals that vary over time. These data signals are continuous and have a fixed direction of transmission, similar to the flow of water in a pipe. Therefore, abstract data flow (I/O flow) is the concept of a sequential collection of bytes with a starting and ending point.

Abstract out the function of data flow: realize the decoupling of program logic and the underlying hardware, by introducing data flow as the abstraction layer between the program and hardware devices, oriented to the general data flow input and output interface programming, rather than specific hardware characteristics, the program and the underlying hardware can be independently flexible replacement and expansion

I/O working principles

1 disk I/O

The typical I/O disk working principles are as follows:

Tips: DMA: Direct Memory Access is a mechanism that allows peripheral devices (hardware subsystems) to directly Access the system’s main Memory. Based on DMA access, the data transmission between the main memory and hardware devices can save the whole scheduling of CPU

It is worth noting:

Read and write operations are implemented based on system calls
Read and write operations through the user buffer, the kernel buffer, application processes do not directly manipulate the disk
Application processes block reading operations until data is read

2 network I/O

Here is the most classic blocking I/O model:

Tips: Recvfrom, a function that receives data through the socket

It is worth noting:

Network I/O read and write operations through the user buffer, Sokcet buffer
The server thread blocks from the time it calls recvFROM until it returns a datagram ready. After recvFROM returns successfully, the thread starts processing the datagram

Java I/O design

1 I/O classification

In Java, data flow is materialized and implemented. The following points are generally concerned about Java data flow:

(1) The direction of the flow from the outside to the program is called the input flow; The flow from the program to the outside is called the output stream
(2) The data unit of stream the program takes byte as the minimum read and write data unit, called byte stream, and takes character as the minimum read and write data unit, called character stream
(3) Functional roles of streams

A stream that reads/writes data from/to a specific IO device (such as a disk, network) or storage object (such as an array of memory). It connects and encapsulates an existing stream, and realizes the data read/write function through the encapsulated stream, which is called processing stream (or filtering stream).

2 I/O interface

The java.io package contains a bunch of I/O classes that may seem confusing to beginners, but a closer look shows a pattern: These I/O classes are based on four basic abstract flows, either node flows or processing flows

2.1 Four basic abstract flows

The java. IO package contains all the classes needed for streaming I/O. The java. IO package has four basic abstract streams that handle byte streams and character streams:

InputStream
OutputStream
Reader
Writer

2.2 the node flow

Node flow I/O class name consists of node flow type + abstract flow type. Common node types are as follows:

The File File
A thread communication pipe within the Piped process
ByteArray/CharArray (ByteArray/character array)
StringBuffer/String (String buffer/String)

A node stream is usually created by passing in a constructor to the data source, for example:

FileReader reader = new FileReader(new File("file.txt"));
FileWriter writer = new FileWriter(new File("file.txt"));
Copy the code

2.3 processing flow

The process stream I/O class name consists of functions encapsulated for existing streams + abstract stream types. Common functions include:

Buffering: The buffering function is provided for data read and written to node flows. Data can be read and written in batches based on the buffering function, improving efficiency. Common ones are BufferedInputStream, BufferedOutputStream
Byte stream to character stream: implemented by InputStreamReader, OutputStreamWriter
Byte stream and basic type data conversion: here the basic data type data such as int, long, short, DataInputStream, DataOutputStream implementation
Byte stream and object instance conversion: used to implement object serialization, implemented by ObjectInputStream, ObjectOutputStream

A process stream is created by passing in an existing node stream or process stream in a constructor to apply the adapter/decorator pattern and transform/extend an existing stream:

FileOutputStream fileOutputStream = new FileOutputStream("file.txt"); BufferedOutputStream = new BufferedOutputStream(fileOutputStream); DataOutputStream out = new DataOutputStream(bufferedOutputStream); DataOutputStream out = new DataOutputStream(bufferedOutputStream);Copy the code

3 Java NIO

3.1 Standard I/O Problems Exist

Java NIO(New I/O) is an IO API that can replace the standard Java I/O API(starting with Java 1.4). Java NIO provides a different way of working for I/O than standard I/O. The purpose is to solve the following problems of standard I/O:

(1) Duplicate data

In standard I/O processing, data is read from the underlying hardware to the kernel space, then to the user file, then to the kernel space, and then to the underlying hardware

In addition, when the underlying I/O system calls are made through write, read and other functions, the start address and length of the buffer where the data is passed in are required. Due to the existence of the JVM GC, the position of the object in the heap is often changed, and the address parameter passed into the system function is not the real buffer address

System calls made using standard I/O can cause an additional copy of data from the JVM’s heap to contiguous-space memory outside the heap (out-of-heap memory).

Therefore, a total of six data copies are performed and the execution efficiency is low

(2) Operation blocking

In traditional network I/O processing, the thread blocks because the request establishes the connection (CONNECT), reads the network I/O data (READ), and sends the data (send)

// Wait for connection
Socket socket = serverSocket.accept();

// Connection established, read request message
StringBuilder req = new StringBuilder();
byte[] recvByteBuf = new byte[1024];
int len;
while((len = socket.getInputStream().read(recvByteBuf)) ! = -1) {
	req.append(new String(recvByteBuf, 0, len, StandardCharsets.UTF_8));
}

// Write the return message
socket.getOutputStream().write(("server response msg".getBytes()));
socket.shutdownOutput();
Copy the code

For example, when a connection is established and the request message is read, the server calls the read method. The client data may not be ready (for example, the client data is still being written or transferred), and the thread needs to block in the read method until the data is ready

In order to realize the concurrent response of the server, each connection needs to be processed by an independent thread. When the number of concurrent requests is large, the overhead of memory and thread switching is too large to maintain the connection

3.2 Buffer

The three core components of the Java NIO core are Buffer, Channel, and Selector

Buffer provides a Buffer of bytes commonly used for I/O operations. Common buffers include ByteBuffer, CharBuffer, DoubleBuffer, FloatBuffer, IntBuffer, LongBuffer, ShortBuffer, Corresponding to basic data types: Byte, char, double, float, int, long, short The Buffer layer supports Java HeapByteBuffer or DirectByteBuffer.

Out-of-heap memory refers to the corresponding memory that allocates memory objects outside the JVM heap. This memory is directly managed by the operating system (rather than the VIRTUAL machine). The advantages of using out-of-heap memory in I/O operations over in-heap memory are:

Does not have to be collected by the JVM GC line, reducing GC thread resource usage
Direct manipulation of out-of-heap memory during I/O system calls saves a copy of out-of-heap and in-heap memory

Using the malloc and free functions, the allocateDirect method allocates out-of-heap memory and returns a DirectByteBuffer object that inherits ByteBuffer:

public static ByteBuffer allocateDirect(int capacity) {
    return new DirectByteBuffer(capacity);
}
Copy the code

The recovery of out-of-heap memory is based on DirectByteBuffer member variable Cleaner class, which provides the clean method can be used for active recovery. In Netty, most of the out-of-heap memory can be recovered by recording the existence of the Cleaner and actively calling the clean method. In addition, when a DirectByteBuffer object is GC, the associated out-of-heap memory is also reclaimed

tips: The JVM parameter -xx :+DisableExplicitGC is not recommended because some frameworks that rely on Java NIO (such as Netty) will actively call System.gc() to reclaim DirectByteBuffer objects when they run out of memory. As the last safeguard mechanism to reclaim out-of-heap memory, setting this parameter will result in out-of-heap memory not being cleaned up in this case

Out-of-heap memory is based on the DirectByteBuffer class member variable of the underlying ByteBuffer class: the Cleaner object, which executes unsafe.freememory (address) when appropriate to reclaim out-of-heap memory

Buffer can be seen as an array of basic data types that store contiguous addresses. It supports read and write operations, corresponding to read and write modes. The current position state of the data is stored by several variables: Capacity, position, and limit:

Capacity Total length of the buffer array
Position Indicates the position of a data element to operate on
Limit The position of the next non-operable element in the buffer array: limit <= capacity

3.3 the Channel

NIO I/O operations are based on channels: read data from a Channel: create a buffer and ask a Channel to read data and write data from a Channel: Create a buffer, populate the data, and ask the Channel to write the data

A Channel is very similar to a stream, with the following differences:

A Channel can read and write, whereas a standard I/O stream is one-way
A Channel can be read or written asynchronously, and a standard I/O stream requires the thread to block and wait until the read or write operation completes
A Channel always reads and writes to the Buffer

Implementation of the most important channels in Java NIO:

FileChannel: Used to read and write files. The FileChannel method can reduce the number of times for copying read and write files, as described later
DatagramChannel: used for reading and writing UDP data
SocketChannel: Data read and write for TCP, representing the client connection
ServerSocketChannel: Listens for TCP connection requests. Each request creates a SocketChannel, which is used by the server

With standard I/O, the first step might be to get the input stream as follows, read the data from disk in bytes into the program, and then proceed, whereas in NIO programming, get the Channel first, and then do the read and write

FileInputStream fileInputStream = new FileInputStream("test.txt");
FileChannel channel = fileInputStream.channel();
Copy the code

Tips: FileChannel can only run in blocking mode, file asynchronously I/O is added in the JDK 1.7 Java nio. Channels. AsynchronousFileChannel

// server socket channel:
ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
serverSocketChannel.bind(new InetSocketAddress(InetAddress.getLocalHost(), 9091));

while (true) {
	SocketChannel socketChannel = serverSocketChannel.accept();
	ByteBuffer buffer = ByteBuffer.allocateDirect(1024);
	int readBytes = socketChannel.read(buffer);
	if (readBytes > 0) {
		// Reverse from writing data to buffer to reading data from buffer
		buffer.flip();
		byte[] bytes = new byte[buffer.remaining()];
		buffer.get(bytes);
		String body = new String(bytes, StandardCharsets.UTF_8);
		System.out.println("Server received:"+ body); }}Copy the code

3.4 the Selector

A Selector, one of the core components of Java NIO, that checks whether the state of one or more NIO channels is readable and writable. Realize single-thread management of multiple channels, that is, can manage multiple network connections

The core of Selector is the I/O multiplexing function provided by the operating system. A single thread can monitor multiple connection descriptors at the same time. Once a connection is ready (usually read or write ready), it can notify the program to perform corresponding read and write operations

Java NIO Selector works as follows:

(1) initialize the Selector object and the ServerSocketChannel object
(2) Register the socketAccept event for ServerSocketChannel with the Selector
(3) The thread blocks selector. Select () and exits the block when a client requests the server
(4) Obtain all ready events based on the selector. In this case, the socket-accept event is obtained first, and the data ready and readable events of the SocketChannel of the client are registered with the selector
(5) The thread blocks again at selector. Select (), when the client connection data is ready to read
(6) Read the client request data based on ByteBuffer, then write the response data and close the channel

The following is an example. The complete working code has been uploaded to github(github.com/caison/cais…). :

Selector selector = Selector.open();
ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
serverSocketChannel.bind(new InetSocketAddress(9091));
// Set the channel to non-blocking mode
serverSocketChannel.configureBlocking(false);
// Register the socket-accept event on the server
serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);

while (true) {
	// selector. Select () blocks until a channel-related operation is ready
	selector.select();
	// All channels associated with SelectionKey have ready events
	Iterator<SelectionKey> keyIterator = selector.selectedKeys().iterator();

	while (keyIterator.hasNext()) {
		SelectionKey key = keyIterator.next();
		// Server-side socket-accept
		if (key.isAcceptable()) {
			// Get the channel to which the client is connected
			SocketChannel clientSocketChannel = serverSocketChannel.accept();
			// Set to non-blocking mode
			clientSocketChannel.configureBlocking(false);
			// Register to listen for the client channel readable event and associate the newly allocated buffer with the channel
			clientSocketChannel.register(selector, SelectionKey.OP_READ, ByteBuffer.allocateDirect(1024));
		}

		/ / channel can be read
		if (key.isReadable()) {
			SocketChannel socketChannel = (SocketChannel) key.channel();
			ByteBuffer buf = (ByteBuffer) key.attachment();

			int bytesRead;
			StringBuilder reqMsg = new StringBuilder();
			while ((bytesRead = socketChannel.read(buf)) > 0) {
				// Switch from buF write mode to read mode
				buf.flip();
				int bufRemain = buf.remaining();
				byte[] bytes = new byte[bufRemain];
				buf.get(bytes, 0, bytesRead);
				// When the size of the packet is larger than byteBuffer, there may be sticky/unpack problems
				reqMsg.append(new String(bytes, StandardCharsets.UTF_8));
				buf.clear();
			}
			System.out.println("The server receives a packet:" + reqMsg.toString());
			if (bytesRead == -1) {
				byte[] bytes = "[This is the message returned by the service]".getBytes(StandardCharsets.UTF_8);

				int length;
				for (int offset = 0; offset < bytes.length; offset += length) { length = Math.min(buf.capacity(), bytes.length - offset); buf.clear(); buf.put(bytes, offset, length); buf.flip(); socketChannel.write(buf); } socketChannel.close(); }}// Selector does not remove SelectionKey instances from selectedKeys by itself
		// You have to remove the channel itself when you're done with it and the next time that channel becomes ready, the Selector will put it back in the selectedKeys againkeyIterator.remove(); }}Copy the code

Tips: Implementing high-performance network I/O based on Selector in Java NIO is cumbersome and not friendly to use. Generally, the industry uses Netty framework based on Java NIO for encapsulation optimization and extended rich functions to achieve elegant implementation

High-performance I/O optimization

This section introduces the optimization of high-performance I/O based on popular open source projects in the industry

1 zero copy

Zero copy technology is used to reduce or completely avoid unnecessary CPU copy during data read and write, reduce memory bandwidth usage, and improve execution efficiency. Zero copy has several implementation principles. The following describes the zero-copy implementation principles in common open source projects

1.1 Zero copy Kafka

Kafka is based on the Linux 2.1 kernel, with the improved SendFile + hardware-provided DMA Gather Copy implemented in the 2.4 kernel for zero-copy, transferring files over sockets

The function transfers files through a single system call, reducing the need for read/write mode switching. At the same time, data copy is reduced. The detailed process of Sendfile is as follows:

The basic process is as follows:

(1) The user process initiates the sendFile system call
(2) The kernel copies file data from disk to the kernel buffer based on DMA Copy
(3) The kernel copies the file description (file descriptor, data length) in the kernel buffer to the Socket buffer
(4) The kernel copies the kernel buffer data to the nic based on the file description information in the Socket buffer and the Gather Copy function provided by THE DMA hardware
(5) User process sendfile system call completed and returned

Compared with the traditional I/O method, sendFile + DMA Gather Copy realizes zero Copy, The Times of data Copy is reduced from 4 times to 2 times, The Times of system call is reduced from 2 times to 1 time, and The Times of user process context switch is changed from 4 times to 2 times DMA Copy, greatly improving the processing efficiency

Kafka’s FileChannel transferTo is based on the java.nio package:

public abstract long transferTo(long position, long count, WritableByteChannel target)
Copy the code

TransferTo sends the file associated with the FileChannel to a specified channel. When Comsumer consumes the data, Kafka Server sends the message data from the file to a SocketChannel based on the FileChannel

1.2 RocketMQ Zero copy

RocketMQ implements zero copy using mmap + write: mmap() maps the address of the kernel buffer to the user-space buffer, enabling data sharing without copying data from the kernel buffer to the user-space buffer

tmp_buf = mmap(file, len); 
write(socket, tmp_buf, len);
Copy the code

The basic process for mMAP + Write to achieve zero copy is as follows:

(1) The user process initiates the system MMAP call to the kernel
(2) Memory address mapping is carried out between the read buffer of user process kernel space and the cache area of user space
(3) The kernel copies file data from disk to kernel buffer based on DMA Copy
(4) The user process MMAP system call completes and returns
(5) The user process makes a write system call to the kernel
(6) The kernel copies data from the kernel buffer to the Socket buffer based on CPU Copy
(7) The kernel copies data from the Socket buffer to the nic based on DMA Copy
(8) The user process write system call completes and returns

RocketMQ messages based on mmap implementation in storage and loading of logic is written in the org. Apache. RocketMQ. Store. MappedFile, internal implementation based on Java nio provide. Nio. MappedByteBuffer, Mmap buffer based on FileChannel map method:

/ / initialization
this.fileChannel = new RandomAccessFile(this.file, "rw").getChannel();
this.mappedByteBuffer = this.fileChannel.map(MapMode.READ_WRITE, 0, fileSize);
Copy the code

Query CommitLog messages based on the mappedByteBuffer offset pos, data size:

public SelectMappedBufferResult selectMappedBuffer(int pos, int size) {
	int readPosition = getReadPosition(); / /... Various security check / / return mappedByteBuffer view ByteBuffer ByteBuffer = this. MappedByteBuffer. Slice (); byteBuffer.position(pos); ByteBuffer byteBufferNew = byteBuffer.slice(); byteBufferNew.limit(size);return new SelectMappedBufferResult(this.fileFromOffset + pos, byteBufferNew, size, this);
}
Copy the code

tips: Part of Java NIO Mmap memory is not resident memory and can be swapped to swap memory (virtual memory). RocketMQ introduces memory locking to improve message sending performance. Map the CommitLog files that need to be operated recently to memory and provide memory locking to ensure that these files always exist in memory. The control parameter of this mechanism is transientStorePoolEnable

Therefore, there are two ways to save MappedFile data as CommitLog:

1 Enable transientStorePoolEnable: Write to memory buffer -> Commit from memory buffer to fileChannel -> fileChannel -> Flush to disk
2 transientStorePoolEnable: Write mappedByteBuffer -> mappedByteBuffer -> Flush to disk

RocketMQ implements zero-copy based on MMAP + Write, which is suitable for data persistence and transmission of small block files such as business-level messages. Kafka implements zero-copy based on SendFile, which is suitable for data persistence and transmission of large files such as system log messages with high throughput

Kafka uses mmap+write for index files and SendFile for data files

1.3 Netty Zero Copy

There are two types of Netty zero-copy:

1 Zero copy based on the OS implementation and the underlying transferTo method based on FileChannel
2 Based on Java layer operation optimization, the array cache object (ByteBuf) encapsulation optimization, through the establishment of the data view of ByteBuf data, support ByteBuf object merge, segmentation, when only one data store is retained at the bottom, reduce unnecessary copy

2 multiplexing

The I/O multiplexing code implemented in Netty is much more elegant after the encapsulation optimization of Java NIO functionality:

/ / create mainReactor
NioEventLoopGroup boosGroup = new NioEventLoopGroup();
// Create a worker thread group
NioEventLoopGroup workerGroup = new NioEventLoopGroup();

final ServerBootstrap serverBootstrap = new ServerBootstrap();
serverBootstrap 
	 / / assembly NioEventLoopGroup
	.group(boosGroup, workerGroup)
	 // Set the channel type to NIO
	.channel(NioServerSocketChannel.class)
	// Set connection configuration parameters
	.option(ChannelOption.SO_BACKLOG, 1024)
	.childOption(ChannelOption.SO_KEEPALIVE, true)
	.childOption(ChannelOption.TCP_NODELAY, true)
	// Configure inbound and outbound event handlers
	.childHandler(new ChannelInitializer<NioSocketChannel>() {
		@Override
		protected void initChannel(NioSocketChannel ch) {
			// Configure inbound and outbound event channelsch.pipeline().addLast(...) ; ch.pipeline().addLast(...) ; }});// Bind ports
int port = 8080;
serverBootstrap.bind(port).addListener(future -> {
    if (future.isSuccess()) {
        System.out.println(new Date() + ": port [" + port + "] Binding successful!");
    } else {
        System.err.println("Port [" + port + "] Binding failed!); }});Copy the code

3 PageCache (PageCache)

PageCache is a file cache used by the operating system to reduce I/O operations on the disk. PageCache is the unit of page, the content is the physical block on the disk. PageCache can help programs to sequential read and write files almost as fast as memory read and write. The main reason is that the OS uses PageCache to optimize the performance of read and write operations:

Page cache read strategy: When a process initiates a read operation (for example, a process initiates a read() system call), it first checks whether the required data is in the page cache:

If so, access to disk is abandoned and read directly from the page cache
If not, the kernel schedules block I/O operations to read the data from disk, read the few pages that follow (no less than one page, usually three pages), and then put the data into the page cache

Page cache write strategy: When a process makes a write system call to write data to a file, the data is written to the page cache before the method returns. At this point, the data is not actually saved to the file, Linux simply marks the page in the page cache as “dirty” and adds it to the dirty page list

The flusher write back thread then periodically writes the dirty pages to disk to keep the data in disk consistent with memory, and finally cleans the dirty flag. Dirty pages are written back to disk in three cases:

Free memory falls below a specific threshold
Dirty pages reside in memory above a specific threshold
When the user process invokes sync() and fsync() system calls

In RocketMQ, the ConsumeQueue logical consumption Queue stores less data and is read sequentially. Under the prefetch effect of page cache mechanism, the read performance of ConsumeQueue file is almost close to read memory, and the performance is not affected even in the case of message accumulation. Two message flushing strategies are provided:

Synchronous flush: RocketMQ’s Broker does not actually return a successful ACK response to the Producer until the message has been persisted to disk
Asynchronous flush can make full use of the PageCache advantage of the operating system. As long as the message is written into the PageCache, the successful ACK can be returned to the Producer end. Flush messages are committed by background asynchronous threads, which reduces read and write latency and improves MQ performance and throughput

Kafka implements high performance message reads and writes using page caching, which is no longer expanded

reference

“Understanding the Linux Kernel in Depth — Daniel P.Bovet”

Netty Java heap out of memory literacy paste – Jiangnan white clothes

Java NIO? Just read this one! – small si zhu

RocketMQ Message Storage Process — Zhao Kun

This article understands Netty model architecture — Caison

More exciting, welcome to the public number distributed system architecture

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

Java I/O system from principle to application, this article is all clear

Basic concept

I/O working principles

1 disk I/O

2 network I/O

Java I/O design

1 I/O classification

2 I/O interface

2.1 Four basic abstract flows

2.2 the node flow

2.3 processing flow

3 Java NIO

3.1 Standard I/O Problems Exist

3.2 Buffer

3.3 the Channel

3.4 the Selector

High-performance I/O optimization

1 zero copy

1.1 Zero copy Kafka

1.2 RocketMQ Zero copy

1.3 Netty Zero Copy

2 multiplexing

3 PageCache (PageCache)

reference

Java I/O system from principle to application, this article is all clear

Basic concept

I/O working principles

1 disk I/O

2 network I/O

Java I/O design

1 I/O classification

2 I/O interface

2.1 Four basic abstract flows

2.2 the node flow

2.3 processing flow

3 Java NIO

3.1 Standard I/O Problems Exist

3.2 Buffer

3.3 the Channel

3.4 the Selector

High-performance I/O optimization

1 zero copy

1.1 Zero copy Kafka

1.2 RocketMQ Zero copy

1.3 Netty Zero Copy

2 multiplexing

3 PageCache (PageCache)

reference

Related Posts

The final chapter of the Java Atom Class

MySQL query optimization (4) : In-depth understanding of MySQL query optimization process

HashMap source code details