Recently, I plan to deepen my knowledge of Java network programming (IO, NIO, Socket programming, Netty).

Java I/O streams

Java Network Programming

Java NIO: Buffers

Java NIO: Channels

Java NIO: selector

Java NIO requires an understanding of the three core concepts of buffers, channels, and selectors as a complement to Java I/O to improve the efficiency of mass data transfer.

basis

In addition to the need to have a certain understanding of Java network programming, also need to have a certain understanding of user space, kernel space, memory space multiple mapping knowledge

User space versus kernel space

To provide operating system stability, the operating system divides the virtual address space into user space and kernel space

Where user processes (our own programs) can only operate in user space

Data flow during I/O

Suppose we need to read data from a file on disk. The process makes a read() system call to enter kernel state, which then issues commands to the disk control hardware to read data from disk, and the disk controller writes the data directly to the kernel memory buffer (this is done by DMA, without the CPU). The kernel then copies the data from the temporary buffer in kernel space to the user buffer (with CPU involvement), and the process switches back to user mode to continue execution.

The data flow is summarized as follows: disk > kernel buffer > user buffer

The question is: can you avoid copying data from the kernel buffer to the user buffer?

Memory space multiple mapping

We know that for a virtual address space, more than one virtual address can point to the same physical memory address

If the virtual address of user space and the virtual address of kernel space are mapped to the same physical address, then the space represented by the physical address is visible to the kernel and user process! This saves the overhead of copying data back and forth between the kernel and user buffers. (This is the idea of a direct buffer.)

Buffer (Buffer)

Java NIO data transfer process: the data is first put into the send buffer –> sent over the channel to the receiver –> the receiving channel receives the data and populates the receive buffer

So the buffer is really connecting the channel as the destination or source of the data transfer (or the input or output of the channel)

The core concept

attribute

To understand how buffers work, you need to understand the meaning of a few attributes

  • Capacity Capacity of the buffer, specified when the buffer is created

  • Position The index of the next element to be read or written

  • Limit The first position in the buffer that cannot be read or written

  • Mark the location of a memo

    Mark <= position <= limit <= capacity mark <= position <= limit <= capacity

access

The core of Buffer is the access operation. Buffer provides two ways of relative location access and absolute location access

  • Relative position access: Writes or reads data at the current position and then increments the value of position

  • Absolute position access: Writes or reads data at a specified position without changing position

 // Relative location access
 public abstract ByteBuffer put(byte b);
 public abstract byte get(a);

 // Absolute location access
 public abstract ByteBuffer put(int index, byte b);
 public abstract byte get(int index);
Copy the code

Flip (flip)

Reversal is the core concept of Buffer. We can understand that Buffer has two modes: write mode and read mode

In write mode, we allocate a buffer and populate the data directly (position increments from 0); In read mode, we also read data from scratch (position increments from 0)

So how do we switch from write mode to read mode? Turn over!!!!! When flipping, we use limit to record the length of the data to be read, and then set position to 0 to start reading

Below is the flipped source code

public final Buffer flip(a) {
  	// Record the length of the data to be read
    limit = position;
  	// Start reading data from scratch
    position = 0;
    mark = -1;
    return this;
}
Copy the code

demo

A complete example

// Create a buffer
ByteBuffer buffer = ByteBuffer.allocate(100);
/ / write data
for (char c : "hello".toCharArray()) {
  buffer.put((byte) c);
}
/ / flip
buffer.flip();Buffer.limit (buffer.position()).position(0);
/ / read the data
while (buffer.hasRemaining()) {
  char c = (char) buffer.get();
  System.out.println(c);
}
Copy the code

Creating a buffer

Buffers cannot be instantiated directly by constructors; they are created using static factory methods. The following is a static factory method for ByteBuffer

// Create a memory buffer
public static ByteBuffer allocate(int capacity);
// Create a direct buffer
public static ByteBuffer allocateDirect(int capacity) ;

public static ByteBuffer wrap(byte[] array, int offset, int length)
Copy the code

Direct Buffer

For normal I/O processes, the data flow is always: disk or network -> kernel temporary buffer -> user-space buffer, where the kernel-space temporary buffer to user-space buffer copy step is a bit redundant!!

Direct buffers solve this problem by being visible to both kernel and user space, thus avoiding the overhead of “kernel temporary buffer to user space buffer” copy

Although direct buffers are the best choice for I/O, they are more expensive than creating indirect buffers, so we tend to reuse direct buffers (creating them every time would be too expensive).

conclusion

This article mainly explains some basic knowledge of NIO learning and the use of buffers, focusing on the understanding of direct buffers.