preface

The three magic things in Java NIO: Channels, Selectors, and buffers. We’ve spent a lot of time talking about selectors in the last couple of videos, but we’re only going to do buffers today. I hope to understand the basic usage and principle of Buffer through this article.

Grasp the key points:

1. Two important Pointers keep changing

2. A Buffer can be read and written

3. API usage for basic operations

4. ByteBuffer can allocate direct memory outside the JVM heap

Basic operation

In the previous article we simulated the client to send a request when the code is as follows:

InputStream inputStream = socket.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(inputStream));
System.out.printf("Received server response: %s, processed %d\r\n", br.readLine(), (System.currentTimeMillis() - start));
br.close();
inputStream.close();Copy the code

In normal BIO mode, we can only maintain a byte array or a char array for bulk reads and writes, or use BufferedReader and BufferedInputStream for read and write buffers.

buffer.clear();
buffer.put(("Roger that. You sent:" + sb + "\r\n").getBytes("utf-8"));
buffer.flip();Copy the code

Java NIO buffers interact with NIO channels. We read data from a Channel into a Buffer and write data from a Buffer to a Channel. Essentially, there is an area of memory that can be used to write data and read it out later. This memory is wrapped in NIO Buffer and provides a series of read-write interfaces for development.

• Write data to Buffer;

• Call the flip ();

• Read data from Buffer;

• Call clear() or Compact ()

When data is written to a Buffer, the Buffer records the size of the data that has been written. When data needs to be read, flip() is used to change the Buffer from write mode to read mode. In read mode, all data that has been written can be read.

Buffer implementation

Cache area, internal use byte array storage data, and maintain several special variables, to achieve the repeated use of data. There are four member variables defined in java.nio.buffer:

1. Mark: The initial value is -1, used to back up the current position.

2. Position: The initial value is 0. Position represents the position where data can be written or read. After a data is written or read, position moves forward to the next position.

3. Limit: In write mode, limit indicates the maximum amount of data that can be written into the Buffer. It is equal to capacity. In read mode, limit indicates the maximum amount of data that can be read.

4. Capacity: indicates the cache array size

Core point: The operation of Buffer is to constantly change the position of position and limit Pointers to achieve the purpose of locating the reading position and ending position, so that data can be accurately read within the boundary.

Code implementation:

// Invariants: mark <= position <= limit <= capacity
private int mark = -1;
private int position = 0;
private int limit;
private int capacity;Copy the code

In the case of a ByteBuffer, ByteBuffer is an abstract class that cannot be created directly with a new statement. It can only be created with a static method allocate:

ByteBuffer byteBuffer = ByteBuffer.allocate(10);Copy the code

To create a 10-byte buffer, mark = -1, Position = 0, Limit = 10, and Capacity = 10

Let’s take a look at the common methods for buffers and how they are implemented internally:

put

The put method puts a byte variable x into the buffer, position incrementing by 1

public ByteBuffer put(byte x) {
    hb[ix(nextPutIndex())] = x;
    return this;
}

final int nextPutIndex() {                          // package-private
    if (position >= limit)
        throw new BufferOverflowException();
    return position++;
}Copy the code

Let’s take a look at several variables that change when putting data repeatedly:

ByteBuffer byteBuffer = ByteBuffer.allocate(10);
byteBuffer.put((byte) 'l');
byteBuffer.put((byte) 'o');
byteBuffer.put((byte) 'v');
byteBuffer.put((byte) 'e'); System.out.println(byteBuffer.limit()); // result 10 system.out.println (bytebuffer.position ()); // result 4 system.out.println (bytebuffer.capacity ()); // result 10 bytebuffer.put ((byte)' ');
byteBuffer.put((byte) 'x');
byteBuffer.put((byte) 'y');
byteBuffer.put((byte) 'j'); System.out.println(byteBuffer.limit()); // result 10 system.out.println (bytebuffer.position ()); // result 8 system.out.println (bytebuffer.capacity ()); 10 / / resultsCopy the code

get

The get method fetches a byte from the buffer at position

public byte get() {
    return hb[ix(nextGetIndex())];
}

final int nextGetIndex() {                          // package-private
    if (position >= limit)
        throw new BufferUnderflowException();
    return position++;
}Copy the code

flip

If I want to read data from a Buffer, I need to set position to the position I want to read and adjust limit.

byteBuffer.limit(byteBuffer.position())
byteBuffer.position(0);Copy the code

Java encapsulates these two steps in a flip method:

public final Buffer flip() {
    limit = position;
    position = 0;
    mark = -1;
    return this;
}Copy the code

mark

Mark is used to remember the current location

public final Buffer mark() {
    mark = position;
    return this;
}Copy the code

If mark is called and the buffer is read or written, the position will change. In order to return to the original position, we can call reset to restore the position:

public final Buffer reset() {
    int m = mark;
    if (m < 0)
        throw new InvalidMarkException();
    position = m;
    return this;
}Copy the code

clear

Initialize the four special variables in the Buffer as their original values

public final Buffer clear() {
    position = 0;
    limit = capacity;
    mark = -1;
    return this;
}Copy the code

To review the core point: The operation of Buffer is to constantly change the positions of position and limit Pointers to locate the read and stop positions, so that data can be accurately read within the boundary.

Direct Buffer

When creating ByteBuffer, we use static methods to allocate a buffer object directly:

ByteBuffer buf = ByteBuffer.allocate(1024);Copy the code

In the JVM, objects are created and placed in the heap. For example, when Object o = new Object(), a chunk of heap memory is allocated to new Object(), and the stack space holds the memory address of the Object referenced by O. The JVM does garbage collection by copying objects from the heap back and forth between different partitions. The memory address will change frequently, and the Buffer itself will be frequently read and written, which will lead to tedious memory collation. Is there a way out of JVM object management? There is another static method to create a Buffer:

ByteBuffer buf = ByteBuffer.allocateDirect(1024);Copy the code

Let’s compare the implementation of the method:

public static ByteBuffer allocateDirect(int capacity) {
    return new DirectByteBuffer(capacity);
}

public static ByteBuffer allocate(int capacity) {
    if (capacity < 0)
        throw new IllegalArgumentException();
    return new HeapByteBuffer(capacity, capacity);
}Copy the code

Call allocate() creates a HeapByteBuffer, and call allocateDirect() creates a DirectByteBuffer. One is “heap” memory and the other is “direct” memory.

Take a look at the DirectByteBuffer implementation:

// Primary constructor
    //
DirectByteBuffer(int cap) {                   // package-private

    super(-1, 0, cap.cap);
    boolean pa = VM.isDirectMemoryPageAligned();
    int ps = Bits.pageSize();
    long size = Math.max(1L, (long)cap + (pa ? ps : 0));
    Bits.reserveMemory(size, cap);

    long base = 0;
    try {
        base = unsafe.allocateMemory(size);
    } catch (OutOfMemoryError x) {
        Bits.unreserveMemory(size, cap);
        throw x;
    }
    unsafe.setMemory(base, size, (byte) 0);
    if(pa && (base % ps ! = 0)) { // Round up to page boundary address = base + ps - (base & (ps - 1)); }else {
        address = base;
    }
    cleaner = Cleaner.create(this, new Deallocator(base, size, cap));
    att = null;
}Copy the code

Unsafe. allocateMemory is a native method that calls the malloc method to allocateMemory outside the JVM.

Anyway, this is a block of memory outside the Java heap, and the address of this memory is recorded. Any future use of this ByteBuffer will directly access the memory starting with address.

An intuitive advantage of DirectBuffer is that it is not managed by GC, so there is less pressure to clean up memory when GC occurs. Of course, it is not completely unmanaged by the GC or can be collected, but it is certainly not handled by the GC when it normally defiles memory.

Class structure

We are just using the common ByteBuffer as an example. There are also various types of buffers available in NIO that we won’t go into here.

conclusion

1. There are two important Pointers in the Buffer: position and limit

2. A Buffer is readable and writable and contains an array of capacity sizes

3. API usage for basic operations, put, Get, Flip, Mark, clear

4. The flip method changes the position of the Pointers position and limit

5. Direct memory can be allocated outside the JVM heap

The important thing to remember is that if you don’t pay attention to the sequence of Buffer operations, there will be various problems.

A series of

NIO: Linux/IO fundamentals

NIO sees and says (2) – The two BIO in Java

NIO sees and says (3) — different IO models

NIO: Java NIO

NIO also said that (v) : Do it, do it today, understand Buffer

Pay attention to my

If you read on wechat, please click the link to follow me. If you read on PC, please scan the code to follow me. Welcome to communicate with me and point out mistakes at any time