This is the 10th day of my participation in the August More Text Challenge. For details, see:August is more challenging

preface

The basic unit of network data transmission is byte, and the buffer is the container for storing bytes. When bytes are accessed, they are first put into the buffer and then stored in bulk in the operation buffer to improve performance.

Java NIO provides ByteBuffer as its buffer, but this class is too complex to use and a little tedious. Therefore, Netty implements ByteBuf itself in place of ByteBuffer.

This article introduces Netty’s own buffers.

ByteBuf class

Buffer can be as simple as a memory area, and in some cases, if the program of frequent operation a resource (e.g., file or database), the performance will be very low, at this time in order to improve performance, it can be part of an area of data read into memory for the time being, can be directly read data from the area in the future. Because reading memory will be fast, this can improve the performance of the program.

Thus, buffers determine the performance of network data processing.

ByteBuf is designed as a buffer type that can solve ByteBuffer problems from the bottom layer and meet the needs of daily network application development. Its characteristics are as follows:

  • Custom buffer types are allowed.
  • Transparent zero copy is achieved through the built-in compound buffer type.
  • Capacity can grow on demand (similar to JDKStringBuilder).
  • Switching between read and write modes does not require calling ByteBufferflip()Methods;
  • Normally has a faster response time than ByteBuffer.

Implementation principle of ByteBuffer

Reading and writing data using ByteBuffer generally follows the following four steps.

  • Writes data to the ByteBuffer.
  • callflip()Methods.
  • Read data from the ByteBuffer.
  • callclear()Methods orcompact()Methods.

When data is written to the ByteBuffer, the ByteBuffer keeps track of how much data is written. Once the data is read, you need to switch the ByteBuffer from write mode to read mode via the flip() method. In read mode, all data previously written to the ByteBuffer can be read.

Once you’ve read all the data, you need to empty the buffer so it can be written again. There are two ways to clear the buffer: by calling the clear() method or the compact() method. The clear() method clears the entire buffer. The compact() method only clears the data that has been read. Any unread data is moved to the beginning of the buffer, and new data is written to the end of the buffer.

Here is an example of using ByteBuffe in a Java NIO implementation server-side instance:

/ / write
if (key.isWritable()) {
    SocketChannel client = (SocketChannel) key.channel();
    ByteBuffer output = (ByteBuffer) key.attachment();
    output.flip();
    client.write(output);

    System.out.println("NonBlokingEchoServer -> " 
                       + client.getRemoteAddress() + "：" + output.toString());

    output.compact();

    key.interestOps(SelectionKey.OP_READ);
}
Copy the code

For a ByteBuffer, there are five main attributes: Mark, position, LIMIT, Capacity, and array. These five attributes do the following:

  • Mark: Records the index subscript of the previous mark.
  • Position: For write mode, the index of the data that can be written currently, and for read mode, the index of the data that can be read next.
  • Limit: In write mode, indicates the current size of the array that can be written. The default value is the maximum length of the array. In read mode, indicates the positional index of the maximum number of data that can be read.
  • Capacity: indicates the capacity of the current array.
  • Array: Stores the data being written.

The above variables have the following relationships:

0 <= mark <= position <=limit <= capacity
Copy the code

In addition to array, which is used to hold data, the three attributes of position, LIMIT, and Capacity are of most concern, because they have very different meanings for write and read modes.

ByteBuffer write mode

The following figure shows the initial state and the state of the position, LIMIT, and Capacity attributes after three bytes are written:

As can be seen from the figure, in write mode:

  • limitAlways points to the array index index that is currently writable.
  • positionPoints to the index location of the next data to be written.
  • capacityThat is, the array size.

ByteBuffer Read mode

Suppose we write three bytes of data to a ByteBuffer of initial length 6 as described above, and we switch to read mode. The position, limit, and Capacity values will be as follows:

As you can see, after switching to read mode:

  • Limit points to the position next to the last data that can be read, indicating the maximum number of data that can be read.

  • Position points to the initial position of the array, indicating the position of the next data to be read.

  • Capacity is also the maximum size of the array.

In this case, when we read data one by one, position will switch down, and when the limit coincides, there is no data available in the current ByteBuffer.

The write mode of the ByteBuffer is changed to read mode

The flip() method is called when a read/write switch occurs. The core source code for the flip() method is as follows:

public final Buffer flip(a) {
    limit = position;
    position = 0;
    mark = -1;
    return this;
}
Copy the code

As you can see from the source code above, after executing flip(), you set limit to position and then set that position to 0.

Clear () and compact() methods

Once the data in the ByteBuffer is read, you need to get the ByteBuffer ready to be written again. This can be done through the clear() or compact() methods.

If the clear() method is called, position will be set to 0 and limit will be set to the value of capacity. In other words, the ByteBuffer is emptied. The data in the ByteBuffer is not erased, but these markers tell us where to start writing data into the ByteBuffer.

If there is some unread data in the ByteBuffer, the clear() method is called and the data is forgotten, meaning that there is no longer any marker to indicate which data has been read and which has not.

The core source code for the clear() method is as follows:

public final Buffer clear(a) {
    position = 0;
    limit = capacity;
    mark = -1;
    return this;
}
Copy the code

If you still have unread data in the ByteBuffer and need it later, but want to write some data first, use the compact() method.

The compact() method copies all unread data to the start of the ByteBuffer. Then set position to the end of the last unread element. The limit property is still set to capacity, just like the clear() method. Now the ByteBuffer is ready to write data, but it will not overwrite the unread data.

The core source code for the compact() method is as follows:

public ByteBuffer compact(a) {

    int pos = position();
    int lim = limit();
    assert (pos <= lim);
    int rem = (pos <= lim ? lim - pos : 0);

    unsafe.copyMemory(ix(pos), ix(0), (long)rem << 0);
    position(rem);
    limit(capacity());
    discardMark();
    return this;

}
Copy the code

Use case of ByteBuffer

To better understand ByteBuffer, let’s look at some examples:

public class ByteBufferDemo {

	public static void main(String[] args) {
		// Create a buffer
		ByteBuffer buffer = ByteBuffer.allocate(10);
		System.out.println("------------ initial buffer ------------");
		printBuffer(buffer);

		// Add some data to the buffer
		System.out.println("------------ Add data to buffer ------------");

		String s = "love";
		buffer.put(s.getBytes());
		printBuffer(buffer);

		// Switch to read mode
		System.out.println("------------ Execute flip to switch to read mode ------------");
		buffer.flip();
		printBuffer(buffer);

		// Read the data
		System.out.println("------------ Read data ------------");

		// Create an array of bytes of size limit() (because there is only so much data to read)
		byte[] bytes = new byte[buffer.limit()];

		// Put the read data into our byte array
		buffer.get(bytes);
		printBuffer(buffer);

		/ / compact execution
		System.out.println("-- -- -- -- -- -- -- -- -- -- -- -- perform compact -- -- -- -- -- -- -- -- -- -- -- --");
		buffer.compact();
		printBuffer(buffer);

		The clear / / execution
		System.out.println("------------ Run clear to clear the buffer ------------");
		buffer.clear();
		printBuffer(buffer);

	}

	/** * Print out ByteBuffer **@param buffer
	 */
	private static void printBuffer(ByteBuffer buffer) {
		System.out.println("mark：" + buffer.mark());
		System.out.println("The position." + buffer.position());
		System.out.println("limit：" + buffer.limit());
		System.out.println("capacity："+ buffer.capacity()); }}Copy the code

Output results:

[pos=0 lim=10 cap=10] position: 0 limit: 10 capacity: 0 10 ------------ add data to buffer ------------ mark: java.nio.heapbytebuffer [pos=4 lim=10 cap=10] position: 4 limit: 10 capacity: ------------ mark: java.nio.heapbyteBuffer [pos=0 lim=4 cap=10] Position: 0 limit: 4 Capacity: 10 ------------ read data ------------ mark: java.nio.heapbytebuffer [pos=4 lim=4 cap=10] position: 4 limit: 4 capacity: 10 ------------ Execute compact------------ mark: java.nio.HeapByteBuffer[pos=0 lim=10 cap=10] position: 0 limit: 10 Capacity: ------------ mark: java.nio.HeapByteBuffer[pos=0 lim=10 cap=10] Position: 0 limit: 10 Capacity: 10 Process finished with exit code 0Copy the code

conclusion

This article first shows the internal state of the ByteBuffer in both write and read modes, then analyzes the source code for the clear() and compact() methods, and finally explains a use case for ByteBuffer.

In the next section, we will analyze how ByteBuf is implemented.

At the end

I am a code is being hit is still trying to advance. If the article is helpful to you, remember to like, follow yo, thank you!