Cabbage Java self study room covers core knowledge

1. Zero copy introduction

Zero copy (English: zero-copy; Zero copy (zero copy) is a technique in which a computer performs an operation without the CPU having to copy data from one part of memory to another specific area. This technique is commonly used to save CPU cycles and memory bandwidth when transferring files over the network.

Zero-copy technology for certain components of the operating system, such as drivers, file systems, and network protocol stacks, can greatly enhance the performance of a particular application and make more efficient use of system resources. Performance is also enhanced by enabling the CPU to perform tasks other than copying data from the machine to another location. In addition, the zero copy operation reduces the number of modes to switch between user space and kernel space.

For example, if you want to read a file and send it over the network, the traditional way requires two copies of data and two context switches per read/write cycle, and data replication is cpu-dependent. The same operation is accomplished with zero-copy technology, reducing context switches to two times and eliminating the need for the CPU to copy data.

The zero-copy protocol is particularly important for high-speed networks where link capacity approaches or exceeds CPU processing capacity. In this network, the CPU spends almost all of its time copying the data to be transmitted, and thus becomes a bottleneck that keeps the communication rate below link capacity.

2. Zero-copy principle

2.1. A case

To explain this concept, let’s start with a requirement. Let’s say someone gives you a task one day to complete a small program that reads data from a file and sends it over the network. The code is simple:

First we find the file in our operating system, then we read the data into the buffer, and finally we send the data from the buffer to the network.Copy the code

The code is very simple. Now let’s consider the entire process of data transfer from the computer to the network:

Now we can see that the whole process of 1->2->3->4 went through four copies in total, but it was the second and third copies that really cost resources and wasted time, because they both needed to go through our CPU and switch back and forth between kernel and user mode. Think about how valuable our CPU resources are to handle a large number of tasks. There’s a lot of data to copy. If you can get rid of these two copies of the CPU, it is not too fast! It can not only save CPU resources, but also avoid switching between kernel mode and user mode.

Here is the difference between user mode and kernel mode:

In user mode, the access to memory space and objects is limited, and the occupied processor is preemptable. In kernel mode, the access to all memory space and objects is restricted, and the occupied processor is not allowed to be preempted.

2.2. Optimization scheme

To remove copies between the second and third times, Linux developers have been aware of this problem for a long time, so in the Linux 2.1 kernel, we added “data is copied to the socket buffer” action, so our Java NIO, This can be achieved by directly calling a method called transferTo().

Right now, it feels like performance resources have been greatly improved, but it’s not perfect yet. Because these three copies also use the CPU copy technology, is the second copy. But don’t worry. Linux developers are more forward-thinking than we are.

2.3. Zero-copy optimization

Instead, descriptors containing only information about the location and length of the data are appended to the socket buffer. The DMA engine transfers data directly from the kernel buffer to the Protocol engine, eliminating the last CPU copy. After the above process, the data is copied from the disk only twice. This is the true zero-copy:

Note: here zero copy is actually divided according to the kernel state, there is no copy through the CPU, data in the user state, experienced zero copies, so it is called zero copy, but does not mean no copy.

3. Zero-copy applications

3.1. Java NIO

In Java NIO, channels are the buffer of the operating system’s Kernel Space. A Buffer corresponds to a User Buffer in the operating system’s User Space.

Out-of-heap memory (DirectBuffer) needs to be reclaimed manually by the application after use, whereas HeapBuffer data may be reclaimed automatically during GC. Therefore, when using HeapBuffer to read or write data, NIO copies the HeapBuffer data into a temporary DirectBuffer Native Memory to avoid the loss of the buffer data due to GC. This copy involves sun. Misc. Unsafe. CopyMemory () call, and the implementation of the principle behind memcpy (). Finally, the memory address of the data inside the provisionally-generated DirectBuffer is passed to the I/O calling function, thus eliminating the need to access Java objects for I/O reading and writing.

  • MappedByteBuffer

MappedByteBuffer is an implementation of NIO’s zero-copy mMAP-based approach, which maps the size area of a file from position to a memory image file. This adds the address map without copying it.

  • DirectByteBuffer

The object reference of DirectByteBuffer is located in the heap of the Java memory model. The JVM can allocate and recycle the object of DirectByteBuffer. It is the concrete implementation class of MappedByteBuffer. So it also has zero copy technology.

  • FileChannel

FileChannel defines two abstract methods transferFrom() and transferTo(), which transfer data by establishing connections between channels.

Looking directly at Linux2.4, the socket buffer has been adjusted and DMA has collection functionality.

(1) DMA is copied from the kernel buffer

(2) add the location and length of the data descriptor to the kernel space (socket buffer).

(3) DMA copies data from the kernel to the protocol engine

This replication process is zero-copy.

3.2. Netty

Zero copy in Netty is not quite the same as zero copy at the operating system level mentioned above, which is completely user-based.

  1. Netty wraps the FileChannel tranferTo() method through the DefaultFileRegion class, which is equivalent to zero copy via Java indirectly.

  2. Our data transmission are generally through the TCP/IP protocol implementation, in practice, is likely to be a complete message is divided into multiple data packets for network transmission, and a single packet is meaningless for you, only when these packets of a complete message will you be able to make the right treatment, Netty can combine these packets into a complete message for you to use with zero copy.

In this case, the scope of zero copy is only in user space. So how does Netty work? To do this, we need to find the Netty interface for data transfer. This interface must contain the zero-copy function. This interface is the ChannelBuffer.

The main implementation class is CompositeChannelBuffer, which is composed of several channelbuffers into a virtual ChannelBuffer for operation

This is virtual because the CompositeChannelBuffer does not actually combine the multiple channelbuffers but only holds their references, thus avoiding copying data and achieving zero-copy.

  1. ByteBuf can wrap a byte array, ByteBuf, and ByteBuffer into a ByteBuf object via the wrap operation, thus avoiding copy operations

  2. ByteBuf supports slice, so you can split ByteBuf into multiple BytebuFs that share the same storage area, avoiding memory copying

3.3. Kafka

Kafka uses mmap + write for index files and SendFile for data files. It is suitable for data persistence and transmission of large files with high throughput, such as system log messages.

If there are 10 consumers, the number of data copies in the traditional way is 4 x 10=40 times. However, in the zero-copy technology, only 1+10=11 times is required. One time is to copy data from disk to page cache, and 10 times means that 10 consumers each read the page cache once.