Related to the project
JAVA InputStream and OutputStream read files and send them over sockets. How many times did JAVA NIO copy files and send them over sockets? JAVA IO topic 3: JAVA memory mapping and application scenarios JAVA IO topic 4: JAVA sequential IO principles and corresponding application scenarios
In this article, Java InputStream and OutputStream read files and send them through sockets. In the end, it involves several copies. From BIO to NIO, we can better understand the role of out-of-heap memory and the so-called zero copy.
Zero copy of the kernel
Zero copy of the kernel means that it consumes no CPU resources and is completely handed over to DMA. There are no redundant copies of data in the kernel space. The main development process is as follows:
Read + send
1. Call the read function of the operating system, and the DMA copies the file to the kernel. Then the CPU copies the kernel data to the user buffer (out of the heap memory). Finally, the DMA copies the socket buffer data to the nic for transmission.
During this process, kernel data is copied to user space, and user space is copied back to memory, with two redundant copies.
2. Sendfile initial version
Call SendFile directly to send the file as follows: 1. Read the data from disk to kernel via DMA; 2. Copy the data from kernel to socket buffer via CPU; 3
Sendfile contains one less CPU copy than read + send. However, copying from the kernel buffer to the socket buffer is not necessary.
3. Improved version of SendFile, true zero copy
For Linux 2.4 or later, the improved processing process is as follows: 1. DMA copies disk data to the kernel buffer and appends the location and offset of the current data to the socket buffer. 2. After the above process, the data is copied from the disk only twice. And there is no CPU involved.
Zero copy of Java
First, use directBuffer
In the previous article, Java InputStream and OutputStream read files and sent them through sockets. In the previous article, we mentioned that there were six copies of files and sent messages based on BIO. Copies of files outside the heap and in the heap are redundant. We can use directBuffer to reduce these two copies:
// Open the file channel
FileChannel fileChannel = FileChannel.open(Paths.get("/test.txt"));
// Apply for off-heap memory
ByteBuffer byteBuffer = ByteBuffer.allocateDirect(1024);
// Read into off-heap memory
fileChannel.read(byteBuffer);
// Open the socket channel
SocketChannel socketChannel = SocketChannel.open(new InetSocketAddress("localhost".9099));
// Out-of-heap memory is written to the socket channel
socketChannel.write(byteBuffer);
Copy the codeEach line of code is clearly commented out, so let’s focus on what filechannel. read and socketchannel.write do:
- FileChannel. Read the analysis
//FileChannelImpl
public int read(ByteBuffer dst) throws IOException {... Ignoring a bunch of unimportant codesynchronized (positionLock) {
int n = 0;
int ti = -1;
try {
do {
// call IOUtil to read data into the direct buffer DST according to the file descriptor fd
n = IOUtil.read(fd, dst, -1, nd);
} while ((n == IOStatus.INTERRUPTED) && isOpen());
return IOStatus.normalize(n);
} finally {
threads.remove(ti);
end(n > 0);
assertIOStatus.check(n); }}}//IOUtil
static int read(FileDescriptor fd, ByteBuffer dst, long position,
NativeDispatcher nd)
throws IOException
{
ByteBuffer bb = Util.getTemporaryDirectBuffer(dst.remaining());
try {
int n = readIntoNativeBuffer(fd, bb, position, nd);
bb.flip();
if (n > 0)
dst.put(bb);
return n;
} finally{ Util.offerFirstTemporaryDirectBuffer(bb); }}private static int readIntoNativeBuffer(FileDescriptor fd, ByteBuffer bb,
long position, NativeDispatcher nd)
throws IOException
{
int pos = bb.position();
int lim = bb.limit();
assert (pos <= lim);
int rem = (pos <= lim ? lim - pos : 0);
if (rem == 0)
return 0;
int n = 0;
if(position ! = -1) {
n = nd.pread(fd, ((DirectBuffer)bb).address() + pos,
rem, position);
} else {
// The first read will go to here, otherwise go to the above branch
n = nd.read(fd, ((DirectBuffer)bb).address() + pos, rem);
}
if (n > 0)
bb.position(pos + n);
return n;
}
//FileDispatcherImpl
int read(FileDescriptor fd, long address, int len) throws IOException {
return read0(fd, address, len);
}
Copy the codeHere the call chain is quite deep, let’s comb through it step by step:
- Calling filechannel. read actually goes to the FileChannelImpl. Read method, and then goes to
n = IOUtil.read(fd, dst, -1, nd);The read of IOUtil is called, passing in the file descriptor, directBuffer - IOUtil calls its own
readIntoNativeBufferMethod, which literally means reading data into native cache, or out-of-heap memory - The IOUtil
readIntoNativeBufferThe method calln = nd.read(fd, ((DirectBuffer)bb).address() + pos, rem);, the NativeDispatcher’s read method, passing in the file descriptor, the off-heap memory address, and the length to read - The NativeDispatcher implementation class here is FileDispatcherImpl, which actually calls the native method read0 and passes in the file descriptor, out-of-heap memory address and read length
Let’s take a quick look at what the Native read0 method does:
/ / the following content from the JDK/SRC/solairs/native/sun/nio/ch/FileDispatcherImpl. C
JNIEXPORT jint JNICALL
Java_sun_nio_ch_FileDispatcherImpl_read0(JNIEnv *env, jclass clazz, jobject fdo, jlong address, jint len)
{
// Get the file descriptor
jint fd = fdval(env, fdo);
// Get a pointer to off-heap memory based on the address
void *buf = (void *)jlong_to_ptr(address);
// Call the system function read directly to read the file descriptor into buF
return convertReturnVal(env, read(fd, buf, len), JNI_TRUE);
}
Copy the codeAs you can see, the native read0 method directly calls the system function read, and reads the file data into the out-of-heap memory according to the out-of-heap memory address passed by the JVM (the function of the read method has been mentioned in the kernel zero-copy section). That is, when operating the out-of-heap memory directly, instead of using DirectByteBuffer, the out-of-heap memory needs to be copied to the heap for reading and writing (see Java InputStream and OutputStream reading files and sending them through sockets, exactly how many copies are involved). Therefore, using out-of-heap memory +channel can avoid in-heap memory copy and improve efficiency to some extent.
- SocketChannel. Write analysis
//SocketChannelImpl.java
public int write(ByteBuffer buf) throws IOException {
synchronized(writeLock) { ... Ignore unimportant codeint n = 0;
try {
for (;;) {
// Call ioutil. write to write data
n = IOUtil.write(fd, buf, -1, nd);
if ((n == IOStatus.INTERRUPTED) && isOpen())
continue;
returnIOStatus.normalize(n); }}finally{ writerCleanup(); }}}//IOUtil.java
static int write(FileDescriptor fd, ByteBuffer src, long position,
NativeDispatcher nd)
throws IOException
{
if (src instanceof DirectBuffer)
//directBuffer goes directly here
return writeFromNativeBuffer(fd, src, position, nd);
}
private static int writeFromNativeBuffer(FileDescriptor fd, ByteBuffer bb,
long position, NativeDispatcher nd) throws IOException
{
int pos = bb.position();
int lim = bb.limit();
assert (pos <= lim);
int rem = (pos <= lim ? lim - pos : 0);
int written = 0;
if (rem == 0)
return 0;
if(position ! = -1) {
written = nd.pwrite(fd,
((DirectBuffer)bb).address() + pos,
rem, position);
} else {
// Call SocketDispatcher to write data
written = nd.write(fd, ((DirectBuffer)bb).address() + pos, rem);
}
if (written > 0)
bb.position(pos + written);
return written;
}
//SocketDispatcher.java
int write(FileDescriptor fd, long address, int len) throws IOException {
// Call FileDispatcherImpl native method write0 directly
return FileDispatcherImpl.write0(fd, address, len);
}
Copy the codeBefore looking at native methods, let’s do a simple sorting:
- Socketchannelimp. write actually calls socketChannelImp. write and then calls
IOUtil.write(fd, buf, -1, nd);Pass in file descriptors and out-of-heap memory references IOUtil.writeCall your own private methodswriteFromNativeBufferInternally calledwritten = nd.write(fd, ((DirectBuffer)bb).address() + pos, rem);, gives the file descriptor, off-heap memory address to NativeDispatcher- NativeDispatcher is SocketDispatcher
FileDispatcherImpl.write0(fd, address, len);Native method
Then trace FileDispatcherImpl. Write0 (fd, address, len); This native method:
/ / the following content from the JDK/SRC/solairs/native/sun/nio/ch/FileDispatcherImpl. C
JNIEXPORT jint JNICALL
Java_sun_nio_ch_FileDispatcherImpl_write0(JNIEnv *env, jclass clazz, jobject fdo, jlong address, jint len)
{
// Convert file descriptors
jint fd = fdval(env, fdo);
// convert to an out-of-heap memory pointer
void *buf = (void *)jlong_to_ptr(address);
// Call the system function write directly to send data out of the heap
return convertReturnVal(env, write(fd, buf, len), JNI_FALSE);
}
Copy the codeIt can be seen that the native write0 method directly calls the system function write to send data out of the heap (the function of write method has been mentioned in the kernel zero copy section).
- summary
FileChannel and socketChannel work with directBuffer, which operate directly on off-heap memory in conjunction with the system functions write and read pair file descriptors. So it saves two copies compared to BIO.
Second, the channel transferTo
The zero copy in Java is realized by relying on the sendfile function of the operating system, providing channel.transferTo method, which allows the data of one channel to be directly sent to another channel. Next, we analyze and verify the above statement through sample code and specific source code. Example code is as follows:
/ / open the socketChannel
SocketChannel socketChannel = SocketChannel.open(new InetSocketAddress("localhost".9099));
//
FileChannel fileChannel = FileChannel.open(Paths.get("/test.txt"));
fileChannel.transferTo(0, fileChannel.size(), socketChannel);
Copy the codeFilechannel.transferto (0, filechannel.size (), socketChannel); Write the file data to the socket, continue to look at the source code:
//FileChannelImpl.java
public long transferTo(long position, long count,
WritableByteChannel target)
throws IOException
{... Ignore unimportant codelong sz = size();
if (position > sz)
return 0;
int icount = (int)Math.min(count, Integer.MAX_VALUE);
if ((sz - position) < icount)
icount = (int)(sz - position);
long n;
// Try tranfer directly first, if the kernel supports it
if ((n = transferToDirectly(position, icount, target)) >= 0)
return n;
// Try mappedTransfer, only for trusted channel types
if ((n = transferToTrustedChannel(position, icount, target)) >= 0)
return n;
// Channel is not trusted, it will go the slowest way
return transferToArbitraryChannel(position, icount, target);
}
// FileChannelimpl.java
private long transferToDirectly(long position, int icount,
WritableByteChannel target)
throws IOException
{
if(! transferSupported)// Return if the system does not support it
return IOStatus.UNSUPPORTED;
FileDescriptor targetFD = null;
if (target instanceof FileChannelImpl) { // If the target is fileChannel go here
if(! fileSupported)return IOStatus.UNSUPPORTED_CASE;
targetFD = ((FileChannelImpl)target).fd;
} else if (target instanceof SelChImpl) {
SocketChannel implements the SelChImpl interface, so it goes here
if ((target instanceofSinkChannelImpl) && ! pipeSupported)return IOStatus.UNSUPPORTED_CASE;
// Assign the targetFD value
targetFD = ((SelChImpl)target).getFD();
}
if (targetFD == null)
return IOStatus.UNSUPPORTED;
// Convert the fd corresponding to fileChannel and socketChannel to a specific value
int thisFDVal = IOUtil.fdVal(fd);
int targetFDVal = IOUtil.fdVal(targetFD);
// Self-transmission is not supported
if (thisFDVal == targetFDVal)
return IOStatus.UNSUPPORTED;
long n = -1;
int ti = -1;
try {
begin();
ti = threads.add();
if(! isOpen())return -1;
do {
// Call native method transferTo0
n = transferTo0(thisFDVal, position, icount, targetFDVal);
} while ((n == IOStatus.INTERRUPTED) && isOpen());
if (n == IOStatus.UNSUPPORTED_CASE) {
if (target instanceof SinkChannelImpl)
pipeSupported = false;
if (target instanceof FileChannelImpl)
fileSupported = false;
return IOStatus.UNSUPPORTED_CASE;
}
if (n == IOStatus.UNSUPPORTED) {
// Don't bother trying again
transferSupported = false;
return IOStatus.UNSUPPORTED;
}
return IOStatus.normalize(n);
} finally {
threads.remove(ti);
end (n > -1); }}Copy the codeThe code is a bit long:
- The transferTo of FileChannelImpl is invoked in three cases. If the system supports zero copy, the transferToDirectly goes
- The transferToDirectly method makes all kinds of judgments in front of it, which can be interpreted as a direct call
n = transferTo0(thisFDVal, position, icount, targetFDVal);Native method
To trace transferTo0:
/ / the following content from the JDK/SRC/solairs/native/sun/nio/ch/FileChannelImpl. C
JNIEXPORT jlong JNICALL
Java_sun_nio_ch_FileChannelImpl_transferTo0(JNIEnv *env, jobject this,
jint srcFD,
jlong position, jlong count,
jint dstFD)
{
#if defined(__linux__)
off64_t offset = (off64_t)position;
Call sendfile directly
jlong n = sendfile64(dstFD, srcFD, &offset, (size_t)count);
if (n < 0) {
if (errno == EAGAIN)
return IOS_UNAVAILABLE;
if ((errno == EINVAL) && ((ssize_t)count >= 0))
return IOS_UNSUPPORTED_CASE;
if (errno == EINTR) {
return IOS_INTERRUPTED;
}
JNU_ThrowIOExceptionWithLastError(env, "Transfer failed");
return IOS_THROWN;
}
return n;
}
Copy the codeIn fact, there are implementations of Linux, Solaris, APPLE and other platforms in this method. Here, only the implementation under Linux is intercepted, and it can be seen that the system function sendfile is directly called to realize data sending. The specific number of copies depends on the version of the Linux kernel.
conclusion
- NIO reads the file and sends it over the socket. What is the minimum number of copies?
If channel.transferTo is directly invoked and the Linux kernel version is greater than or equal to 2.4, the number of copy times can be reduced to 2 and the CPU does not participate in the copy.
- What is the relationship between off-heap memory and zero copy
I understand the net said zero copy, can be considered kernel level zero copy and Java level zero copy two kinds. And the so-called “zero” is not no copy at all, but to minimize the number of copies in different scenarios. Therefore, using DirectBuffer to reduce the number of copies and channel.transferTo can be regarded as a manifestation of “zero copy”. Of course, the only truly recognized zero-copy might be transferTo.
Refer to the article
Java out-of-heap memory, zero copy, direct memory, and network IO for FileChannel in NIO