FileChannel provides a way to access files through channels. FileChannel can be used to locate files anywhere in the file using the position(int) method. FileChannel also maps files to direct memory, improving access efficiency for large files. This article will introduce its usage and principle in detail.
1. Obtain channels
A FileChannel can be obtained by the getChannel() method in the FileInputStream, FileOutputStream, RandomAccessFile object, You can also use the static method filechannel. open(Path, OpenOption…). To open.
1.1 Obtained from FileInputStream/FileOutputStream
The channels obtained from the FileInputStream object open the file as read, and the channels obtained from the FileOutpuStream object open the file as write.
FileOutputStream ous = new FileOutputStream(new File("a.txt")); FileChannel out = ous.getChannel(); FileInputStream ins = new FileInputStream(new File("a.txt")); FileChannel in = ins.getChannel(); // Get a write-only channelCopy the code1.2 Get from RandomAccessFile
The channels retrieved from RandomAccessFaile depend on how the RandomAccessFaile object is created, with “R”, “w”, and “Rw” corresponding to read mode, write mode, and read/write mode respectively.
RandomAccessFile file = new RandomAccessFile("a.txt", "rw"); FileChannel channel = file.getChannel(); // Get a read/write file channelCopy the code1.3 Run filechannel.open () to open the file
Channels opened by the static static method filechannel.open () can specify the opening mode, which is specified by the StandardOpenOption token type.
FileChannel channel = FileChannel.open(Paths.get("a.txt"), StandardOpenOption.READ); // Open a file a.tuck channel in read-only modeCopy the code2. Read data
The read(ByteBuffer buf) method that reads data returns a value representing the number of bytes read, or -1 if the end of the file is read. Position moves backwards when the data is read.
2.1 Reading data into a single buffer
As with normal channel operations, data is read into a buffer and then retrieved from the buffer. When you call the read method to read data, you can pass in the parameters Position and length to specify the position and length to start reading.
FileChannel channel = FileChannel.open(Paths.get("a.txt"), StandardOpenOption.READ); ByteBuffer buf = ByteBuffer.allocate(5); while(channel.read(buf)! =-1){ buf.flip(); System.out.print(new String(buf.array())); buf.clear(); } channel.close();Copy the code2.2 Multiple Buffers are read
FileChannel implements the ScatteringByteChannel interface, which allows the contents of a FileChannel to be read simultaneously into multiple bytebuffers, which is useful when dealing with files containing several fixed-length chunks of data.
ScatteringByteChannel channel = FileChannel.open(Paths.get("a.txt"), StandardOpenOption.READ); ByteBuffer key = ByteBuffer.allocate(5), value=ByteBuffer.allocate(10); ByteBuffer[] buffers = new ByteBuffer[]{key, value}; while(channel.read(buffers)! =-1){ key.flip(); value.flip(); System.out.println(new String(key.array())); System.out.println(new String(value.array())); key.clear(); value.clear(); } channel.close();Copy the code3. Write data
3.1 Write from a single buffer
The single buffer operation is also very simple, returning the number of bytes written to the channel.
FileChannel channel = FileChannel.open(Paths.get("a.txt"), StandardOpenOption.WRITE);
ByteBuffer buf = ByteBuffer.allocate(5);
byte[] data = "Hello, Java NIO.".getBytes();
for (int i = 0; i < data.length; ) {
buf.put(data, i, Math.min(data.length - i, buf.limit() - buf.position()));
buf.flip();
i += channel.write(buf);
buf.compact();
}
channel.force(false);
channel.close();
Copy the code3.2 Write from Multiple Buffers
FileChannel implements the GatherringByteChannel interface, which is echoed by ScatteringByteChannel. You can write data from multiple buffers to a channel at once.
FileChannel channel = FileChannel.open(Paths.get("a.txt"), StandardOpenOption.WRITE); ByteBuffer key = ByteBuffer.allocate(10), value = ByteBuffer.allocate(10); byte[] data = "017 Robothy".getBytes(); key.put(data, 0, 3); value.put(data, 4, data.length-4); ByteBuffer[] buffers = new ByteBuffer[]{key, value}; key.flip(); value.flip(); channel.write(buffers); channel.force(false); // Flush data to disk channel.close();Copy the code3.3 Data Flushing
To reduce disk access times, files may not be flushed out to disks immediately after being accessed through the file channel. In this case, if the system crashes, data will be lost. To reduce this risk, the force() method should be called to force the data to be flushed out to disk after an operation with important data.
Whenever the force(metaData) method is invoked, an I/O operation is performed regardless of whether the file has been modified or not, even if the file channel is opened in read-only mode. The metaData parameter specifies whether metaData (for example, access time) is also flushed to disk.
channel.force(false); // Flush data to disk, but not metadataCopy the code4. Lock file
You can obtain a file lock by calling the FileChannel lock() or tryLock() method, specifying the starting position, lock size, and whether or not to share the lock. If no arguments are specified, the default arguments are position = 0, size = long. MAX_VALUE, shared = false.
Position and size do not need to be exactly the same as the file. Both position and size can exceed the size range of the file. For example, if the file size is 100, you can specify location 200 and size 50. When the file size expands to 250, the portion [200,250] is locked.
The shared parameter specifies whether it is exclusive or shared. To obtain a shared lock, the file channel must be readable; To obtain an exclusive lock, the file channel must be writable.
Because Java’s file lock maps directly to the operating system’s file lock implementation, the file lock is acquired on behalf of the entire virtual machine, not the current thread. If the operating system does not support a shared file lock, the file lock is changed to an exclusive lock even if the file lock is specified as shared.
FileLock lock = channel.lock(0, Long.MAX_VALUE, false); System.out.println("Channel locked in exclusive mode."); Thread.sleep(30 * 1000L); // lock 30 s lock.release(); Lock = channel.lock(0, long. MAX_VALUE, true); System.out.println("Channel locked in shared mode."); Thread.sleep(30 * 1000L); // lock 30 s lock.release();Copy the codeIn contrast to lock(), tryLock() is non-blocking and returns immediately whether or not the lock can be obtained. If tryLock() is trying to lock an area that is already locked by another process in the operating system, null is returned; Lock () blocks until a lock is acquired, the channel is closed, or the thread is interrupted.
5. Channel conversion
The normal read/write method is to use a ByteBuffer buffer as a container for data. But in the case of data interaction between two channels, using buffers as a medium is redundant. File channels allow data to be entered directly from a ReadableByteChannel and written directly to a WritableByteChannel. TransferFrom (ReadableByteChannel SRC, position, count) and transferTo(position, count, WritableChannel target) method.
These two methods are more efficient than using ByteBuffer as a medium for data transmission between channels. Many operating systems support file system caching, and copying between files may not actually occur.
TransferFrom or transferTo does not change position after being called.
The following example is a tool method from a Spring source.
public static void copy(File source, File target) throws IOException {
FileInputStream sourceOutStream = new FileInputStream(source);
FileOutputStream targetOutStream = new FileOutputStream(target);
FileChannel sourceChannel = sourceOutStream.getChannel();
FileChannel targetChannel = targetOutStream.getChannel();
sourceChannel.transferTo(0, sourceChannel.size(), targetChannel);
sourceChannel.close();
targetChannel.close();
sourceOutStream.close();
targetOutStream.close();
}
Copy the codeIt is important to note that in some cases it is not guaranteed that all data will be transferred after these two conversion methods are called, and the exact number of bytes transferred depends on the value returned. For example, data sent from a non-blocking SocketChannel cannot be transmitted all at once, or data sent from a file channel to a non-blocking SocketChannel cannot be transmitted all at once.
In the following example, the client connects to the server and downloads a file called video.mp4 from the server. The file exists in the current directory.
Examples of errors:
/** ServerSocketChannel ServerSocketChannel = serverSocketChannel.open (); // Open the server channel serverSocketChannel.bind(new InetSocketAddress(9090)); / / bind port SocketChannel clientChannel = serverSocketChannel. The accept (); SocketChannel FileChannel FileChannel = filechannel. open(path.get ("video.mp4"), standardOpenOption.read); // open the fileChannel filechannel.transferto (0, filechannel.size (), clientChannel); // File channel data output to socket channel, output range is the entire file. /** Client **/ SocketChannel SocketChannel = socketChannel. open(new InetSocketAddress("localhost", 9090)); FileChannel FileChannel = FileChannel. Open (path.get (" video-download.mp4 "), StandardOpenOption.TRUNCATE_EXISTING, StandardOpenOption.WRITE, StandardOpenOption.CREATE); // Open filechannel. transferFrom(socketChannel, 0, long.max_value); // [error in non-blocking mode] filechannel.force (false); // Make sure data is flushed out to diskCopy the codeThe correct posture is: When transferTo/transferFrom, a loop should be used to check whether the actual output size is consistent with the expected output size. In particular, the channel is in non-blocking mode and the transmission cannot be completed in one time.
So the correct conversion on the server side is:
long transfered = 0;
while (transfered < fileChannel.size()){
transfered += fileChannel.transferTo(transfered, fileChannel.size(), clientChannel);
}
Copy the codeIn this case the client is using a blocking mode, the service side channel closed output (socketChannel. ShutdownOutput ()) later transferFrom exit, the server normally closed pipeline data transmission under the condition of not wrong, don’t deal with non-normal off here. (Full code).
6. Intercept files
Filechannel. truncate(long size) Truncates a specified file. Contents after the specified size are discarded. Size can exceed the size of the file, and nothing is intercepted or added.
FileChannel fileChannel = FileChannel.open(Paths.get("a.txt"), StandardOpenOption.WRITE);
fileChannel.truncate(1);
System.out.println(fileChannel.size()); // 输出 1
fileChannel.write(ByteBuffer.wrap("Hello".getBytes()));
System.out.println(fileChannel.size()); // 输出 5
fileChannel.force(true);
fileChannel.close();
Copy the code7. Map files to the direct memory
The FileChannel FileChannel maps the specified range of a file to the program’s address space. The mapping part is represented by an object called MappedByteBuffer, a subclass of the byte buffer, which manipulates the file. In contrast, the previous sections manipulate files by manipulating file channels and HeapByteBuffer, a byte buffer in heap memory.
Allocate buffer by bytebuffer.allocate () is a HeapByteBuffer that exists in the JVM heap; Filechannle.map () maps files to direct memory and returns an MappedByteBuffer that exists outside of the heap in direct memory; The memory is valid until the MappedByteBuffer object itself is recycled.
Main memory Main memory JVM process memory HeapByteBuffer JVM heap memory A) HeapByteBuffer in memory b) MappedByteBuffer in memory MappedByteBuffer
7.1 Principles of Memory Mapping
The previous operation on the file using the heap buffer ByteBuffer and the FileChannel FileChannel used the read()/write() system call. The data is read from the I/O device to the kernel cache and copied from the kernel cache to the user-space cache, which is the JVM’s heap memory. A mapped disk file uses the mmap() system call to map the specified portion of the file into the program address space. Data interaction occurs between the I/O device and user space and does not need to go through the kernel space.
File kernel space User space cache I/O device cache File kernel space User space cache I/O device cache A) Common I/O b) Memory-mapped I/O
Although mapping disk files reduces a data copy, mapping files to memory is inherently expensive for most operating systems. If the operating file is small, only tens of kilobytes, the benefits of mapping the file will not outweigh the overhead. Therefore, large files are mapped to direct memory only when they are manipulated.
7.2 Mapping Buffer Usage
File channel FileChanle maps files to application memory using member methods map(MapMode Mode, Long Position, long Size).
FileChannel fileChannel = FileChannel.open(Paths.get("a.txt"), StandardOpenOption.READ, StandardOpenOption.WRITE); MappedByteBuffer buf = filechannel.map (filechannel.mapmode.read_write, 0, filechannel.size ()); // Map the entire file to memoryCopy the codeMode indicates the open mode. It is an enumeration value. The value can be READ_ONLY, READ_WRITE, or PRIVATE.
+ If the mode is READ_ONLY, buF cannot be written.
+ If the mode is READ_WRITE, the fileChannel channel must have read and write permissions on files. A write to the BUF takes effect on the file, but does not guarantee immediate synchronization to the I/O device;
+ If the mode is PRIVATE, the fileChannle channel must have read and write permissions on files. But changes to a file are not propagated to the I/O device, but instead copy a copy of the data in memory. Changes made to the file are not visible to other threads and processes.
Position specifies where the start of the file is mapped to memory.
Size Specifies the size of the mapping. The value is a non-negative int integer.
After the map() method is called, the returned MappedByteBuffer is separated from the fileChannel. Turning off fileChannel has no effect on buF. Meanwhile, calling force(metaData) in FileChannel does not work if the force() method in MappedByteBuffer is called without arguments to ensure that data modified to buF is synchronized to the file I/O device.
You can now manipulate the file by manipulating the buffer. However, the mapped content exists in the off-heap memory of the JVM program. This part of memory is virtual memory, meaning that the contents of the BUF are not necessarily in physical memory. To load the contents into physical memory, you can call the load() method in MappedByteBuffer. Alternatively, isLoaded() can be called to determine whether the contents of the BUF are in physical memory.
FileChannel fileChannel = FileChannel.open(Paths.get("a.txt"), StandardOpenOption.WRITE, StandardOpenOption.READ); MappedByteBuffer buf = fileChannel.map(FileChannel.MapMode.READ_WRITE, 0, fileChannel.size()); fileChannel.close(); System.out.println(buf.capacity())); Filechannel.size () system.out.println (buf.limit()); filechannel.size () system.out.println (buf.limit()); Filechannel.size () system.out.println (buf.position()); filechannel.size () system.out.println (buf.position()); // output 0 buf.put((byte)'R'); // Write content to buf.compact(); Buf.force (); // Flush the data out to the I/O deviceCopy the codeNodule 8.
1) The FileChannel can read data from an I/O device into a byte buffer, or write data from a byte buffer to an I/O device.
2) The file channel can be transferTo a writable channel or transferFrom a readable channel. This is a more efficient way to transfer data between channels than using buffers.
3) File channels can map portions of a file into JVM off-heap memory, which is suitable for large files but not for small files because the mapping process itself is expensive.
4) After important operations on files, data should be flushed out to disks to avoid data loss caused by operating system crashes.