This is the 30th day of my participation in the August Text Challenge.More challenges in August
One, foreword
In the previous series of articles, we learned about Fsync, Fwrite, Fflush, Mmap, Write Barriers and Direct memory for Linux I/O IO) principle analysis and use, through the previous study I think we should know how we can safely write data to disk in the development process. When it comes to safely writing data to disk, the first thing that comes to mind is the fsync or O_SYNC identifier. But if we don’t use the fsync or O_SYNC flag, is there a way to get the kernel to write data from the Page Cache to disk as quickly as possible? Of course, let’s study the posix_fadvise function with you.
The posix_fadvise function
1, define,
It sets an intent to tell the kernel what will happen to the file data in the future, allowing the kernel to optimize the data.
Man7.org/linux/man-p…
2. Advice parameter
The important thing is the advice parameter. If we set different advice parameters to the file fd, the Linux kernel can optimize accordingly.
POSIX_FADV_NORMAL
Indicates that the application has no recommendations to provide about its specified data access mode. If there is no comment given an open file, this is the default assumption.
POSIX_FADV_SEQUENTIAL
The application needs to access the specified data order (with the previous high person reading the low offset).
POSIX_FADV_RANDOM
The specified data will be accessed in random order.
POSIX_FADV_NOREUSE
The data to be specified will be accessed only once.
POSIX_FADV_WILLNEED
The data to be specified will be accessed in the near future.
POSIX_FADV_DONTNEED
The specified data will not be accessed in the short term.
3. Use method
If the C language is used directly, it is relatively simple, refer to the official documentation.
If you’re using Java, it’s a little more complicated. Since Java itself does not support this, we need to introduce additional JNA packages, implemented by calling the underlying C functions directly. For details, see the use of posix_Fadvice in Ingite’s ignite direct-io package.
3. Application scenarios of POSIX_FAdvise
What scenarios do we need to use Posix_fadvise in real development? In the previous article, “Direct IO for Linux I/O”, we studied the use of Direct memory in Linux. We mentioned that there is also a cache in the kernel, but there is very little cache. So if we do not actively use fsync or O_SYNC to synchronize data to disk, we can tell the kernel that we will no longer use this data by setting POSIX_FADV_DONTNEED with the posix_fadvise function. This allows the kernel to clean as much data from the kernel as possible the next time it scans the Page Cache and synchronizes the data to disk. In this way, it is possible to experience near-real time falling of cached data ensuring direct memory.
Note that the kernel does its best to drop disks, but it does not guarantee that the next scan will drop disks.
So the POSIX_FADV_DONTNEED parameter is the focus of this article. We can set this parameter to file data so that the kernel can do its best to clear data from the kernel cache and write it to disk. So if our business scenario accepts near-synchronous data drop, we can use the POSIX_FADV_DONTNEED parameter instead of manually executing fsync after each write. It is clear that using the POSIX_FADV_DONTNEED parameter provides much better performance than manually calling fsync every time you write data.
Once you’ve learned the POSIx_fadvise function, the questions at the beginning of this article are easy to answer.
Four, practice
If you like this article or find it helpful to you, welcome one button three link support, thank you very much.
If you have any questions or comments about this article, please add lifeofCoder.