We learned about the disk overhead of an empty file in the previous article, “Does creating a new empty file Take up disk space? How much?” Let’s think about another question today. If we write only 1 byte to a file, does the actual disk usage of the file also take up 1 byte?

View a 1-byte file

As mentioned above, forget about the principle and go straight to the hands-on.

# mkdir tempDir
# cd tempDir
# du -h
0    .
# touch test
# du -h
0    .
Copy the code

After you create an empty file in a directory, the du command does not change the footprint of that folder. This is true because empty files only take up inodes. Ok, so let’s modify the file and add a letter

Echo "a" > test # du -h 4.0kCopy the code

Then check the space usage of the directory. We found that we went from 0 to 4K. So no matter how small the contents of a file are, even a single byte, the operating system will actually assign you 4K. Oh, and of course there is the amount of space used to store inodes and file names in folder data structures. So, don’t maintain a bunch of shredded files on your system. File again small, take up disk actually all a lot!

Note that my experimental environment is under the Ext file system. XFS might perform a little differently.

Let’s keep talking about 4K

/ext2/ext2.h /ext2.h/fs/ext2/ext2.h/fs/ext2/ext2.h/fs/ext2/ext2.h/fs/ext2/ext2.h/fs/ext2/ext2.h/fs/ext2/ext2.h

struct ext2_inode { ...... __le32 i_block[EXT2_N_BLOCKS]; # point to an array of blocks storing file data......Copy the code

This array is empty when the file has no data to store. After we write a byte, the file system needs to apply a block to store it, and when we do, we put Pointers in this array. Even if the contents of a file are only one byte, an entire Block will still be allocated because this is the smallest unit of work for the file system. How big is this block? Ext can check it by dumpe2fs.

#dumpe2fs -h /dev/mapper/vgroot-lvroot
......
Block size:               4096
Copy the code

On my machine, a Block is 4KB.

What if the file content is bigger

If you noticed, the size of block arrays defined in inodes is only EXT2_N_BLOCKS. If we look at the definition of this constant, we find that it is 15.

#define EXT2_NDIR_BLOCKS 12 #define EXT2_IND_BLOCK EXT2_NDIR_BLOCKS #define EXT2_DIND_BLOCK (EXT2_IND_BLOCK + 1) #define  EXT2_TIND_BLOCK (EXT2_DIND_BLOCK + 1) #define EXT2_N_BLOCKS (EXT2_TIND_BLOCK + 1)Copy the code

In terms of 4K block size, 15 blocks are only enough to store 15*4=60K files. This file size I believe you must not be satisfied, you save a large avi have to G. So how does Linux implement large file storage? Only 12 arrays hold block Pointers directly, and the rest are used for indirect indexes (EXT2_IND_BLOCK), secondary indirect indexes (EXT2_DIND_BLOCK), and tertiary indexes (EXT2_TIND_BLOCK).

Thus, the space available for a file is exponentially expanded. When the file is small, use direct index, disk IO less, good performance. When a file is large, it may take three IO to access a block, which is slightly slower, but with the support of page caching and directory item caching at the OS level, it is ok.

conclusion

File systems are managed by blocks, so no matter how small your file is, even if it’s only one byte, it consumes a whole block. This block size can be viewed with commands such as dumpe2fs. What if I want to change the block size? Sorry, can only reformat.

The development of internal training of hard disk album:

  • 1. Disk opening: remove the hard coat of mechanical hard disk!
  • 2. Disk partitioning is also a technical trick
  • 3. How can we solve the problem that mechanical hard disks are slow and easy to break?
  • 4. Disassemble the SSD structure
  • 5. How much disk space does a new empty file occupy?
  • 6. How much disk space does a 1-byte file occupy
  • 7. Why is the ls command stuck when there are too many files?
  • 8. Understand formatting principles
  • 9. Read file How much disk IO actually occurs per byte?
  • 10. When to initiate disk I/O write after one byte of the write file?
  • 11. Mechanical hard drive random IO is slower than you might think
  • 12. How much faster is an SSD server than a mechanical one?

My public account is “developing internal Skills and Practicing”. Here I am not simply introducing technical theories, nor only introducing practical experience. But to combine theory with practice, with practice to deepen the understanding of theory, with theory to improve your technical practice ability. Welcome you to follow my public number, also please share with your friends ~~~