[toc]

Original is not easy, more dry goods, public concern: Qiya cloud storage

Antecedents feed

The previous article introduced some commands and methods to explore the file system, and mentioned the kernel file system development is difficult, I said to take readers friends together to do a minimalist file system.

But the author can’t take delicate readers to wade into the core of a muddy water, right? But what should I do if I had put out my word?

Coincidentally, kernel developers have wrestled with this question as well, eventually providing a framework called FUSE that allows users to create file systems in user mode.

I hope that after reading this article, you have some understanding of the following questions:

  1. What is FUSE?
  2. What does FUSE do?
  3. What are the implementations of FUSE?

The purpose of this article is to help you mentally model the FUSE framework and familiarize yourself with the entire IO path of a user-mode file system.

What is FUSE?

The Linux kernel official documentation explains FUSE as follows:

What is FUSE? FUSE is a userspace filesystem framework. It consists of a kernel module (fuse.ko), a userspace library (libfuse.*) and a mount utility (fusermount).

FUSE is a framework for implementing user-mode file systems. The FUSE framework consists of three components:

  1. Fuse. Ko: receives IO requests from the VFS, encapsulates the IO and pipes it to user mode;
  2. User-mode lib library libfuse: parsing the protocol package forwarded from the kernel mode and disassembly it into conventional I/O requests;
  3. Mount tool fuserMount;

These are the three main elements of the FUSE framework, which we’ll explain below. These three components serve only one purpose: to allow IO to move freely between kernel and user states.

FUSE is commonly thought of as Filesystem in Userspace, often referred to as user-mode file systems.

The FUSE principle

Let’s take a look at the IO path to understand how FUSE works. First take a look at the wiki illustration of the ls -l/TMP/FUSE command:

The picture means:

  1. Background: a user-mode file system with mount point/tmp/fuse, the user binary file is./hello ；
  2. When you performls -l /tmp/fuseThe process is as follows:
    1. IO requests to the advanced kernel are passed through VFS to the kernel FUSE file system module;
    2. The kernel FUSE module sends requests to the user state from./helloThe program receives and processes. After the processing is complete, the response is returned in the original route.

Simplified IO animation schematic diagram:

From these two figures, the flow of FUSE IO should be clear. The kernel FUSE module does protocol wrapping and protocol parsing in the middle of the kernel state. Requests from VFS are accepted and forwarded to the user state using the FUSE protocol, and the user state response is received and sent back to the user.

FUSE in this IO path is meant to act as a transparent hub, completely invisible to the user. It’s easier to understand if we cover FUSE in the middle as a black box.

What to think about: the kernel’s fuse.ko module, and the Libfuse library. What do these two roles do?

Key points: these two modules, one in the kernel and one in the user mode, are used together, and the core function is protocol encapsulation and parsing.

For example, the kernel fuse.ko is used to accept I/O requests from VFS, encapsulate them into FUSE packets, and forward them to the user. At this point, the user-mode file system receives the FUSE packet, and in order to understand it, it must implement a FUSE protocol code that is transparent, common to the FUSE framework, and cannot be repeated by all user file systems. The Libfuse user library was born.

Going back to the opening question, what does FUSE do?

As should be clear from this, FUSE can transfer IO requests from VFS to the user state, which is then processed by the user application and responded to by the FUSE framework. Thus, the implementation of the file system can be put into the user mode implementation.

FUSE protocol format

If we look at the data format of the FUSE data transport (the format of the FUSE protocol), what do request packets and response packets look like? Curious?

The FUSE request packet

The FUSE request package is divided into two parts:

  1. Header: This is common to all requests, for exampleopenThe request,readThe request,writeThe request,getxattrRequest, the head has at least this structure,HeaderThe structure describes the entire FUSE request, with fields distinguishing between request types;
  2. PayloadThe thing is that each IO type will be different, for examplereadThe request doesn’t have that,writeThat’s what the request is becausewriteRequests carry data;

A packet consists of header and payload.

type inHeader struct {
	Len    uint32
	Opcode uint32
	Unique uint64
	Nodeid uint64
	Uid    uint32
	Gid    uint32
	Pid    uint32
	_      uint32
}
Copy the code
  • Len: Is the length of the entire request in bytes (Header + Payload)
  • Opcode: Types of requests, such as open, read, write, etc.
  • Unique: the Unique identifier of the request (and the corresponding one in the response)
  • Nodeid: specifies the Nodeid of the requested file and the Nodeid of the target file or folder.
  • Uid: indicates the user ID of the file/folder operation process
  • Gid: user group ID of the file/folder operation process
  • Pid: indicates the ID of the file/folder process

The FUSE response packet

The FUSE response package is also divided into two parts:

  • HeaderThis structure is also in the data header, and all IO type responses have at least this structure. This structure is used to describe the entire response request;
  • Payload: Each request type may be different, for examplereadThe request will have this because it will carryreadThe user data that comes out,writeThere would be no request;
type outHeader struct {
	Len    uint32
	Error  int32
	Unique uint64
}
Copy the code
  • Len: The length of the response in bytes (Header + Payload);
  • Error: response Error code, return 0 on success, other corresponding system Error code, negative;
  • 7. Unique: the Unique identification of a request made by a respondent, corresponding to the request;

The link between kernel mode and user mode

We now know the format of the data protocol, the forwarding and transport modules. One crucial point remains: the channel of the packet, the highway.

In other words, where does the kernel module “package” go? Where does the user program pick up the “package” from?

The answer is:/dev/fuseThe virtual device file is the bridge between the kernel module and the user program.

The kernel acts as a messenger in this process. The user’s IO comes in through a normal system call and goes to fuse, the kernel file system, which wraps the IO request into a specific format and passes it through the /dev/fuse pipeline to the user state. Before that, a daemon listens to the pipe, sees a message coming out, reads it, parses the protocol using the Libfuse library, and then the user file system code logic.

The schematic diagram is as follows (the unpacking step is omitted) :

The use of the FUSE

Now that you know the three components of the FUSE framework, the FUSE packet protocol, it’s time to try using the FUSE file system.

The following command is executed on ubuntu 16.

Does the Linux kernel support it?

As mentioned earlier, the kernel also has a fuse.ko module, which is common, and the kernel is located in the file system layer. If we want to make our own file system, the first step is to make sure that the kernel supports this module. You can simply run the following command. If no error is reported, your Linux machine supports the FUSE module and it has been loaded.

root@ubuntu:~# modprobe fuse
Copy the code

If Linux does not currently support the kernel module, an error is reported, such as (Ubuntu16) :

root@ubuntu:~# modprobe xyz
modprobe: FATAL: Module xyz not found inThe directory/lib/modules / 4.4.0-142 - genericCopy the code

Or you can go to /lib/modules/4.0-142-generic /kernel/fs/ to see if fuse is available.

These pre-knowledge, in the last article mentioned ha.

Mount fuse kernel file systems for easy administration

Fuse the fuse kernel file system can be mounted or unmounted to facilitate the management of multiple user systems. The fuse kernel file system Type name is fusectl and the mount command is:

mount -t fusectl none /sys/fs/fuse/connections
Copy the code

You can use the df -at command to view:

root@ubuntu:~# df -aT|grep -i fusectl
fusectl                     fusectl              0        0         0    - /sys/fs/fuse/connections
Copy the code

By mounting the kernel fuse file system, you can see all implemented user file systems as follows:

root@ubuntu:~# ls -l /sys/fs/fuse/connections/
total 0
dr-x------ 2 root root 0 May 29 19:58 39
dr-x------ 2 root root 0 May 29 20:00 42
Copy the code

/sys/fs/fuse/connections corresponds to two directories named Unique ID, which uniquely identifies a user file system. The fuse module builds two communication channels through the /dev/fuse device file, corresponding to two user file systems respectively.

root@ubuntu:~# df -aT|grep -i fusefusectl fusectl 0 0 0 - /sys/fs/fuse/connections lxcfs fuse.lxcfs 0 0 0 - /var/lib/lxcfs helloworld fuse.hellofs 0 0 0 -  /mnt/myfsCopy the code

Under each Uniqe ID directory, there are several files from which we can retrieve the state of the current user’s file system or interact with the FUSE file system, for example:

root@ubuntu:~# ls -l /sys/fs/fuse/connections/42/
total 0
--w------- 1 root root 0 May 29 20:00 abort
-rw------- 1 root root 0 May 29 20:00 congestion_threshold
-rw------- 1 root root 0 May 29 20:00 max_background
-r-------- 1 root root 0 May 29 20:00 waiting
Copy the code
  • Waiting file: cat finds the number of I/O requests currently being processed
  • Abort file: Any string written to this file terminates the user file system and all requests on it;

How do I mount a user file system?

This leaves just one more question: how do the user file systems (such as hellofs and LXCFS above) mount?

This brings us to the third component of the FUSE framework, the fusermount tool, which was created specifically to facilitate mounting user file systems.

fusermount -o fsname=helloworld,subtype=hellofs -- /mnt/myfs/
Copy the code

The role of the FUSE is that enables users to bypass the kernel code to write the file system, but please note that the file system to realize to the operation of the specific equipment must provide interfaces to use the device drivers, and device driver in the kernel space, then can be directly read/write block device file, is equivalent to only pick the file system to user mode, Users directly manage block device space.

What is FUSE?

There are numerous examples of user-mode file systems that implement FUSE, such as GlusterFS, SSHFS, CephFS, Lustre, GmailFS, EncFS, S3FS, etc

These are all user-mode applications that implement FUSE:

  • GmailFS allows us to manage files as well as emails;
  • S3FS allows us to manage files like manage objects;

Original is not easy, more dry goods, public concern: Qiya cloud storage

conclusion

From this article, we learned about FUSE, which is summarized as follows:

  1. The FUSE framework was developed by kernel developers to meet increasingly diverse user needs, making it possible for user-mode applications to participate in IO path processing.
  2. The FUSE framework consists of three components: kernel FUSE module and user modelibfuseLibrary,fusermountMount tool;
  3. The kernel fuse module is used to accept VFS requests and pass/dev/fuseThe established pipe sends the encapsulated request to the user state;
  4. libfuseUser-mode encapsulates the library code used to parse the FUSE packet protocol, serving all user-mode file systems.
  5. fusermountIs the user mode file system used to mount tools;
  6. /dev/fuseIs the link between kernel fuse and user-mode file system;
  7. The GIF above demonstrates the complete IO path of the FUSE filesystem. Did you learn FEI?

Original is not easy, more dry goods, public concern: Qiya cloud storage

Afterword.

You have all the prerequisites for making your own file system, including how to view, learn about, mount, and unmount the file system. Now that you’ve learned about the FUSE framework, it’s time to get a real working file system, so stay tuned.

Original is not easy, more dry goods, public concern: Qiya cloud storage