Familiar friends may know that after joining the architecture group, I have been doing various development AIDS for the team this year, including a server.

Recently, this server has been online for more than three months, and has been collecting a lot of data, including users’ various behavior operations, API call data and so on.

In a recent upgrade, I adjusted the directory structure of the source code and removed the database on a whim.

The one-day data recovery process began, and the end result was tragic, but mercifully insignificant, which can be discussed later.

File back

It should be noted that, since it is a micro service, the simplest SQLite database is used. What differentiates SQLite from other databases is that it provides only the most basic data services, but does not enable port listening. You can simply think of it as a file, and if it is deleted, it will disappear from the system in the same way as a regular file. So in general, if you use SQLite to save your data, you need to back it up regularly.

So although this blog is called “delete library”, but in fact, it can be simply understood as file retrieval.

rm -rf

I used the RM-rf command to delete the repository by mistake, which is of course a well-known and notorious command (so much so that when I asked the backend guy about it, he was like “Oh, you finally deleted the repository”).

Rm is the Linux command used to delete a file or directory

-f indicates the force.

Do not prompt before removing write-protected files.

-r Indicates that:

Allows circular deletion of directories and their contents when File is a directory

FYI: More references

Extundelete with disk storage

Extundelete is the next powerful data recovery tool in Linux. Extundelete uses information stored in the partition logs to try to recover files that have been deleted from the partition.

Let’s review computer basics a bit:

The hard disk

A hard disk is a magnetic data storage device. Data is stored “physically” on several disks. On each side of the disk, concentric circles with a rotating axis as the axis and a certain magnetic density as the interval are divided into tracks, and each track is divided into several sectors.

Primary boot sector and partition table

Sector 1 of track 0 cylinder 0 of a hard Disk is the primary Boot sector, including the Main Boot Record (MBR) and Disk Partition Table (DPT). The operating system divides the hard disk into several partitions through partition tables, and then creates file systems in each partition and writes data files.

Partition log

The partitioned logging system is a file system used to fix any inconsistencies caused by improper computer shutdown. Such shutdowns are usually caused by power outages or software problems. That is, data read and write operations of partitions are recorded.

From these logs, you can know the history of the operations in the partition.

Data is stored

The specific data storage principles are more detailed and will not be described here. For ease of understanding, I hereby interpret it as:

Operating system files in the form of the hard drive is a data area in hard disk record binary information, and by the operating system of a pointer to the physical address And operating system level “delete files”, which removes the “pointer” not “pointer pointing to the” within the original physical address, equivalent to be released, when the operating system needs, New data can be copied.

From this point of view, ExtunDelete tells the user what data to try to recover by looking at the partition log and finding the deleted pointer.

However, since the operating system can copy the spare disk at any time, the data cannot be retrieved if the physical address to be recovered has been overwritten.

DEMO

To desensitize, I use my own server and record the process of file recovery in detail.

The environment

• CentOS 7.4 64
• File directory: /root/sparrow /db.sqlite3

Delete the file

# rm -rf db.sqlite3
Copy the code

Install extundelete

# yum install extundelete -y
Copy the code

Mount the disk

First we query the disk on which the file or parent folder is located

# df /root/Sparrow/File system 1K- Block Used Available Used % Mount point /dev/vda1 41151808 2614640 36423736 7% /Copy the code

The first step is to mount the disk. After mounting the disk, it will not be written to further protect the site

umount /dev/vda1
Copy the code

/dev/vda1 is the disk where the file resides.

It is also ok not to mount the file, because in this way, the disk may be written by other processes, thus erasing the original data, so I did not retrieve the file in the end because I did not mount the file in time.

inode

Inode is the identifier of a file or folder in Linux. You can see this through ls-id.

Read the inode value of the root directory:

# ls -id /
2 /
Copy the code

Now we know that the inode of the disk root directory is 2.

Extundelete Queries recoverable data

Query the recoverable information in the root directory of the disk

# extundelete /dev/vda1 --inode 2
Copy the code

If the disk is not mounted, a message is displayed.

Results obtained:

Root inode = 131073; /dev = 131073;

# extundelete /dev/vda1 --inode 131073
Copy the code

/Sparrow 262194 /dev /Sparrow 262194

# extundelete /dev/vda1 --inode 262194
Copy the code

Delete message found:

You can see that DB. sqlite3 is identified as Deleted.

Extundelete Restores data

Run –restore-directory to restore the specified directory

# extundelete /dev/vda1 --restore-directory /root/Sparrow/db.sqlite3
Copy the code

Or run –restore-all to restore all recoverable data

# extundelete /dev/vda1 --restore-all
Copy the code

After the command is executed, a RECOVERED_FILES folder is generated in the current path, containing all restored files.

If the disk has been read/written and cannot be recovered, a message similar to the following is displayed:

Loading filesystem metadata ... 320 groups loaded.
Loading journal descriptors ... 27896 descriptors loaded.
Searching for recoverable inodes in directory /root/Sparrow/db.sqlite3 ...
120 recoverable inodes found.
Looking through the directory structure for deleted files ...
120 recoverable inodes still lost.
No files were undeleted.
Copy the code

Afterword.

This time “delete library” event, gave me a very good lesson. Data on the server must be backed up or Dr In a timely manner to prevent loss. At the same time, this time also took the opportunity to learn the file recovery.

Of course, I hope that we will not appear such a tragedy 😂.