As open source products become more and more popular, as a Linux operation and maintenance engineer, it is very important to be able to clearly identify whether abnormal machines have been hacked. Based on my own work experience, I have sorted out several common cases of machine hacking for reference. The following information is displayed on CentOS 6.9, similar to other Linux distributions.
1. The intruder may delete the logs of the machine. You can run the following command to check whether the logs still exist or are deleted:
2. An intruder may create a new file to store the username and password. Check the /etc/passwd and /etc/shadow files. View the last successful login event and the last unsuccessful login event of the machine, corresponding to the log “/var/log/lastlog”.
4. Run the who command to view all current login users in the /var/run/utmp log file
5. Run the last command to view the log file /var/log/wtmp that has been logged in since the machine was created
6. View the connection time (hours) of all users in the /var/log/wtmp log file.
7. View the /var/log/secure log file to discover information about intruders
8
The a. TAB command is used to check the PID of the abnormal process
B. Search for the executable file of the process in the virtual file system directory
9. If you are sure that the machine has been hacked and important files have been deleted, try to retrieve the deleted files. Assume that the intruder deleted the /var/log/secure file. Perform the following operations to recover the /var/log/secure file: 1> When a process opens a file, it still exists on the disk as long as the process keeps the file open. This means that the process does not know that the file has been deleted, and it can still read and write to the file descriptor provided to it when it opens the file. This file is not visible except for the process because its corresponding directory index node has been removed. 2> In the /proc directory, which contains various files that reflect the kernel and process tree. The /proc directory mounts a memory-mapped region, so these files and directories do not exist on disk, so when we read and write these files, we are actually fetching information from memory. Most lsOF related information is stored in a directory named after the PID of the process, that is, /proc/1234 contains information about the process whose PID is 1234. There are various files in each process directory that allow applications to simply learn about the memory space of the process, the list of file descriptors, symbolic links to files on disk, and other system information. The LSOF program uses this and other information about the internal state of the kernel to produce its output. Therefore, lsOF can display the process file descriptor and related file name information. That is, we can find information about the file by accessing the file descriptor of the process. 3> When a file is accidentally deleted from the system, it can be recovered from the /proc directory using lsof, as long as the file is still being accessed by processes in the system.
Assume that the intruder deleted the /var/log/secure file. Perform the following operations to recover the /var/log/secure file:
A. View the /var/log/secure file and find that the file does not exist.
B. Run the lsof command to check whether /var/log/secure is enabled
C. The file descriptor of PID 1032 (rsyslogd) is 4. You can also see that /var/log/secure has been marked as removed. So we can see the corresponding information in /proc/1032/fd/4 (each numerically named file under fd represents the corresponding file descriptor for the process) as follows:
D. View /proc/1264/fd/4 to obtain the data to be restored. If the corresponding data can be viewed through the file descriptor, it can be redirected to the file using I/O redirection, such as: e. Check /var/log/secure again and the file already exists. For many applications, especially log files and databases, this method of recovering deleted files is very useful. Rsyslogd restart /etc/init.d/rsyslogd restart
