1. Identify the problem

When I found that I could not log in to the website, the interface failed to call, and I logged in to MySQL after timeout, I found that I could only log in to Ali Cloud, and the disk was full. At that time, I was surprised, how could the disk be full without anything installed?

Enter the following command to view the current disk capacity

df -h
Copy the code

2. Locate the large-capacity disk

This is the time to look at the large volume folder, locate the problem

Enter the following command

du -h --max-depth=1
Copy the code

So, why is the root folder so big when there’s nothing in it?

Hidden folder, go to root folder, continue to enter the command view

So many configuration folders…

The original is PM2 problem, then why PM2 will occupy such a large capacity!!

3. Solve the problem of large PM2 files

3-1. Find the cause of the problem

Then I googled the disk capacity of the log files, so I tried to erase the log files to see if I could fix the problem

3-2. Clear PM2 log files

To disable PM2, my Node process ID is 0

  1. Enter the command to shut down the Node process
pm2 stop 0
Copy the code

If there are multiple processes in PM2, you need to enter the following command to shut down all of them

pm2 stop all
Copy the code
  1. Enter the command to delete PM2 logs
pm2 flush
Copy the code

As you can see, all of a sudden, 35 G is missing, which is the PM2 log problem

4. Permanently resolve the log file size

This time the problem is fixed, but you can’t check the log once a week or once a month to see if it is too large and then delete it manually…

I did a Google search for PM2 logs and found a plug-in for log management called PM2 Logrotate

4-1. Install the plug-in PM2-Logrotate

Input order

pm2 install pm2-logrotate
Copy the code

Waiting for installation to complete

4-2. Pm2-logrotate Specific configuration parameters:

Configuration items Introduction to the
Compress Whether to use gzip to compress logs
max_size The size of a single log file, such as 1K in the figure above (this is too small, the actual file size is not strictly 1K)
retain If the number of retained log files is set to 10, the earliest log files will be deleted when the number reaches 10
dateFormat The default date format in the log file name is YYYY-MM-DD_hh-MM-ss. Note that the set log name + this format. If the set log name is abc_YYYY-MM-DD_hh-MM-ss
rotateModule Pm2 itself logs are also split
workerInterval Set the size of the monitoring log for several worker processes. The minimum size is 1
rotateInterval Set mandatory split. The default value is 0, 0 * * *, which means split every night at 0

4 – (3) set the pm2 – logrotat

Here we set the old log files to be automatically deleted when the number of log files exceeds 50

pm2 set pm2-logrotate:retain 50
Copy the code

After the setup, we see the retain changed from the default 30 to 50

pm2 conf pm2-logrotate

5. Set the PM2 to start automatically upon startup

Pm2 startup, this command will generate a pm2-root.service file in the /etc/systemd/system/ directory to start the PM2 service on the system.

Pm2 save: Saves all applications running on pM2 to /root/.pm2/dump.pm2, and reads the contents in the file to start related applications upon startup and restart.

6. Conclusion

Later, I thought it might be the node-schedule timer package used in Node. I deleted the file without looking at the log content of PM2 this time, and I couldn’t locate whether it was the cause or not. I will confirm it again after a month

To use this package, see my article adding a scheduled task using Node-schedule in NodeJs from scratch

The resources

Server – Save your disk and delete the pM2 runtime logs in time

Pm2 Log management PM2-Logrotate Describes how to manage logs

Controls the pM2 log file size

Use PM2 to deploy the NodeJS project

Process manager PM2 operation and maintenance summary