Hello everyone, I am Xiao Yi

Today’s article is based on a tip at work that involves automating everyday tasks.

Let’s say you have to do some repetitive tasks every day, such as putting out a report, crunching a statistic, sending an email, etc

Then you can give this task to the computer to let it automatically complete for you every day, and you just need to calmly open the thermos cup, quietly soak a cup of goji berries


Today, I will mainly use the service cron under Linux. Ok, let’s start


1. Install the cron

Virtually all Linux distributions have the CRon tool preinstalled by default.

Even if cron is not pre-installed, it is easy to install manually with a few simple commands

Check whether cron service cron status is preinstalledCopy the code

Install and start the service

Installation: apt-get install cron Start /stop/restart: service cron start/stop/restart Query the current task: crontab -lCopy the code


2. Check the installation

After the installation is complete, run the status command to check whether the installation is successful

The installation is successful if the following information is displayed:

Also, in Ubuntu you might get a message like this:

This also means it can be used normally


3. The cron usage

There are a few simple uses of cron that you can look at, and a case study that explains how to use cron

First, list the crON jobs planned by the current user:

crontab -l
Copy the code

View other users’ cron jobs:

Crontab - l - u usernameCopy the code

Remove planned CRON jobs:

Crontab - rCopy the code


4. Schedule crontab schedules

First, add or update tasks in crontab with the following command

The first entry will ask you to select an editor, which is your custom.

The selection will bring you to a screen like this:

If you have used Vim, you should be familiar with this interface. Similar operation: press A to start editing, press ESC to enter wq to save and exit

The emphasis is on the bottom paragraph:

m h dom mon dow commmand
Copy the code

This is an introduction to the crontab scheduling job, which can be used to set scheduled tasks.

The specific syntax looks like this:

M h dom mon dow command * * * * * command -- -- -- -- -- - | | | | | | | | | | | - advance execution command | | | | -- -- -- -- -- Said week 0 ~ 7 (which can be used on Sunday said 0 or 7) | | | -- -- -- -- -- -- -- said in 1 ~ 12 | | -- -- -- -- -- -- -- -- -- said date 1 ~ 31 | -- -- -- -- -- -- -- -- -- -- -- said hours 1 ~ 23 (0, 0) -- -- -- -- -- -- -- -- -- -- -- -- -- The value ranges from 1 to 59 minutes. The value is * or */1Copy the code


A few simple application cases:

  • Every day at 02:00
0 2 * * * command
Copy the code
  • Work at 5:00 and 17:00 every day
0, 5,17 * * * commandCopy the code
  • Perform a task every 10 minutes
*/10 * * * * command
Copy the code
  • Run at 17:00 on a Sunday in a given month
0 17 * jan,may,aug sun command
Copy the code

About the commonly used on these, the use of more case can also refer to this link: linux.51yip.com/search/cron…


The command in the above example indicates the specific task you need to perform, such as printing a paragraph:

echo "Hello xiaoyi" >> /tmp/test.txt 
Copy the code

Or print this paragraph in TXT:

echo "Hello xiaoyi" >> /tmp/test.txt 
Copy the code

Or you need to execute a Python script:

python demo.py filepath
Copy the code

The following filepath indicates the input parameter args. This may be used by some students, such as in the following example, where the file download path is required.


5. Actual combat

Now that we’re clear on the top, we can start the main event of the day.

First of all, we need to download the latest task data from the FTP server every day. After downloading the data to the local computer, we use Python to conduct data summary and statistics. Finally, we save the results into the database.

1) Python scripts

First, you need a Python script to do the following:

  • Get the date of the latest data from the database
  • Download the latest data from FTP to the local PC
  • Collect the latest local data
  • The statistical results are stored in a database for summary
  • Email notification

The general pseudocode for the above process looks like this:

if __name__ == '__main__':
    """ Date of latest data acquisition """
    latest_date = get_max_date()
    Create a folder with the name of the latest date
    download_dir = os.path.join(sys.argv[1], latest_date)
    if not os.path.exists(download_dir):
        os.makedirs(download_dir)
        
    """ Download the latest data from FTP """
    download_file(latest_date, download_dir)
    """ Process and save the latest data. ""
    process_data(latest_date, download_dir)
Copy the code

Mail monitoring can catch exceptions with a try catch and send messages when exceptions occur

Python is used to edit mail contents and send them


② Write crON tasks

Open crontab, edit the following information to the last line, save and exit

Crontab will automatically update the task list in real time. You can also restart the cron service by using the restart command.

Here’s a tip: Fill all paths with absolute paths


③ Effect monitoring

If there is no problem with the Python code, the task will execute periodically.

It is recommended that you run your own command separately from the console and write it into the Cron task list when there is no problem.

The final screenshots of xiao Yi’s scheduled task running are as follows:

At the bottom is the FTP file download, at the top is the data summary statistics




A little digression

If your daily work has a lot of repetitive tasks, such as daily indicators collection, data collection statistics, automatic mail forwarding and so on

Once you can set up the logic with a script, automated tasks can do the same, requiring you to check your email every day to see if there are any errors.


Original is not easy, welcome to like oh

Article first: public number [autumn xiao Yi]

Article synchronization: Nuggets of gold, Jane book, CSDN


Original link:Here’s a simple job tip: Automate Python in three steps