1. The XXL – JOB is introduced

Xxl-job is a distributed task scheduling platform. Its core design goal is to develop quickly, learn simply, be lightweight, and be easy to expand. CRUD operations can be performed on a task through the Web page, which is easy to operate and can be started in one minute. You can dynamically change the task status, start or stop a task, and terminate a running task. Official Address Chinese version: www.xuxueli.com/xxl-job Gitee Address: gitee.com/xuxueli0323…

2. Application scenarios

We deployed the XXL-Job scheduling system on two server instances. The XXL-job scheduling system is mainly used to execute the scripts of each service system, such as sending short messages, changing user credits, generating BI reports, and so on.

3. Problem review

The job system runs normally every day without any major problems. One morning, the test girl reported a question to the R&D department: how did the user’s daily and weekly points increase? It’s just twice as much. Look at it. Hurry! Dude, that’s great. I immediately consulted the guy sitting next to me to see if this had ever happened before. He said it was rare, almost none. So I found the place to process the daily integral, and found that a timing script was being executed, which would be executed almost once every night. Then I logged in the background of the Job system to query the script and search for the date, and found a strange problem, as shown in the figure:

Two problems were found from this screenshot:

1. The script task is scheduled twice at the same time. Why is the script task scheduled twice? Because we have two server instances that trigger the scheduling task.

2. The script is executed twice in less than one minute.

4. Solutions

Looking at the code tells us that the script is executed only once a day, but the code level does not determine whether it has been executed, and multiple calls are evaluated multiple times. This leads to errors in the calculation of data. For example, the user’s credits should have been reduced by 10 points, but because the dispatching system scheduled twice, the user’s credits were reduced by 20 points. It is easy to solve the problem when you find out the specific reason, which is to avoid repeated execution of the script.

4.1 Using XXL-Job Configuration to Avoid Duplicate Scheduling (Recommended)

The recommended solution is to use the “task scheduling lock table” to avoid cluster scheduling at the same time.

4.2 Avoid duplicate scheduling at the code level

Plan 1:

Check the update time where the script was executed to see if it was executed today, and you’ll be done. There is another problem with this processing, however, as there is also some script code that does not determine whether it has been executed. There are a lot of things that need to be modified, and we need to find other, more convenient ways.

Scheme 2:

In the configuration file, define which scripts cannot be scheduled repeatedly and the Redis expiration time. Use Redis to record that the script has been executed. Check whether the script has been executed during the second execution. If the script has been executed, exit the execution process. Implementation process: 1. Define the configuration file

        
    return [
        // Define the interface for deferred invocation
        'job_delay_request_api_config'= > ['cache_time'= >1800.// Expiration time in seconds
           'list'= > ['UserChangeLog/day'.// Daily integral statistics
                'UserChangeLog/week'.// Weekly score statistics],]];? >
Copy the code

2. Check whether the script is executed on the parent controller

      
    public function __construct()
    {
        // Get the controller and method names of the current schedule, which are used as key values
        $apiUrl = CONTROLLER_NAME . '/' . ACTION_NAME;
        // Get the interface that defines the deferred invocation in the configuration item
        $config = config('job_delay_request_api_config');
        // Check that the interface currently invoked is not in the deferred script configuration array
        if(! in_array($apiUrl.$config['list']) {return;
        }
        // Check whether the interface has been called
        $key = 'job_delay_request_api:' . $apiUrl;
        $redis = new Redis();
        $result = $redis->get($key);
        // If redis has been invoked, the script cannot be invoked again. Wait until redis expires and stop running the script
        if (!empty($result)) {
            exit('This scheduling task was executed at :' . date('Y-m-d H:i:s'.$result));
        }
        // Mark that the job task has been executed and record the execution time
        $redis->set($key.$config['cache_time'], time());
    }
? >
Copy the code

5. To summarize

Currently, scenario 2 in Section 4.2 is used to prevent tasks from being repeatedly scheduled by reading the pre-defined configuration and judging interception at the code level and specifying which scripts need to be intercepted. Why wasn’t 4.1 adopted? Since I am not engaged in Java, and my colleagues in Java are not familiar with this system, I did not adopt the method recommended by 4.1 official website. I will consider the method recommended by 4.1 official website in the future, which will be more reasonable and stable. Thanks for reading, and I’ll see you next time.