Hello, everyone! I am Nai Nai, a fat boy in the subway
Today we are going to take a look at the Spring Boot application, which timing task technology selection ~
This article provides the complete code sample, see github.com/YunaiV/Spri… Lab-28 directory of.
Original is not easy, to point a Star hey, together blunt duck!
😜 to head to tail, there is a day off!
1. An overview of the
In the colorful black requirements of the product, there is a kind of requirement that needs to be executed regularly, at this time, the scheduled task needs to be used. For example, scan overdue orders every minute, clear log files every hour, collect the previous day’s data and generate reports every day, push payroll at the beginning of each month, annual birthday reminder and so on.
Among them, Nai nai likes “the push of the paycheck at the beginning of each month” the most. What about you?
In the JDK, two classes are built in to implement scheduled tasks:
- Java.util. Timer: You can create java.util.TimerTask to schedule tasks that are executed sequentially in the same thread, affecting each other. In other words, for multiple TimerTasks in the same Timer, if one TimerTask is in execution, other TimerTasks can only queue up and wait even if they reach the execution time. Because Timer is serial and has pits, ScheduledExecutorService was introduced by JDK later, and Timer is basically no longer used.
- Java. Util. Concurrent. ScheduledExecutorService: new in JDK 1.5, based on the timing of the thread pool design task, every scheduling tasks assigned to the concurrent execution in the thread pool, mutually influence. In this way, ScheduledExecutorService solves the Timer serialization problem.
In daily development, we rarely use Timer or ScheduledExecutorService directly to fulfill the requirements of scheduled tasks. There are several main reasons:
- They only support scheduled scheduling according to the specified frequency, not the specified time directly. We need to calculate by ourselves in combination with Calendar to realize the scheduling of complex time. For example, every day, every Friday, 2019-11-11 and so on.
- They are at the process level, and we need to deploy multiple processes to achieve high availability for scheduled tasks. In this case, you need to wait for more considerations. The same task cannot be executed repeatedly in multiple processes at the same time.
- A project may have many scheduled tasks and require unified management. In this case, secondary encapsulation must be performed.
Therefore, under normal circumstances, we will choose professional scheduling task middleware.
As for the term “task”, there is also a term “homework”. In English, there are tasks and jobs. The essence is the same. This article uses both.
Then, in general, the task is scheduled, timed execution. So you will see the word “scheduling” or “timing” in this or any other article.
In the Spring architecture, there are two built-in solutions for scheduled tasks:
-
The first, the Spring Task module of the Spring Framework, provides a lightweight implementation of scheduled tasks.
-
The second, Spring Boot 2.0, integrates the Quartz job scheduling framework and provides a powerful implementation of scheduled tasks.
Note: Quartz integration is already built into the Spring Framework. Spring Boot 1.X does not provide automated configuration of Quartz, while 2.X does.
In the Java ecosystem, there are a number of excellent open source scheduling task middleware:
-
Elastic-Job
Vipshop evolved Saturn on top of Elastice-Job.
-
Apache DolphinScheduler
-
XXL-JOB
At present, elastice-job and XXl-job are mainly used in China. From the information learned by Nai, the team using XXL-job will be more, mainly because it is easier to get started and the operation and maintenance function is more perfect.
In this article, we will start in the order of Spring Task, Quartz and XXL-job. At the end of the article, we will briefly talk about the implementation of distributed scheduled tasks.
2. Quickly start Spring Task
Example code for repository: Lab-28-task-demo.
Considering that we rarely use Spring Task in real world scenarios, this section will be brief. If you are interested in Spring Task, you can read the Spring Framework Documentation — Task Execution and Scheduling for yourself. It contains detailed documentation related to Spring Task.
In this section, we use the Spring Task feature to implement a scheduled Task that prints a line of execution logs every 2 seconds.
2.1 Importing Dependencies
In the POM.xml file, introduce related dependencies.
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.2.1. RELEASE</version>
<relativePath/> <! -- lookup parent from repository -->
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>lab-28-task-demo</artifactId>
<dependencies>
<! -- Automatic configuration of Spring MVC -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
</dependencies>
</project>
Copy the codeSince Spring Task is a module of the Spring Framework, we don’t need to specifically introduce it after we introduce the Spring-boot-starter-Web dependency.
At the same time, we introduced the spring-boot-starter-Web dependency because we wanted the project to start without terminating the JVM process automatically.
2.2 ScheduleConfiguration
In cn. Iocoder. Springboot. Lab28. Task. The config directory, create a ScheduleConfiguration class, configure Spring task. The code is as follows:
// ScheduleConfiguration.java
@Configuration
@EnableScheduling
public class ScheduleConfiguration {}Copy the code- Add the @enablesCheduling annotation on the class to enable Spring Task scheduling.
2.3 DemoJob
In cn. Iocoder. Springboot. Lab28. Task. The job package path, create DemoJob class, sample task class regularly. The code is as follows:
// DemoJob.java
@Component
public class DemoJob {
private Logger logger = LoggerFactory.getLogger(getClass());
private final AtomicInteger counts = new AtomicInteger();
@Scheduled(fixedRate = 2000)
public void execute(a) {
logger.info("[execute][execute for the ({}) time]", counts.incrementAndGet()); }}Copy the code- On the class, add
@ComponentAnnotate to create the DemoJob Bean object. - create
#execute()Method to print logs. Also, on this method, add@ScheduledComment, set the method to execute every 2 seconds.
Although @scheduled annotations can be added to multiple methods on a class, Nai nai’s personal custom is still a Job class, a Scheduled task. 😈
2.4 Application
Create the Application. Java class and configure the @SpringBootApplication annotation. The code is as follows:
@SpringBootApplication
public class Application {
public static void main(String[] args) { SpringApplication.run(Application.class, args); }}Copy the codeRun the Application class to start the sample project. The output logs are condensed as follows:
Initialize a task scheduler called ThreadPoolTaskScheduler2019-11-30 18:02:58.415 INFO 83730 --- [ main] o.s.s.c.ThreadPoolTaskScheduler : Initializing ExecutorService 'taskScheduler'# every2Second, perform the DemoJob task once2019-11-30 18:02:58.449 INFO 83730 --- [ pikaqiu-demo-1] C.I.S pringboot. Lab28. Task. Job. DemoJob: [the execute] [timing (1) times]2019-11-30 18:03:00.438 INFO 83730 --- [ pikaqiu-demo-1] C.I.S pringboot. Lab28. Task. Job. DemoJob: [the execute] [timing (2) times]2019-11-30 18:03:02.442 INFO 83730 --- [ pikaqiu-demo-2] C.I.S pringboot. Lab28. Task. Job. DemoJob: [the execute] [timing (3) times]Copy the code- From the log, we can see that a ThreadPoolTaskScheduler is initialized. After that, the DemoJob is executed every 2 seconds.
At this point, we have completed our introduction to the Spring Task scheduling function. In fact, Spring Task also provides asynchronous tasks, which we’ll cover in more detail in other articles.
Following “2.5@Scheduled” and “2.6 Application configuration files” two sections, is supplementary knowledge, recommended to have a look.
2.5 @ Scheduled
Scheduled annotation to set the execution schedule of Scheduled tasks.
Common attributes are as follows:
cronProperty: Spring Cron expression. For example,"0, 0, 12 * *?It is executed at noon every day."11, 11, 11, 11, 11?"It is executed at 11:11:11 on November 11 (hahaha). See for more examples and explanationsSpring Cron ExpressionsThe article. Note to callCompletion timeTime to start.fixedDelayProperty: Fixed execution interval in milliseconds. Note to callCompletion timeTime to start.fixedRateProperty: Fixed execution interval in milliseconds. Note to callStart timeTime to start.- The differences between fixedRate, fixedDelay, and cron for @scheduled Scheduled tasks are very similar.
Less commonly used attributes are as follows:
initialDelayProperty: Execution delay of the initialized scheduled task, in milliseconds.zoneProperty: Resolves the time zone to which the Spring Cron expression belongs. By default, the local time zone of the server is used.initialDelayStringProperties:initialDelayIs a string.fixedDelayStringProperties:fixedDelayIs a string.fixedRateStringProperties:fixedRateIs a string.
2.6 Applying a Configuration File
In application.yml, add the Spring Task scheduled Task configuration as follows:
spring:
task:
Spring Task schedules the configuration of tasks, corresponding to the TaskSchedulingProperties configuration class
scheduling:
thread-name-prefix: pikaqiu-demo- The prefix of the thread name for the thread pool. Scheduling - is the default value, and you are advised to set it based on your application
pool:
size: 10 # Thread pool size. The default value is 1, depending on your application
shutdown:
await-termination: true Whether to wait for scheduled tasks to complete when the application is closed. The default value is false. You are advised to set it to true
await-termination-period: 60 # Maximum time to wait for a task to complete, in seconds. The default value is 0, depending on your application
Copy the code- in
spring.task.schedulingConfiguration item, Spring Task scheduling Task configuration, correspondingTaskSchedulingPropertiesThe configuration class. - Spring Boot TaskSchedulingAutoConfiguration automation, configuration, realize the Spring Task automatically configure, create ThreadPoolTaskScheduler Task scheduler based on thread pool. Essentially, ThreadPoolTaskScheduler is an encapsulation of ScheduledExecutorService to enhance scheduling.
Note that spring. The task. The scheduling. Shutdown configuration items, in order to realize the spring task timing task elegant shut down. If the application starts to shut down and the Spring Bean used by the scheduled task is destroyed, for example, the database connection pool, then the scheduled task is still in execution. Once the database is accessed, an error may be reported.
- So, through configuration
await-termination = trueTo shut down the application and wait for the scheduled task to complete. In this way, when an application is shut down, Spring preferentially waits for ThreadPoolTaskScheduler to complete its task before starting Spring Bean destruction. - At the same time, considering that we cannot wait indefinitely for all scheduled tasks to finish, it can be configured
await-termination-period = 60, the maximum duration of waiting for the task to complete, in seconds. The specific waiting period can be set according to the needs of your application.
3. Quick start Quartz PC
Lab-28-task-quartz -memory
The company used Quartz as its task scheduling middleware in The early days of Nai nai’s internship. Given the high availability of scheduled tasks, we need to deploy multiple JVM processes. Comfortably, Quartz comes with its own clustering solution. It stores the job information in the relational database and uses the row lock of the relational database to realize the competition of executing jobs, so as to ensure that the same task cannot be executed repeatedly in the same time in multiple processes.
For those who are not familiar with Quartz, let’s take a look at a brief introduction:
FROM www.oschina.net/p/quartz
Quartz is an open source job scheduling framework written entirely in Java and designed for USE in J2SE and J2EE applications. It offers great flexibility without sacrificing simplicity. You can use it to create simple or complex schedules for executing a job.
It has many features, such as database support, clustering, plug-ins, EJB job prebuilds, JavaMail and more, support for Cron-like expressions, and more.
Three components are important in the Quartz architecture:
- Scheduler: the Scheduler
- -Blair: I don’t know.
- Job: task
For those of you who don’t know, read Quartz 101. Here, Nai – nai will not repeat repeat.
The FROM medium.com/@ChamithKod…
The overall architecture of Quartz
Quartz is divided into standalone and cluster modes.
- In this section, we will first learn Quartz single-player mode, which is relatively quick to get started.
- In the following section, we will learn about the Quartz cluster mode again. In a production environment, always, always use Quartz cluster mode to ensure high availability of scheduled tasks.
😈 below, let’s start our tour ~
3.1 Importing Dependencies
In the POM.xml file, introduce related dependencies.
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.2.1. RELEASE</version>
<relativePath/> <! -- lookup parent from repository -->
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>lab-28-task-quartz-memory</artifactId>
<dependencies>
<! -- Automatic configuration of Spring MVC -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<! -- Implement automatic configuration of Quartz -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-quartz</artifactId>
</dependency>
</dependencies>
</project>
Copy the codeSpecific role of each dependency, fat friends carefully look at the nai nai added all notes oh.
3.2 the sample Job
In cn. Iocoder. Springboot. Lab28. Task. Config. The job package path, let’s create a sample job.
Create class DemoJob01 for example scheduled task 01. The code is as follows:
// DemoJob01.java
public class DemoJob01 extends QuartzJobBean {
private Logger logger = LoggerFactory.getLogger(getClass());
private final AtomicInteger counts = new AtomicInteger();
@Autowired
private DemoService demoService;
@Override
protected void executeInternal(JobExecutionContext context) throws JobExecutionException {
logger.info("[executeInternal][Timed ({}) time, demoService is ({})]", counts.incrementAndGet(), demoService); }}Copy the code-
Inherit the QuartzJobBean abstract class and implement the #executeInternal(JobExecutionContext Context) method to execute the logic of custom scheduled tasks.
-
QuartzJobBean implements the org.Quartz.Job interface, which provides the dependency properties of the JobBean to be injected each time Quartz creates a Job to execute timing logic. For example, DemoJob01 requires the demoService attribute injected by @AutoWired. The core code is as follows:
// QuartzJobBean.java public final void execute(JobExecutionContext context) throws JobExecutionException { try { // Wrap the current object as a BeanWrapper object BeanWrapper bw = PropertyAccessorFactory.forBeanPropertyAccess(this); // Set the property to bw MutablePropertyValues pvs = new MutablePropertyValues(); pvs.addPropertyValues(context.getScheduler().getContext()); pvs.addPropertyValues(context.getMergedJobDataMap()); bw.setPropertyValues(pvs, true); } catch (SchedulerException ex) { throw new JobExecutionException(ex); } // Executes the abstract methods provided to the subclass implementation this.executeInternal(context); } protected abstract void executeInternal(JobExecutionContext context) throws JobExecutionException; Copy the code- Look like this, isn’t it a lot clearer. Don’t be afraid of the middleware source code, wondering which class or method, just click on it. Anyway, it doesn’t cost anything.
-
Counts attribute. We’ll show later that every DemoJob01 is created by Quartz and a new Job object is created to execute the Job. This is very important and very careful.
Create the DemoJob02 class for example scheduled task 02 class. The code is as follows:
// DemoJob02.java
public class DemoJob02 extends QuartzJobBean {
private Logger logger = LoggerFactory.getLogger(getClass());
@Override
protected void executeInternal(JobExecutionContext context) throws JobExecutionException {
logger.info("[executeInternal][I started executing]"); }}Copy the code- It’s a little simpler, just for the sake of the case.
3.3 ScheduleConfiguration
In cn. Iocoder. Springboot. Lab28. Task. The config directory, create a ScheduleConfiguration classes, configuration of the above two sample Job. The code is as follows:
// ScheduleConfiguration.java
@Configuration
public class ScheduleConfiguration {
public static class DemoJob01Configuration {
@Bean
public JobDetail demoJob01(a) {
return JobBuilder.newJob(DemoJob01.class)
.withIdentity("demoJob01") // Name is demoJob01
.storeDurably() // Whether the task is retained when there is no Trigger association. Since the JobDetail was created without Trigger pointing to it, it needs to be set to true to indicate retention.
.build();
}
@Bean
public Trigger demoJob01Trigger(a) {
// A simple scheduler
SimpleScheduleBuilder scheduleBuilder = SimpleScheduleBuilder.simpleSchedule()
.withIntervalInSeconds(5) / / frequency.
.repeatForever(); / / number.
// Trigger constructor
return TriggerBuilder.newTrigger()
.forJob(demoJob01()) // Corresponding Job is demoJob01
.withIdentity("demoJob01Trigger") // The name is demoJob01Trigger
.withSchedule(scheduleBuilder) // The corresponding Schedule is scheduleBuilder.build(); }}public static class DemoJob02Configuration {
@Bean
public JobDetail demoJob02(a) {
return JobBuilder.newJob(DemoJob02.class)
.withIdentity("demoJob02") // Name is demoJob02
.storeDurably() // Whether the task is retained when there is no Trigger association. Since the JobDetail was created without Trigger pointing to it, it needs to be set to true to indicate retention.
.build();
}
@Bean
public Trigger demoJob02Trigger(a) {
// Schedule constructor based on Quartz Cron expression
CronScheduleBuilder scheduleBuilder = CronScheduleBuilder.cronSchedule("0/10 * * * *? *");
// Trigger constructor
return TriggerBuilder.newTrigger()
.forJob(demoJob02()) // Corresponding Job is demoJob02
.withIdentity("demoJob02Trigger") // Name is demoJob02Trigger
.withSchedule(scheduleBuilder) // The corresponding Schedule is scheduleBuilder.build(); }}}Copy the code- Two configuration classes DemoJob01Configuration and DemoJob02Configuration are created to configure DemoJob01 and DemoJob02 Quartz jobs respectively.
- ========== DemoJob01Configuration ==========
#demoJob01()Method to create the JobDetail Bean object of DemoJob01.#demoJob01Trigger()Method to create the Trigger Bean object of DemoJob01. Which we useSimpleScheduleBuilderA simple schedule builder that creates a schedule that executes every 5 seconds and repeats indefinitely.- ========== DemoJob2Configuration ==========
#demoJob2()Method to create the JobDetail Bean object of DemoJob02.#demoJob02Trigger()Method to create the Trigger Bean object of DemoJob02. Which we useCronScheduleBuilderA scheduler constructor based on a Quartz Cron expression that creates everyThe firstA schedule that executes once every 10 seconds. Here, I recommend oneQuartz/Cron/Crontab expression generation tool online, which helps us generate Quartz Cron expressions and calculate the last 5 runtime times.
😈 Because JobDetail and Trigger usually come in pairs, It is customary for Nai to be configured as a Configuration Configuration class.
3.4 Application
Create the Application. Java class and configure the @SpringBootApplication annotation. The code is as follows:
@SpringBootApplication
public class Application {
public static void main(String[] args) { SpringApplication.run(Application.class, args); }}Copy the codeRun the Application class to start the sample project. The output logs are condensed as follows:
The Quartz QuartzScheduler is created and startedThe 2019-11-30 23:40:05. 92812-123 the INFO [main] org. Quartz. Impl. StdSchedulerFactory: Using the default implementationforThreadExecutor 23:40:05 2019-11-30. 92812-130 the INFO [main] org. Quartz. Core. SchedulerSignalerImpl: Initialized Scheduler Signaller oftype: Class org. Quartz. Core. SchedulerSignalerImpl 23:40:05 2019-11-30. 92812-130 the INFO [main] org.quartz.core.QuartzScheduler : Quartz Scheduler v. 2.3.2 created. The 2019-11-30 23:40:05. 92812-131 the INFO [main] org. Quartz. Simpl. RAMJobStore: RAMJobStore initialized. 2019-11-30 23:40:05. 132 INFO 92812 - [the main] org. Quartz. Core. QuartzScheduler: Scheduler Meta-Data: Quartz Scheduler (V2.3.2)'quartzScheduler' with instanceId 'NON_CLUSTERED'
Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.
NOT STARTED.
Currently in standby mode.
Number of jobs executed: 0
Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 10 threads.
Using job-store 'org.quartz.simpl.RAMJobStore' - whichDoes not support persistence. And is not. 2019-11-30 23:40:05.132 INFO 92812 -- [main] org.quartz.impl.StdSchedulerFactory : Quartz scheduler'quartzScheduler'In the past decades, the contents of the farm have provided properties instance. 2019-11-30 23:40:05.132 INFO 92812 -- [main] org.quartz.impl.StdSchedulerFactory : Quartz scheduler version: The 2019-11-30 23:40:05 2.3.2. 132 INFO 92812 - [the main] org. Quartz. Core. QuartzScheduler: JobFactorysetto: Org. Springframework. Scheduling. Quartz. SpringBeanJobFactory @ 203 dd56b 23:40:05 2019-11-30. 92812-158 the INFO [main] o.s.s.quartz.SchedulerFactoryBean : Starting Quartz Scheduler now 2019-11-30 23:40:05. 158 INFO 92812 - [the main] org. Quartz. Core. QuartzScheduler: Scheduler quartzScheduler_$_NON_CLUSTERED started.
# DemoJob01The 23:40:05 2019-11-30. 92812-164 the INFO [eduler_Worker - 1] C.I.S pringboot. Lab28. Task. Job. DemoJob01: [executeInternal][Scheduled execution for (1), DemoService for (cn. Iocoder. Springboot. Lab28. Task. Service. DemoService @ 23 d75d74)] 23:40:09 2019-11-30. 92812-866 the INFO [eduler_Worker-2] c.i.springboot.lab28.task.job.DemoJob01 : [executeInternal][Scheduled execution for (1), DemoService for (cn. Iocoder. Springboot. Lab28. Task. Service. DemoService @ 23 d75d74)] 23:40:14 2019-11-30. 92812-865 the INFO [eduler_Worker-4] c.i.springboot.lab28.task.job.DemoJob01 : [executeInternal] [the first (1) to perform regularly, demoService for (cn. Iocoder. Springboot. Lab28. Task. Service. DemoService @ 23 d75d74)]# DemoJob02The 2019-11-30 23:40:10. 92812-004 the INFO [eduler_Worker - 3] C.I.S pringboot. Lab28. Task. Job. DemoJob02: [eduler_worker-6][eduler_worker-6][eduler_worker-6] c.i.springboot.lab28.task.job.DemoJob02 : [eduler_worker-9][eduler_worker-9] C.I.S pringboot. Lab28. Task. Job. DemoJob02: [executeInternal] [I started the execution of the]Copy the code- When the project starts, the Quartz QuartzScheduler is created and started.
- Considering the convenience of reading the log, Nai nai has separated the log of DemoJob01 and DemoJob02.
- For DemoJob01, it is executed every 5 seconds or so. And we can see,
demoServiceSuccessful injection, whilecountsIf the value is 1 each time, DemoJob01 is newly created each time. - For DemoJob02, the command is executed every 10 seconds.
The following two sections of “3.5 Application Configuration Files” provide supplementary knowledge. You are advised to read them.
3.5 Applying the Configuration File
In application.yml, add the Quartz configuration as follows:
spring:
The Quartz configuration corresponds to the QuartzProperties configuration class
quartz:
job-store-type: memory Job storage type. The default value is memory. Optionally, JDBC uses the database.
auto-startup: true # Quartz starts automatically
startup-delay: 0 Start delay N seconds
wait-for-jobs-to-complete-on-shutdown: true Whether to wait for scheduled tasks to complete when the application is closed. The default value is false. You are advised to set it to true
overwrite-existing-jobs: false # Whether to overwrite the configurations of existing jobs
properties: # add Quartz Scheduler additional properties, more can see the http://www.quartz-scheduler.org/documentation/2.4.0-SNAPSHOT/configuration.html documentation
org:
quartz:
threadPool:
threadCount: 25 # Thread pool size. The default value is 10.
threadPriority: 5 # Thread priority
class: org.quartz.simpl.SimpleThreadPool Thread pool type
# JDBC: JobStore JDBC: JobStore
Copy the code- in
spring.quartzConfiguration item, Quartz configuration, corresponding toQuartzPropertiesThe configuration class. - The Spring Boot QuartzAutoConfiguration class implements automatic configuration of Quartz and creates Quartz Scheduler beans.
Note that spring.Quartz. Wait-for-jobs-to-complete-on-shutdown is recommended for elegant Quartz shutdown. This is consistent with what we mentioned in Spring Task’s “2.6 Application Profiles”.
4. Get started with Quartz cluster again
Lab-28-task-quartz -memory
In a practical scenario, we would have to consider the high availability of scheduled tasks, so basically, we would use Quartz’s clustering solution. So in this section, we use JobStoreTX, Quartz’s JDBC storage, and MySQL as the database.
Here’s a comparison of Quartz:
The FROM blog.csdn.net/Evankaka/ar…
| type | advantages | disadvantages |
|---|---|---|
| RAMJobStore | No external database, easy to configure, fast to run | Because scheduler information is stored in memory allocated to the JVM, all scheduling information is lost when the application stops running. There is also a limit to how many jobs and triggers can be stored in the JVM memory |
| JDBC Job store | Clustering is supported because all the task information is saved in the database, things can be controlled, and the task information is not lost if the application server is shut down or restarted, and the tasks that failed due to the server shutdown or restart can be recovered | The speed of the operation depends on the speed of the database connection |
Nai nai: In fact, there are solutions that can achieve the best of both approaches, as described in “666. Eggs”.
In addition, the examples provided in this section are basically the same as the “3. Quick Start Quartz Single machine”. 😈 below, let’s start our tour ~
4.1 Importing Dependencies
In the POM.xml file, introduce related dependencies.
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.10. RELEASE</version>
<relativePath/> <! -- lookup parent from repository -->
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>lab-28-task-quartz-jdbc</artifactId>
<dependencies>
<! Automatic configuration of database connection pool
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
<dependency> <! -- In this example, we use MySQL -->
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.48</version>
</dependency>
<! -- Automatic configuration of Spring MVC -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<! -- Implement automatic configuration of Quartz -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-quartz</artifactId>
</dependency>
<! Write unit tests later -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
</project>
Copy the code- and”3.1 Introducing Dependencies”Pretty much the same, just an extra introduction
spring-boot-starter-testDependency, two unit test methods will be written later.
4.2 the sample Job
In cn. Iocoder. Springboot. Lab28. Task. Config. The job package path, create DemoJob01 and DemoJob02 class. The code is as follows:
// DemoJob01.java
@DisallowConcurrentExecution
public class DemoJob01 extends QuartzJobBean {
private Logger logger = LoggerFactory.getLogger(getClass());
@Autowired
private DemoService demoService;
@Override
protected void executeInternal(JobExecutionContext context) {
logger.info("[executeInternal][I started demoService is ({})]", demoService); }}// DemoJob02.java
@DisallowConcurrentExecution
public class DemoJob02 extends QuartzJobBean {
private Logger logger = LoggerFactory.getLogger(getClass());
@Override
protected void executeInternal(JobExecutionContext context) {
logger.info("[executeInternal][I started executing]"); }}Copy the code- Compared to 3.2 sample Job “, “on the class add Quartz @ DisallowConcurrentExecution annotations, ensure that the same JobDetail in multiple JVM process, the one and only one node in the execution.
Note that the dimension is not Quartz Job, which ensures that there is only one node executing in multiple JVM processes, but JobDetail. Although, for the most part, we’re going to make sure that there’s a one-to-one correspondence between a Job and its detail. 😈 So, fat friends who don’t understand this concept better understand this concept. It’s kind of confusing, but you just want to make sure that the detail of a Job and the detail of a Job correspond one to one.
The unique identification of the JobDetail is the JobKey, which uses the name + group attributes. Normally, we only need to set name, and Quartz defaults to group = DEFAULT.
It should be noted that in Quartz, nodes with the same Scheduler name form a Quartz cluster. In the following, we can set the name of the scheduler using the spring.Quartz. Scheduler -name configuration item.
【 Important 】 Why say this? Because we need to refine the above statement: Through Job implementation class to add @ DisallowConcurrentExecution annotations, implementation in the same Quartz Scheduler cluster, same JobDetail JobKey, guarantee in multiple JVM process, One and only one node is executing.
4.3 Applying the Configuration File
In application.yml, add the Quartz configuration as follows:
spring:
datasource:
user:
url: JDBC: mysql: / / 127.0.0.1:3306 / lab - 28 - quartz - JDBC - the user? useSSL=false&useUnicode=true&characterEncoding=UTF-8
driver-class-name: com.mysql.jdbc.Driver
username: root
password:
quartz:
url: JDBC: mysql: / / 127.0.0.1:3306 / lab - 28 - quartz - JDBC - quartz? useSSL=false&useUnicode=true&characterEncoding=UTF-8
driver-class-name: com.mysql.jdbc.Driver
username: root
password:
The Quartz configuration corresponds to the QuartzProperties configuration class
quartz:
scheduler-name: clusteredScheduler Scheduler name. The default is schedulerName
job-store-type: jdbc Job storage type. The default value is memory. Optionally, JDBC uses the database.
auto-startup: true # Quartz starts automatically
startup-delay: 0 Start delay N seconds
wait-for-jobs-to-complete-on-shutdown: true Whether to wait for scheduled tasks to complete when the application is closed. The default value is false. You are advised to set it to true
overwrite-existing-jobs: false # Whether to overwrite the configurations of existing jobs
properties: # add Quartz Scheduler additional properties, more can see the http://www.quartz-scheduler.org/documentation/2.4.0-SNAPSHOT/configuration.html documentation
org:
quartz:
# JobStore configuration
jobStore:
# data source name
dataSource: quartzDataSource # Data source used
class: org.quartz.impl.jdbcjobstore.JobStoreTX # JobStore implementation class
driverDelegateClass: org.quartz.impl.jdbcjobstore.StdJDBCDelegate
tablePrefix: QRTZ_ # Quartz table prefix
isClustered: true # is cluster mode
clusterCheckinInterval: 1000
useProperties: false
Thread pool configuration
threadPool:
threadCount: 25 # Thread pool size. The default value is 10.
threadPriority: 5 # Thread priority
class: org.quartz.simpl.SimpleThreadPool Thread pool type
jdbc: # JDBC configuration when using JobStore
initialize-schema: never Whether to automatically initialize Quartz table structures with SQL This is set to never and we create the table structure manually.
Copy the code-
There are many configuration items. We mainly compare them with “3.5 Application Configuration Files”.
-
Under the Spring. datasource configuration item, a configuration for creating multiple data sources.
userConfiguration, connectionlab-28-quartz-jdbc-userLibrary. The purpose is to simulate the business database that we use for our normal projects.quartzConfiguration, connectionlab-28-quartz-jdbc-quartzLibrary. The intention is that Quartz will use a separate database. 😈 If we have multiple projects that need to use the Quartz database,You can use one at a time, but pay attention to configurationspring.quartz.scheduler-nameDifferent Scheduler names are set to form different Quartz clusters.
-
Under the Spring. Quartz configuration TAB, there are additional configuration items that we can take a look at one by one.
scheduler-nameConfiguration, Scheduler name. We have explained this many times above, if you don’t understand, please shoot yourself to death.job-store-typeConfiguration, set up for use"jdbc"Job storage.properties.org.quartz.jobStoreConfiguration: Added JobStore configuration. The point is, passdataSourceConfiguration item that sets the use name to"quartzDataSource"The DataSource of 😈 in”4.4 DataSourceConfiguration”In, we will usespring.datasource.quartzConfiguration to create the data source.jdbcThe configuration item, despite its name, is primarily set up to initialize Quartz table structures using SQL. Here, we setinitialize-schema = never, we manually create the table structure.
Ahem, there are a lot of configuration items. Spring. Datasource = spring. Datasource = spring. Datasource = spring.
4.4 Initializing the Quartz Table Structure
Download the corresponding release packages from Quartz Download. After decompression, we can in the SRC/org/quartz/impl/jdbcjobstore/directory, see all kinds of quartz table structure of database initialization script. Here, because we use MySQL, the tables_mysql_innodb.sql script is used.
Execute the script in the database to initialize the Quartz table structure. As shown below:
For an explanation of the structure of each Quartz table, see the Quartz Framework (2) : JobStore table fields in detail. 😈 in fact, you can also not watch, hahaha.
We’ll see that each table has a SCHED_NAME field with the Quartz Scheduler name. In this way, each Quartz cluster is split at the data level.
4.5 DataSourceConfiguration
In cn. Iocoder. Springboot. Lab28. Task. The config directory, create a DataSourceConfiguration class, configure the data source. The code is as follows:
// DataSourceConfiguration.java
@Configuration
public class DataSourceConfiguration {
/** * Create a configuration object for the user data source */
@Primary
@Bean(name = "userDataSourceProperties")
@ConfigurationProperties(prefix = "spring.datasource.user") // Read the spring.datasource. User configuration into the DataSourceProperties object
public DataSourceProperties userDataSourceProperties(a) {
return new DataSourceProperties();
}
/** * Create user data source */
@Primary
@Bean(name = "userDataSource")
@ConfigurationProperties(prefix = "spring.datasource.user.hikari") // Read the spring.datasource. User configuration to the HikariDataSource object
public DataSource userDataSource(a) {
// Get the DataSourceProperties object
DataSourceProperties properties = this.userDataSourceProperties();
// Create HikariDataSource object
return createHikariDataSource(properties);
}
/** * Create a configuration object for the Quartz data source */
@Bean(name = "quartzDataSourceProperties")
@ConfigurationProperties(prefix = "spring.datasource.quartz") // Read the spring.datasource. Quartz configuration into the DataSourceProperties object
public DataSourceProperties quartzDataSourceProperties(a) {
return new DataSourceProperties();
}
/** * Create quartz data source */
@Bean(name = "quartzDataSource")
@ConfigurationProperties(prefix = "spring.datasource.quartz.hikari")
@QuartzDataSource
public DataSource quartzDataSource(a) {
// Get the DataSourceProperties object
DataSourceProperties properties = this.quartzDataSourceProperties();
// Create HikariDataSource object
return createHikariDataSource(properties);
}
private static HikariDataSource createHikariDataSource(DataSourceProperties properties) {
// Create HikariDataSource object
HikariDataSource dataSource = properties.initializeDataSourceBuilder().type(HikariDataSource.class).build();
// Set the thread pool name
if (StringUtils.hasText(properties.getName())) {
dataSource.setPoolName(properties.getName());
}
returndataSource; }}Copy the code- Based on the
spring.datasource.userThe configuration item is created with the name"userDataSource"The DataSource Bean. And on top of that we added@PrimayAnnotation, indicating that it isThe mainThe data source. - Based on the
spring.datasource.quartzThe configuration item is created with the name"quartzDataSource"The DataSource Bean. And on top of that we added@QuartzDataSourceAnnotation, indicating that it isQuartzData source of. 😈 attention, must configure ah, here Nai Nai card for a long time ！！！！
4.6 Configuring scheduled Tasks
With that done, we need to configure timed tasks for Quartz. Currently, there are two ways:
- Method 1:4.6.1 Automatic Bean Settings.
- Method 2:4.6.2 Scheduler Manual Settings.
4.6.1 Automatic Bean Settings
In cn. Iocoder. Springboot. Lab28. Task. The config directory, create a ScheduleConfiguration classes, configuration of the above two sample Job. The code is as follows:
// ScheduleConfiguration.java
@Configuration
public class ScheduleConfiguration {
public static class DemoJob01Configuration {
@Bean
public JobDetail demoJob01(a) {
return JobBuilder.newJob(DemoJob01.class)
.withIdentity("demoJob01") // Name is demoJob01
.storeDurably() // Whether the task is retained when there is no Trigger association. Since the JobDetail was created without Trigger pointing to it, it needs to be set to true to indicate retention.
.build();
}
@Bean
public Trigger demoJob01Trigger(a) {
// A simple scheduler
SimpleScheduleBuilder scheduleBuilder = SimpleScheduleBuilder.simpleSchedule()
.withIntervalInSeconds(5) / / frequency.
.repeatForever(); / / number.
// Trigger constructor
return TriggerBuilder.newTrigger()
.forJob(demoJob01()) // Corresponding Job is demoJob01
.withIdentity("demoJob01Trigger") // The name is demoJob01Trigger
.withSchedule(scheduleBuilder) // The corresponding Schedule is scheduleBuilder.build(); }}public static class DemoJob02Configuration {
@Bean
public JobDetail demoJob02(a) {
return JobBuilder.newJob(DemoJob02.class)
.withIdentity("demoJob02") // Name is demoJob02
.storeDurably() // Whether the task is retained when there is no Trigger association. Since the JobDetail was created without Trigger pointing to it, it needs to be set to true to indicate retention.
.build();
}
@Bean
public Trigger demoJob02Trigger(a) {
// A simple scheduler
CronScheduleBuilder scheduleBuilder = CronScheduleBuilder.cronSchedule("0/10 * * * *? *");
// Trigger constructor
return TriggerBuilder.newTrigger()
.forJob(demoJob02()) // Corresponding Job is demoJob02
.withIdentity("demoJob02Trigger") // Name is demoJob02Trigger
.withSchedule(scheduleBuilder) // The corresponding Schedule is scheduleBuilder.build(); }}}Copy the code- This is the same as “3.3 ScheduleConfiguration”.
When the Quartz scheduler starts, it automatically calls the following methods based on this configuration:
Scheduler#addJob(JobDetail jobDetail, boolean replace)Method to persist JobDetail to the database.Scheduler#scheduleJob(Trigger trigger)Method to persist Trigger to the database.
4.6.2 Scheduler Manually sets the Scheduler
In general, It is recommended to use Scheduler for manual setting.
Create the QuartzSchedulerTest class to create a Quartz timed task configuration with DemoJob01 and DemoJob02 added respectively. The code is as follows:
// QuartzSchedulerTest.java
@RunWith(SpringRunner.class)
@SpringBootTest(classes = Application.class)
public class QuartzSchedulerTest {
@Autowired
private Scheduler scheduler;
@Test
public void addDemoJob01Config(a) throws SchedulerException {
/ / create a JobDetail
JobDetail jobDetail = JobBuilder.newJob(DemoJob01.class)
.withIdentity("demoJob01") // Name is demoJob01
.storeDurably() // Whether the task is retained when there is no Trigger association. Since the JobDetail was created without Trigger pointing to it, it needs to be set to true to indicate retention.
.build();
/ / create the Trigger
SimpleScheduleBuilder scheduleBuilder = SimpleScheduleBuilder.simpleSchedule()
.withIntervalInSeconds(5) / / frequency.
.repeatForever(); / / number.
Trigger trigger = TriggerBuilder.newTrigger()
.forJob(jobDetail) // Corresponding Job is demoJob01
.withIdentity("demoJob01Trigger") // The name is demoJob01Trigger
.withSchedule(scheduleBuilder) // The corresponding Schedule is scheduleBuilder
.build();
// Add a scheduling task
scheduler.scheduleJob(jobDetail, trigger);
// scheduler.scheduleJob(jobDetail, Sets.newSet(trigger), true);
}
@Test
public void addDemoJob02Config(a) throws SchedulerException {
/ / create a JobDetail
JobDetail jobDetail = JobBuilder.newJob(DemoJob02.class)
.withIdentity("demoJob02") // Name is demoJob02
.storeDurably() // Whether the task is retained when there is no Trigger association. Since the JobDetail was created without Trigger pointing to it, it needs to be set to true to indicate retention.
.build();
/ / create the Trigger
CronScheduleBuilder scheduleBuilder = CronScheduleBuilder.cronSchedule("0/10 * * * *? *");
Trigger trigger = TriggerBuilder.newTrigger()
.forJob(jobDetail) // Corresponding Job is demoJob01
.withIdentity("demoJob02Trigger") // The name is demoJob01Trigger
.withSchedule(scheduleBuilder) // The corresponding Schedule is scheduleBuilder
.build();
// Add a scheduling task
scheduler.scheduleJob(jobDetail, trigger);
// scheduler.scheduleJob(jobDetail, Sets.newSet(trigger), true);}}Copy the code- The code for creating JobDetail and Trigger is the same as in 4.6.1 Bean Automatic Settings.
- At the end of each unit test method, call
Scheduler#scheduleJob(JobDetail jobDetail, Trigger trigger)Method to persist JobDetail and Trigger to the database. - Call if you want to override the configuration of a Quartz timed task in a database
Scheduler#scheduleJob(JobDetail jobDetail, Set<? extends Trigger> triggersForJob, boolean replace)Method, passed inreplace = trueOverride configuration.
4.7 Application
Create the Application. Java class and configure the @SpringBootApplication annotation. The code is as follows:
// Application.java
@SpringBootApplication
public class Application {
public static void main(String[] args) { SpringApplication.run(Application.class, args); }}Copy the code- Run the Application class to start the sample project. The specific execution log is basically consistent with “3.4 Application”, which will not be repeated here in Nai.
If you want to test your cluster, you can create the Application02.java class and configure the @SpringBootApplication annotation. The code is as follows:
// Application02.java
@SpringBootApplication
public class Application02 {
public static void main(String[] args) {
// Set a random Tomcat port
System.setProperty("server.port"."0");
// Start the Spring Boot applicationSpringApplication.run(Application.class, args); }}Copy the code- Run the Application02 class to start a sample project again. Then, looking at the output log, you can see that the two sample projects that were started have execution logs for DemoJob01 and DemoJob02.
5. Quick start xxL-job
Example code for repository: lab-28-task-xxl-job.
Although Quartz has been able to meet our demands for timed tasks, there is still a long way to go before it can be used in production. When Nai began his internship at the earliest, Quartz only provided the function of task scheduling, but did not provide the management and monitoring console for task management, so he had to do the secondary packaging by himself. At that time, because the community could not find a suitable open source project to implement this function, so we carried out a simple package to meet our management and monitoring needs.
Now, however, there are plenty of good scheduling middleware in the open source community. Among them, the representative one is XXL-job. Its definition of itself is as follows:
Xxl-job is a lightweight distributed task scheduling platform, whose core design goal is rapid development, simple learning, lightweight and easy to expand.
For the introduction of XXL-job, Nai has been in the “impression of xxL-job minimalist introduction” written, fat friends jump to the article to read. The key is to set up an XXL-job scheduling center first. 😈 Because, this article is to implement a xxL-job executor in the Spring Boot project.
5.1 Importing Dependencies
In the POM.xml file, introduce related dependencies.
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.2.1. RELEASE</version>
<relativePath/> <! -- lookup parent from repository -->
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>lab-28-task-xxl-job</artifactId>
<dependencies>
<! -- Automatic configuration of Spring MVC -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<! -- xxl-job dependencies -->
<dependency>
<groupId>com.xuxueli</groupId>
<artifactId>xxl-job-core</artifactId>
<version>2.1.1</version>
</dependency>
</dependencies>
</project>
Copy the codeSpecific role of each dependency, fat friends carefully look at the nai nai added all notes oh. Unfortunately, xxL-Job does not officially provide the Spring Boot Starter package, which is slightly embarrassing. However, the community is already submitting Pull requests, which can be found at github.com/xuxueli/xxl… .
5.2 Applying a Configuration File
In application.yml, add the Quartz configuration as follows:
server:
port: 9090 # Specify a port to avoid conflicts with the port used by the xxL-job scheduling center. For testing purposes only
# xxl-job
xxl:
job:
admin:
addresses: http://127.0.0.1:8080/xxl-job-admin [Optional] : If multiple IP addresses exist in the dispatch center cluster deployment, separate them using commas (,). The executor will use this address for "executor heartbeat registration" and "task result callback". If it is null, auto registration is disabled.
executor:
appname: lab-28-executor # executor AppName [optional] : executor heartbeat registration group based; If empty, auto registration is disabled
ip: [Optional] : The default value is blank to indicate that the IP address is automatically obtained. If multiple network cards are used, you can manually set the specified IP address. The IP address will not be bound to Host and is only used for communication. Address information is used for "actuator registration" and "dispatch center requests and triggers tasks";
port: 6666 # ### Actuator port number [Optional] : The value is automatically obtained if the value is less than or equal to 0. The default port number is 9999. If multiple actuators are deployed on a single machine, configure different actuators.
logpath: /Users/yunai/logs/xxl-job/lab-28-executor [Optional] : You need to have read and write permission on the disk where the run log files are stored. If it is empty, the default path is used.
logretentiondays: 30 [Optional] : Expiration logs are automatically cleared. The expiration logs take effect when the limit value is greater than or equal to 3. Otherwise, for example, -1, disable the automatic clearing function.
accessToken: yudaoyuanma # executor communication TOKEN [optional] : non-space enabled;
Copy the code- Specific role of each parameter, fat friends see their own detailed notes ha.
5.3 XxlJobConfiguration
In cn. Iocoder. Springboot. Lab28. Task. The config directory, create a DataSourceConfiguration classes, configuration XXL – the JOB executor. The code is as follows:
// XxlJobConfiguration.java
@Configuration
public class XxlJobConfiguration {
@Value("${xxl.job.admin.addresses}")
private String adminAddresses;
@Value("${xxl.job.executor.appname}")
private String appName;
@Value("${xxl.job.executor.ip}")
private String ip;
@Value("${xxl.job.executor.port}")
private int port;
@Value("${xxl.job.accessToken}")
private String accessToken;
@Value("${xxl.job.executor.logpath}")
private String logPath;
@Value("${xxl.job.executor.logretentiondays}")
private int logRetentionDays;
@Bean
public XxlJobSpringExecutor xxlJobExecutor(a) {
// Create XxlJobSpringExecutor
XxlJobSpringExecutor xxlJobSpringExecutor = new XxlJobSpringExecutor();
xxlJobSpringExecutor.setAdminAddresses(adminAddresses);
xxlJobSpringExecutor.setAppName(appName);
xxlJobSpringExecutor.setIp(ip);
xxlJobSpringExecutor.setPort(port);
xxlJobSpringExecutor.setAccessToken(accessToken);
xxlJobSpringExecutor.setLogPath(logPath);
xxlJobSpringExecutor.setLogRetentionDays(logRetentionDays);
/ / return
returnxxlJobSpringExecutor; }}Copy the code- in
#xxlJobExecutor()Method to create xxL-Job executor Bean objects in the Spring container. Note that the method is added to@BeanNote that the start and destroy methods are configured.
5.4 DemoJob
In cn. Iocoder. Springboot. Lab28. Task. The job package path, create DemoJob class, sample task class regularly. The code is as follows:
// DemoJob.java
@Component
@JobHandler("demoJob")
public class DemoJob extends IJobHandler {
private Logger logger = LoggerFactory.getLogger(getClass());
private final AtomicInteger counts = new AtomicInteger();
@Override
public ReturnT<String> execute(String param) throws Exception {
// Prints logs
logger.info("[execute][execute for the ({}) time]", counts.incrementAndGet());
// The execution succeeds
returnReturnT.SUCCESS; }}Copy the code- Inherit the XXL – JOBIJobHandlerAbstract class, by implementation
#execute(String param)Method, so as to realize the logic of the scheduled task. - On the method, add the @jobHandler annotation and set the name of JobHandler. Later, we need to use this name when we add tasks in the console of the dispatch center.
The #execute(String param) method returns the result of type ReturnT. If the value is returnt. code == returnt. SUCCESS_CODE, the task is successfully executed. If the value is not returnt. MSG, the task fails to be executed. Thus, task execution results can be easily controlled in task logic.
#execute(String param) specifies the “task parameter” configured when a new task is created in the console of the scheduling center. In most cases, it will not be used.
5.5 Application
Create the Application. Java class and configure the @SpringBootApplication annotation. The code is as follows:
// Application.java
@SpringBootApplication
public class Application {
public static void main(String[] args) { SpringApplication.run(Application.class, args); }}Copy the codeRun the Application class to start the sample project. The output logs are condensed as follows:
# xxl-job Startup log2019-11-29 00:58:42.429 INFO 46957 --- [ main] c.xxl.job.core.executor.XxlJobExecutor : >>>>>>>>>>> xxl-job register jobhandler success, name:demoJob, jobHandler:cn.iocoder.springboot.lab28.task.job.DemoJob@3af9aa66
2019-11-29 00:58:42.451 INFO 46957 --- [ main] c.x.r.r.provider.XxlRpcProviderFactory : >>>>>>>>>>> xxl-rpc, provider factory add service success. serviceKey = com.xxl.job.core.biz.ExecutorBiz, serviceBean = class com.xxl.job.core.biz.impl.ExecutorBizImplThe 2019-11-29 00:58:42. 454INFO46957 -main] c.x.r.r.provider.XxlRpcProviderFactory: > > > > > > > > > > >xxl-rpc.provider factory add service success. serviceKey = com.xxl.job.core.biz.ExecutorBiz, serviceBean = class com.xxl.job.core.biz.impl.ExecutorBizImplThe 2019-11-29 00:58:42. 565INFO46957 -main] o.s.s.concurrent.ThreadPoolTaskExecutor : Initializing ExecutorService 'applicationTaskExecutor'the 2019-11-29 00:58:42. 629INFO46957 -Thread7]com.xxl.rpc.remoting.net.Server: > > > > > > > > > > >xxl-rpc remoting server start success.nettype = com.xxl.rpc.remoting.net.impl.netty_http.server.NettyHttpServer, port = 6666
Copy the codeAt this point, DemoJob will not be executed because we have not configured it in the XXl-job dispatch center. Next, let’s perform the corresponding configuration in the XXL-Job scheduling center.
5.6 Adding actuators
Browser openhttp://127.0.0.1:8080/xxl-job-admin/jobgroupAddress, the “Executive Management” menu. The diagram below:
Click the “Add actuator” button to pop up the “Add Actuator” interface. The diagram below:
Fill in the"lab-28-executor"Actuator information, click “save” button to save. Wait patiently for a while and the actuator will automatically register. The diagram below:
- The list of OnLine actuators is displayed in the actuator list. You can view the cluster machines corresponding to the actuators by using OnLine Machines.
The same actuator can be configured only once.
5.7 Creating a Task
Browser openhttp://127.0.0.1:8080/xxl-job-admin/jobinfoAddress, the Task Management menu. The diagram below:
Click the “Add” button on the far right to pop up the “Add” interface. The diagram below:
Fill in the"demoJob"Click the “Save” button to save the task information. The diagram below:
Click on the"demoJob"Task “Operation” button, select “Start”, confirm, the"demoJob"The state of the mission becomesRUNNING. The diagram below:
At this point, we open the IDE interface of the executor and see that DemoJob has been executed every minute. The log is as follows:
2019-11-29 01:30:00.161 INFO 48374 --- [ Thread-18] C.I.S pringboot. Lab28. Task. Job. DemoJob: [the execute] [timing (1) times]2019-11-29 01:31:00.012 INFO 48374 --- [ Thread-18] C.I.S pringboot. Lab28. Task. Job. DemoJob: [the execute] [timing (2) times]2019-11-29 01:32:00.009 INFO 48374 --- [ Thread-18] C.I.S pringboot. Lab28. Task. Job. DemoJob: [the execute] [timing (3) times]2019-11-29 01:33:00.010 INFO 48374 --- [ Thread-18] C.I.S pringboot. Lab28. Task. Job. DemoJob: [the execute] [timing (4) times]2019-11-29 01:34:00.005 INFO 48374 --- [ Thread-18] C.I.S pringboot. Lab28. Task. Job. DemoJob: [the execute] [timing (5) times]Copy the codeAnd we click on the dispatch center screen"demoJob"Click The Operation button of a task and click Query Log. The corresponding scheduling logs are displayed. The diagram below:
At this point, we have completed the introduction of the XXL-Job actuator.
6. Quickstart Elastic-job
There are probably a lot of fat people who don’t know about elastice-Job middleware. Let’s take a look at the introduction of its official document:
Elastic-job is a distributed scheduling solution consisting of two independent subprojects, elastic-Job-Lite and Elastic-Job-Cloud.
Elastik-job-lite is positioned as a lightweight decentralized solution that uses JAR packages to coordinate distributed tasks.
Elastics-job is basically one of the best open source task scheduling middleware in the country. Currently in a bit of a “broken” state, specific can be seen github.com/elasticjob/… .
Therefore, Nai nai will not provide this example for the time being. If you’re interested in the elastic-Job source code, check out the following two series written by Nai Nai:
- Impression Channel Elastice-Job-Lite Source Code Analysis Series
- Impression Channel Elastice-Job-Cloud Source Analysis Series
666. The eggs
① How to choose?
Maybe you want to know the comparison of different scheduling middleware. The table is as follows:
| features | quartz | elastic-job-lite | xxl-job | LTS |
|---|---|---|---|---|
| Rely on | MySQL, the JDK | The JDK, zookeeper | Mysql, the JDK | JDK, ZooKeeper, and Maven |
| High availability | Multi-node deployment, by competing for database locks to ensure that only one node performs the task | You can dynamically add servers by registering and discovering ZooKeeper | Based on competing database locks, only one node can perform tasks, and horizontal capacity expansion is supported. You can manually add scheduled tasks, start or suspend tasks, and monitor tasks | In cluster deployment, servers can be added dynamically. You can manually add scheduled tasks, start or suspend tasks. A monitor |
| Task fragmentation | x | Square root | Square root | Square root |
| Management interface | x | Square root | Square root | Square root |
| How easy is it | simple | simple | simple | A little complicated |
| Advanced features | – | Flexible capacity expansion, multiple job modes, failover, state collection, multi-threaded data processing, idempotence, fault tolerance, Spring namespace support | Elastic capacity expansion, sharding broadcast, failover, Rolling real-time log, GLUE (support online code editing, no publishing), task progress monitoring, task dependence, data encryption, email alarm, running reports, internationalization | Support for Spring, Spring Boot, service logger, SPI extension support, failover, node monitoring, diversified task execution results support, FailStore fault tolerance, dynamic capacity expansion. |
| Version update | No update for half a year | No update for 2 years | Recently updated | No update for a year |
The following articles are also recommended:
- Technical Selection of Distributed Timing Task Scheduling System
- “Azkaban, XXL-Job & Airflow Comparative Analysis”
If you really don’t know what to do, try xxL-job first.
② Centralization v. decentralization
Below, let’s briefly talk about the classification of distributed scheduling middleware implementation. In a distributed scheduling middleware, there are two roles:
- Scheduler: responsible for scheduling tasks and sending them to the executor.
- Actuator: responsible for receiving tasks and executing specific tasks.
Then, from the perspective of scheduling system, it can be divided into two categories:
- Centralization: The scheduling center is separated from the executor, and the scheduling center uniformly dispatches tasks and notifies an executor to process tasks.
- Decentralization: The scheduling center is integrated with the executor to schedule and perform processing tasks by itself.
In this case, XXL-job is a centralized task scheduling platform. Others currently using this scheme are:
- HOME LINK kob
- Meituan Crane (not open source yet)
Decentralized task scheduling platforms currently include:
- Elastic Job
- Saturn of Vipshop
- Quartz database based clustering solution
- TBSchedule of Taobao (update suspended, only Aliyun SchedulerX service can be used)
Nai: If fat friends want to understand more, you can have a look at the “Centralized V.S Decentralized Scheduling Design” written by Nai Friends.
③ task competition V.S task pre-allocation
Then, from the perspective of task allocation, it can be divided into two categories:
- Task competition: The scheduler sends tasks to the executor through competition tasks.
- Task preassignment: The scheduler preassigns tasks to different actuators without competing.
Therefore, XXL-job belongs to the task scheduling platform of task competition. Others currently using this scheme are:
- HOME LINK kob
- Meituan Crane (not open source yet)
- Quartz database based clustering solution
The task scheduling platform for task pre-allocation currently includes:
- Elastic Job
- Saturn of Vipshop
- TBSchedule of Taobao (update suspended, only Aliyun SchedulerX service can be used)
Generally speaking, task scheduling platforms based on task pre-allocation will use Zookeeper to coordinate task allocation to different nodes. At the same time, the task scheduling platform must be a decentralized scheme, and each node is both a scheduler and an executor. In this way, after the task is preassigned to each node, the subsequent task is scheduled for its own execution.
In contrast, with the increasing number of nodes, the performance of the scheme based on task competition will decline due to task competition. This problem does not exist in the task preassignment scheme. Moreover, the scheme based on task pre-allocation has better performance than the scheme based on task competition.
Here’s an article by Zhang Liang, a developer at Dangdang’s Distributed Job framework, called Elastic Job.
(4) Quartz is an excellent scheduling kernel
For the most part, we don’t directly use Quartz as our scheduling middleware of choice. However, almost all distributed scheduling middleware use Quartz as the scheduling kernel, because Quartz provides strong capabilities in simple task scheduling itself.
However, with the development of a distributed scheduling middleware, we will gradually consider abandoning Quartz as a scheduling kernel and developing our own. For example, xxl-job has been replaced by a self-developed scheduling module in 2.1.0 RELEASE. The reasons for its replacement are as follows:
Xxl-job finally selects its own scheduling component (the early scheduling component is based on Quartz);
- On the one hand, to simplify the system and reduce redundant dependencies.
- On the other hand, it is to provide system controllability and stability.
There is also a plan under development for Elastice-Job 3.X to develop its own scheduling component, replacing Quartz.