tags: springbatch

1. The introduction

After the introduction of Spring Batch in previous articles and combined with examples, from the simplest helloWorld string output, to data synchronization from reading files to database, and then database to database, and then combined with BeetlSql to further simplify database reading and writing, and then realized incremental synchronization through dynamic parameter binding. From shallow to deep, it can basically meet the work of data extraction and data synchronization. The following is the list of previous articles:

  • Data Batch artifact -Spring Batch(1) Introduction and usage scenarios
  • Quick introduction to component – Spring Batch (2) helloWorld
  • Quick use of the component – Spring Batch (3) reads file data to the database
  • Decisive Database – Spring Batch (4) Database to database
  • Convenient data read and write – Spring Batch (5) combines beetlSql for data read and write
  • Incremental Synchronization – Spring Batch (6) Dynamic parameter binding with incremental synchronization

Batch tasks usually run regularly as background services, while service operation and maintenance is an important work besides batch services, including how to monitor service operation, task operation statistics, task execution log viewing and so on. As mentioned earlier, Spring Batch is a Batch framework, not a scheduling framework. As for scheduling framework, quartz and crontab can be used in a simple way. However, there are multi-functional and complete scheduling frameworks on the market. After comparison, I think XXL-Job is relatively easy to use and has complete functions. Use xxl-job to schedule Batch tasks of Spring Batch to implement the monitoring function.

2. XXL – job is introduced

Xxl-job is a lightweight distributed task scheduling platform. Its core design goal is rapid development, simple learning, lightweight and easy to expand. The system architecture is designed reasonably, and the scheduling system is separated from the executor. The scheduling system is responsible for scheduling related logic, and the executor is the implementation logic of specific tasks. Developers can directly use the scheduling system, and then realize their own task logic as the executor, out of the box. Specific usage and detailed instructions can be found in its official documents

Install the xxl-job database and start the xxl-job database as follows:

3. Write the Spring Batch executor

This example is based on the database incremental synchronization used in the previous article. All you need to do is add an executor, which in this example is spring-Batch-xxl-executor. The official xxL-job documentation describes how to configure and deploy the Executor project. In this example, the configuration is as follows:

3.1 Adding maven dependencies

Add xxl-job core dependencies as follows:

<! -- xxl-job-core --> <dependency> <groupId>com.xuxueli</groupId> <artifactId>xxl-job-core</artifactId> The < version > 2.0.2 < / version > < / dependency >Copy the code

3.2 Adding an Actuator Configuration fileexecutor.properties

Configure the actuator, including the actuator name and scheduling system address, as follows:

### xxl-job admin address list, such as "http://address" or "http://address01,http://address02"XXL. Job. Admin. Addresses = http://127.0.0.1:8089/xxl-job-admin### xxl-job executor address
xxl.job.executor.appname=${spring.application.name}
xxl.job.executor.ip=
xxl.job.executor.port=9999

### xxl-job, access token
xxl.job.accessToken=

### xxl-job log path
xxl.job.executor.logpath=logs
### xxl-job log retention days
xxl.job.executor.logretentiondays=-1
Copy the code

3.3 Actuator Component Configuration

Add the Java configuration file to get the executor.properties configuration above. As follows:

@Configuration
@ConfigurationProperties(prefix = "xxl.job")
@PropertySource("classpath:/config/executor.properties")
@Slf4j
public class JobExecutorConfig {
    @Value("${xxl.job.admin.addresses}") private String adminAddresses; . Slightly @ Bean (initMethod ="start", destroyMethod = "destroy")
    public XxlJobSpringExecutor xxlJobExecutor() {
        log.info(">>>>>>>>>>> xxl-job config init.");
        XxlJobSpringExecutor xxlJobSpringExecutor = new XxlJobSpringExecutor();
        xxlJobSpringExecutor.setAdminAddresses(adminAddresses);
        xxlJobSpringExecutor.setAppName(appName);
        xxlJobSpringExecutor.setIp(ip);
        xxlJobSpringExecutor.setPort(port);
        xxlJobSpringExecutor.setAccessToken(accessToken);
        xxlJobSpringExecutor.setLogPath(logPath);
        xxlJobSpringExecutor.setLogRetentionDays(logRetentionDays);

        returnxxlJobSpringExecutor; }}Copy the code

3.4 Writing actuators

Executor, in fact, inherited the IJobHandler class implementation, execute method to achieve task implementation logic. In this example, you just need to implement the logic that started the task you tested earlier. As follows:

@JobHandler(value="incrementUserJobHandler")
@Component
public class JobIncrementUserHandler extends IJobHandler {
    @Autowired
    private JobLauncherService jobLauncherService;

    @Autowired
    private IncrementService incrementService;

    @Autowired
    private Job incrementJob;
    @Override
    public ReturnT<String> execute(String s) throws Exception {
        return JobUtil.runJob4Executor("incrementUser",incrementService,jobLauncherService,incrementJob); }}Copy the code

4. Use xxl-job to schedule Spring Batch tasks

After xxl-job-admin and spring-Batch-xxl-executor are deployed correctly, tasks can be scheduled on the XXl-job-admin page. For details, see xxL-job documentation. Add spring-batch-xxl-executor and add incrementUserJobHandler to specify the cron expression to execute. The following output is displayed after the task is executed:

5. To summarize

Based on the example of database incremental synchronization, this paper uses Spring Batch combined with XXL-Job to realize monitoring task scheduling, which makes Batch task maintainability stronger. We hope that those who need to use Spring Batch for Batch task development can help.