preface

In the process of project development, many scenarios of background tasks are indispensable, such as: large amount of data processing or analysis, timed synchronization of data, large amount of data asynchronous export, message compensation and so on. NET, Quartz NET, Hangfire are the first choice of many partners, but if you want to create a general, flexible configuration scheduling platform, you need to spend a lot of time encapsulation; Of course, there are some friends who made wheels, and it is also good to use; However, I would like to introduce xxL-job distributed task scheduling platform to my friends. It is mentioned in a casual chat with my friends. It is very cool to use in Java, so I want to see if it can be used in Java. NET project.

The body of the

1. The XXL – the JOB description

1.1 an overview of the

Xxl-job is a distributed task scheduling platform, whose core design goal is rapid development, simple learning, lightweight and easy to expand. Now open source and access to many companies online product lines, out of the box;

The project was developed in Java, so at first it was doubtful that it would fail to integrate. NET project, the first reaction was to look for ready-made wheels in the Nuget package. As expected, some friends were already using them, so I definitely had to arrange them.

The documentation for this project is extremely detailed, I won’t waste your time here, you can read the documentation on Gitee or Github directly, and this project is very popular on both the cloud and Github. The address is as follows:

  • Gitee:gitee.com/xuxueli0323…

  • Github:github.com/xuxueli/xxl…

1.2 Classic Functions

Before you go to the demo, let’s talk about a few more attractive features, as follows (of course, not limited to this) :

  • Simple and beautiful Web interface: easy to operate, easy to manage tasks;

  • Registry: The actuator automatically registers tasks periodically. The scheduling center automatically discovers the registered tasks and triggers the execution of the tasks. At the same time, it also supports manual input actuator address;

  • Cluster and distributed are very powerful: the scheduling center and task executor can be clustered to achieve high availability;

  • Health check: ensure scheduling high availability;

  • Consistency: The Scheduling center uses DB locks to ensure the consistency of distributed scheduling in the cluster. A task scheduling task triggers only one execution.

  • Fragment broadcast task: When task executors are deployed in a cluster, for example, when a large amount of data needs to be processed, data can be split among different task executors based on corresponding parameters to improve efficiency.

  • Email alarm: support email alarm when the task fails, support configuring multiple email addresses to send alarm emails;

  • User and permission management: Supports online management of system users, including administrators and common users

If you are not sure about the scheduling center and actuator, you can take a look at the architecture diagram of XXL-job:

We just write the JobHandler in the executor. Other frameworks take care of that.

2 Environment Installation

2.1 Install and initialize the database

Requirement is MySQL5.7 +, here is the MySQL5.7, install the database tutorial not capture one by one, novice tutorial is very detailed (www.runoob.com/docker/dock)… .

Note that after installing MySql data, you must configure to allow remote connections.

MySQL > create table (xxL-job); insert default user (xxl-job); insert default user (xxl-job);

Gitee.com/xuxueli0323…

2.2 Docker Deploys xxL-job scheduling center

The deployment of XXL-job can be used to download the source code for deployment, configuration is more flexible; Docker can also be used for deployment, which is more convenient. Here is a demonstration of the way of Docker, execute the following command (premise is Docker environment, if not familiar with Docker, I previously shared the relevant article “Docker series”) :

Docker run - e PARAMS = "-- spring. The datasource. Url = JDBC: mysql: / / 172.29.211.138:3306 / xxl_job? useUnicode=true&characterEncoding=UTF-8&autoReconnect=true&serverTimezone=Asia/Shanghai --spring.datasource.username=root --spring.datasource.password=123456" -p 5000:8080 -v /tmp:/data/applogs --name XXL - job - admin - d xuxueli/XXL - job - admin: 2.3.0Copy the code
  • -e PARAMS: specifies parameters, such as database connection, character set, and time zone.
  • -p 55:8080: indicates port mapping
  • -v Mount the data volume
  • –name: Sets the container name
  • -d: The container runs in background mode

Once the container is started, you can access the interface (in the case of a cloud server, remember to open ports for both the security instance and the firewall, as discussed in the previous article).

Enter the default user name and password: admin/123456

Enter the main interface, indicating that the dispatching center has been set up:

After the dispatch center is done, the next step is to write business logic.

3..NET project integration

3.1 Installing the Nuget package

DotXxlJob.Core is an executable that can be integrated into xxL-job.

Github.com/NanoFabricF… .

Once the package is installed, prepare the required middleware and configuration information, which is well documented here. By the way, this is still an API project.

3.2 Prepare and register a middleware as required
public class XxlJobExecutorMiddleware { private readonly IServiceProvider _provider; private readonly RequestDelegate _next; private readonly XxlRestfulServiceHandler _rpcService; public XxlJobExecutorMiddleware(IServiceProvider provider, RequestDelegate next) { this._provider = provider; this._next = next; this._rpcService = _provider.GetRequiredService<XxlRestfulServiceHandler>(); } public async Task Invoke(HttpContext context) { string contentType = context.Request.ContentType; / / handle Post types of Request if (" Post ". The Equals (context. Request. Method, StringComparison. OrdinalIgnoreCase) &&! string.IsNullOrEmpty(contentType) && contentType.ToLower().StartsWith("application/json")) { await _rpcService.HandlerAsync(context.Request, context.Response); return; } await _next.Invoke(context); }}Copy the code

To facilitate middleware registration, write an extension method as follows:

public static class ApplicationBuilderExtensions { public static IApplicationBuilder UseXxlJobExecutor(this IApplicationBuilder @this) { return @this.UseMiddleware<XxlJobExecutorMiddleware>(); }}Copy the code

Register services and middleware in startup. cs file as follows:

Add configuration information to the configuration file appsettings.json as follows:

{ "Logging": { "LogLevel": { "Default": "Information", "Microsoft": "Warning", "Microsoft.Hosting.Lifetime": "Information" } }, "xxlJob": { "adminAddresses": "http://112.xxx.xxx.127:5000/xxl-job-admin", "appName": "Xxl-job-executor-dotnet ", "specialBindAddress": "172.29.211.138", "port": 8888, "autoRegistry": true, "accessToken": "", "logRetentionDays": 20 }, "AllowedHosts": "*" }Copy the code
  • AdminAddresses: addresses of the dispatch center. Multiple addresses can be separated by commas.
  • AppName: indicates the name of the actuator, which will be displayed in the scheduling center. If the actuator is manually entered in the scheduling center, the actuator name must be the same.
  • SpecialBindAddress: indicates the IP address submitted to the dispatch center during automatic registration. If this parameter is left blank, the system automatically obtains an internal IP address. It can be understood as the IP address of the service site deployment.
  • Port: indicates the port submitted to the scheduling center during automatic registration. It can be regarded as the port deployed at the service site.
  • AutoRegistry: Whether to automatically register
  • AccessToken: Fill in accessToken authentication if required, leave out accessToken authentication if not required
  • LogPath: indicates the path where logs are saved. If not written, logs are saved to the current path where the program is running
  • LogRetentionDays: indicates the retention days of logs

Now you can start writing business code.

3.3 Writing business code

Note: Be sure to identify the above feature and specify the name, which will be used by the dispatch center for new tasks.

Once written, register it as a service:

Publish the program to the cloud server. Since the local COMPUTER cannot be accessed, ensure that the scheduling center and service site can communicate with each other normally.

Since the runtime environment is not installed on the newly purchased Linux environment, the standalone mode is used for deployment (.net’s two deployment modes are shared in this article) as follows:

3.4 Configuring task scheduling in the Scheduling Center

First go to the executor management for executor maintenance, if it is automatic registration, there will be more related records, you can also manually input, but to ensure that the AppName and business program specified the same.

Details are as follows:

Once you have an executor, you can add a task scheduler:

3.5 the effect of

This can be controlled in the task management interface, as follows:

Then click Execute to test the newly added task in the task management module of the scheduling center. If there is no problem, click Start to see the running effect as follows:

Scheduling normal, about the Cron expression is not introduced in this way, it is left to friends to try their own.

This is how to use the functions of XXL-job. You can use different policies for different services after you experience other policies.

Demo code address: gitee.com/CodeZoe/dot…

conclusion

This distributed scheduling platform is not very powerful, here is just a demonstration of how to integrate use, in fact, there are a lot of functions, basically simple configuration can be done; And his document is really very detailed, so do not understand basically check the document can figure out.

With this platform, there is no need to encapsulate a scheduling platform and just focus on business code development. And the platform is cross-language, offering Java, Python, PHP… About a dozen task modes, and according to his documentation, a lot of big companies use them.

Follow the Code Variety Circle and learn with me.