Writing a reusable distributed scheduling task management WebUI component based on Quartz

premise

Small business teams give priority to cost savings regardless of the options they choose.With regard to the Distributed Timing Scheduling Framework, mature candidates are XXL-JOB, Easy Scheduler, Light Task Scheduler, Elastic Job, etc., which have been used in production environments before.However, to build a highly available distributed scheduling platform, these frameworks (whether or not they are de-centralized) require additional server resources to deploy central dispatch management service instances, and sometimes even rely on middleware such as Zookeeper.Looking back at Quartz's source code and analyzing its threading model, I thought it could be based on MySQL through a less recommended X-lock scheme (SELECT FOR)UPDATE Locking) implements that a single trigger in a service cluster can be executed by only one node (the one that successfully locks) so that distributed dispatch task management can be achieved by relying solely on existing MySQL instance resources.In general, business applications that require relational data preservation will have their own MySQL instances, which will introduce a distributed dispatch management module at almost zero cost.After finalizing the preliminary plan for an overtime Saturday afternoon, it took several hours to build the wheel as follows:

conceptual design

First of all, all the dependencies used:

  • Uikit: A lightweight UI framework for the front end chosen, mainly considering the lightweight, relatively complete documentation and components.
  • JQuery: The js framework was chosen for one reason: simplicity.
  • Freemarker: Template engine, subjectively better than Jsp and Thymeleaf.
  • Quartz: Industrial dispatcher.

The project relies on the following:

<dependencies>
    <dependency>
        <groupId>org.quartz-scheduler</groupId>
        <artifactId>quartz</artifactId>
        <exclusions>
            <exclusion>
                <groupId>com.zaxxer</groupId>
                <artifactId>HikariCP-java7</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter</artifactId>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring-context-support</artifactId>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-jdbc</artifactId>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-freemarker</artifactId>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>com.zaxxer</groupId>
        <artifactId>HikariCP</artifactId>
        <scope>provided</scope>
    </dependency>
</dependencies>

Uikit and JQuery can use existing CDN s directly:

<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/uikit@3.2.2/dist/css/uikit.min.css"/>
<script src="https://cdn.jsdelivr.net/npm/uikit@3.2.2/dist/js/uikit.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/uikit@3.2.2/dist/js/uikit-icons.min.js"></script>
<script src="https://cdn.bootcss.com/jquery/3.4.1/jquery.min.js"></script>

Table Design

After introducing the dependency of Quartz, you can see a series of DDL s under its package org.quartz.impl.jdbcjobstore. In the general MySQL scenario, you can only focus on tables_mysql.sql and tables_mysql_innodb.sql. The engine of my team's development specification MySQL must select innodb, so choose the latter.

Timed task information in applications should be managed separately to provide uniform queries and change APIs.It is worth noting that Quartz's built-in tables use a large number of foreign keys, so try to delete and add the contents of its built-in tables through the API provided by Quartz. Do not operate manually, otherwise unexpected failures may occur.

Two new tables introduced include the schedule_task table and the schedule_task_parameter table:

CREATE TABLE `schedule_task`
(
    `id`               BIGINT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY COMMENT 'Primary key',
    `creator`          VARCHAR(16)     NOT NULL DEFAULT 'admin' COMMENT 'Creator',
    `editor`           VARCHAR(16)     NOT NULL DEFAULT 'admin' COMMENT 'Modifier',
    `create_time`      DATETIME        NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT 'Creation Time',
    `edit_time`        DATETIME        NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT 'Modification Time',
    `version`          BIGINT          NOT NULL DEFAULT 1 COMMENT 'version number',
    `deleted`          TINYINT         NOT NULL DEFAULT 0 COMMENT 'Soft Delete Identity',
    `task_id`          VARCHAR(64)     NOT NULL COMMENT 'Task Identification',
    `task_class`       VARCHAR(256)    NOT NULL COMMENT 'Task class',
    `task_type`        VARCHAR(16)     NOT NULL COMMENT 'Task type,CRON,SIMPLE',
    `task_group`       VARCHAR(32)     NOT NULL DEFAULT 'DEFAULT' COMMENT 'Task Grouping',
    `task_expression`  VARCHAR(256)    NOT NULL COMMENT 'Task expression',
    `task_description` VARCHAR(256) COMMENT 'Task Description',
    `task_status`      TINYINT         NOT NULL DEFAULT 0 COMMENT 'Task Status',
    UNIQUE uniq_task_class_task_group (`task_class`, `task_group`),
    UNIQUE uniq_task_id (`task_id`)
) COMMENT 'Schedule Tasks';

CREATE TABLE `schedule_task_parameter`
(
    `id`              BIGINT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY COMMENT 'Primary key',
    `task_id`         VARCHAR(64)     NOT NULL COMMENT 'Task Identification',
    `parameter_value` VARCHAR(1024)   NOT NULL COMMENT 'parameter values',
    UNIQUE uniq_task_id (`task_id`)
) COMMENT 'Schedule Task Parameters';

Parameters are stored in a JSON string, so a dispatch task entity corresponds to 0 or 1 dispatch task parameter entity.This does not take into account the issue of multiple applications using the same data source. In fact, this issue should consider isolation based on different org.quartz.jobStore.tablePrefix implementations, that is, if different applications share a common library, or if each application has a different table prefix to distinguish, or if all dispatch tasks are separately extracted into the same application.

Quartz's working mode

Quartz actually schedules the trigger Trigger when designing the scheduling model. Generally, when dispatching the corresponding task Job, it is necessary to bind the trigger and the dispatched task instance, then be fired when the trigger reaches the trigger time point, and then call back the execute() method of the Job instance associated with the trigger.It can be easily understood that triggers and Job instances are many-to-many relationships.Simply put, that's it:

To achieve this many-to-many relationship, Quartz defines JobKey and TriggerKey as unique identities for Job (actually JobDetail) and Trigger, respectively.

TriggerKey -> [name, group]
JobKey -> [name, group]

To reduce maintenance costs, the author enforces a one-to-one binding relationship, and assimilates TriggerKey and JobKey as follows:

JobKey,TriggerKey -> [jobClassName, ${spring.application.name} || applicationName]

In fact, most of the scheduling-related work is delegated to org.quartz.Scheduler, for example:

public interface Scheduler {
    ......Omit unrelated code......
    // Add Scheduled Task - Includes Task Content and Triggers
    void scheduleJob(JobDetail jobDetail, Set<? extends Trigger> triggersForJob, boolean replace) throws SchedulerException;

    // Remove Trigger
    boolean unscheduleJob(TriggerKey triggerKey) throws SchedulerException;
    
    // Remove Task Content
    boolean deleteJob(JobKey jobKey) throws SchedulerException;
    ......Omit unrelated code......
}

What I want to do is to manage the timed tasks of the service through the schedule_task table, transfer the specific operations of the task to Quartz through the API provided by org.quartz.Scheduler, and add some extensions.This module has been encapsulated as a lightweight framework called quartz-web-ui-kit, or kit.

kit core logic analysis

All the core functions of the kit are encapsulated in the module quartz-web-ui-kit-core. The main functions include:

The WebUI part is written simply through Freemarker, JQuery, and Uikit and consists of three main pages:

templates
  - common/script.ftl Common scripts
  - task-add.ftl  Add a new task page
  - task-edit.ftl Edit Task Page
  - task-list.ftl task list

The core method of scheduling task management is QuartzWebUiKitService#refreshScheduleTask():


@Autowired
private Scheduler scheduler;

public void refreshScheduleTask(ScheduleTask task,
                                Trigger oldTrigger,
                                TriggerKey triggerKey,
                                Trigger newTrigger) throws Exception {
    JobDataMap jobDataMap = prepareJobDataMap(task);
    JobDetail jobDetail =
            JobBuilder.newJob((Class<? extends Job>) Class.forName(task.getTaskClass()))
                    .withIdentity(task.getTaskClass(), task.getTaskGroup())
                    .usingJobData(jobDataMap)
                    .build();
    // Always Cover
    if (ScheduleTaskStatus.ONLINE == ScheduleTaskStatus.fromType(task.getTaskStatus())) {
        scheduler.scheduleJob(jobDetail, Collections.singleton(newTrigger), Boolean.TRUE);
    } else {
        if (null != oldTrigger) {
            scheduler.unscheduleJob(triggerKey);
        }
    }
}

private JobDataMap prepareJobDataMap(ScheduleTask task) {
    JobDataMap jobDataMap = new JobDataMap();
    jobDataMap.put("scheduleTask", JsonUtils.X.format(task));
    ScheduleTaskParameter taskParameter = scheduleTaskParameterDao.selectByTaskId(task.getTaskId());
    if (null != taskParameter) {
        Map<String, Object> parameterMap = JsonUtils.X.parse(taskParameter.getParameterValue(),
                new TypeReference<Map<String, Object>>() {
                });
        jobDataMap.putAll(parameterMap);
    }
    return jobDataMap;
}

In fact, any task trigger or change directly covers the corresponding JobDetail and Trigger, so that the content and triggers of the scheduled task are completely new and the next round of scheduling will take effect.

The task class is abstracted as AbstractScheduleTask, which hosts task execution and a number of extensions:

@DisallowConcurrentExecution
public abstract class AbstractScheduleTask implements Job {

    protected Logger logger = LoggerFactory.getLogger(getClass());

    @Autowired(required = false)
    private List<ScheduleTaskExecutionPostProcessor> processors;

    @Override
    public void execute(JobExecutionContext context) throws JobExecutionException {
        String scheduleTask = context.getMergedJobDataMap().getString("scheduleTask");
        ScheduleTask task = JsonUtils.X.parse(scheduleTask, ScheduleTask.class);
        ScheduleTaskInfo info = ScheduleTaskInfo.builder()
                .taskId(task.getTaskId())
                .taskClass(task.getTaskClass())
                .taskDescription(task.getTaskDescription())
                .taskExpression(task.getTaskExpression())
                .taskGroup(task.getTaskGroup())
                .taskType(task.getTaskType())
                .build();
        long start = System.currentTimeMillis();
        info.setStart(start);
        // Add traceId to MDC to facilitate tracing call chains
        MappedDiagnosticContextAssistant.X.processInMappedDiagnosticContext(() -> {
            try {
                if (enableLogging()) {
                    logger.info("task[{}]-[{}]-[{}]Start execution......", task.getTaskId(), task.getTaskClass(), task.getTaskDescription());
                }
                // Processor callback before execution
                processBeforeTaskExecution(info);
                // Task execution logic implemented by subclasses
                executeInternal(context);
                // Perform a successful processor callback
                processAfterTaskExecution(info, ScheduleTaskExecutionStatus.SUCCESS);
            } catch (Exception e) {
                info.setThrowable(e);
                if (enableLogging()) {
                    logger.info("task[{}]-[{}]-[{}]Execution Exception", task.getTaskId(), task.getTaskClass(),
                            task.getTaskDescription(), e);
                }
                // Processor callback executing exception
                processAfterTaskExecution(info, ScheduleTaskExecutionStatus.FAIL);
            } finally {
                long end = System.currentTimeMillis();
                long cost = end - start;
                info.setEnd(end);
                info.setCost(cost);
                if (enableLogging() && null == info.getThrowable()) {
                    logger.info("task[{}]-[{}]-[{}]completion of enforcement,time consuming:{} ms......", task.getTaskId(), task.getTaskClass(),
                            task.getTaskDescription(), cost);
                }
                // End of Execution Processor Callback
                processAfterTaskCompletion(info);
            }
        });
    }

    protected boolean enableLogging() {
        return true;
    }

    /**
     * Internal Execution Method-Subclass Implementation
     *
     * @param context context
     */
    protected abstract void executeInternal(JobExecutionContext context);

    /**
     * Copy Task Information
     */
    private ScheduleTaskInfo copyScheduleTaskInfo(ScheduleTaskInfo info) {
        return ScheduleTaskInfo.builder()
                .cost(info.getCost())
                .start(info.getStart())
                .end(info.getEnd())
                .throwable(info.getThrowable())
                .taskId(info.getTaskId())
                .taskClass(info.getTaskClass())
                .taskDescription(info.getTaskDescription())
                .taskExpression(info.getTaskExpression())
                .taskGroup(info.getTaskGroup())
                .taskType(info.getTaskType())
                .build();
    }
    
    // Callback before task execution
    void processBeforeTaskExecution(ScheduleTaskInfo info) {
        if (null != processors) {
            for (ScheduleTaskExecutionPostProcessor processor : processors) {
                processor.beforeTaskExecution(copyScheduleTaskInfo(info));
            }
        }
    }
    
    // Callback at the end of task execution
    void processAfterTaskExecution(ScheduleTaskInfo info, ScheduleTaskExecutionStatus status) {
        if (null != processors) {
            for (ScheduleTaskExecutionPostProcessor processor : processors) {
                processor.afterTaskExecution(copyScheduleTaskInfo(info), status);
            }
        }
    }
    
    // Callback at end of task
    void processAfterTaskCompletion(ScheduleTaskInfo info) {
        if (null != processors) {
            for (ScheduleTaskExecutionPostProcessor processor : processors) {
                processor.afterTaskCompletion(copyScheduleTaskInfo(info));
            }
        }
    }
}

The target dispatch task class that needs to be executed only needs to inherit AbstractScheduleTask to get these functions.In addition, ScheduleTaskExecutionPostProcessor, the Schedule Task Post Processor, references the design of BeanPostProcessor and TransactionSynchronization in Spring:

public interface ScheduleTaskExecutionPostProcessor {
    
    default void beforeTaskExecution(ScheduleTaskInfo info) {

    }

    default void afterTaskExecution(ScheduleTaskInfo info, ScheduleTaskExecutionStatus status) {

    }

    default void afterTaskCompletion(ScheduleTaskInfo info) {

    }
}

With the post-processor, various functions such as task warning and task execution log persistence can be completed.Through ScheduleTaskExecutionPostProcessor, the author has implemented the built-in alert function, abstracting an alert policy interface, AlarmStrategy:

public interface AlarmStrategy {

    void process(ScheduleTaskInfo scheduleTaskInfo);
}

// The default enabled implementation is no alert policy
public class NoneAlarmStrategy implements AlarmStrategy {

    @Override
    public void process(ScheduleTaskInfo scheduleTaskInfo) {

    }
}

Custom warning strategies can be obtained by overriding AlarmStrategy's Bean configuration, such as:

@Slf4j
@Component
public class LoggingAlarmStrategy implements AlarmStrategy {

    @Override
    public void process(ScheduleTaskInfo scheduleTaskInfo) {
        if (null != scheduleTaskInfo.getThrowable()) {
            log.error("Task Execution Exception,Task Content:{}", JsonUtils.X.format(scheduleTaskInfo), scheduleTaskInfo.getThrowable());
        }
    }
}

Through the custom reality of this interface, the author prints all alerts to the nail group inside the team, and prints information such as the execution time, status and time-consuming of the task. If an exception occurs, it will promptly @everyone, so as to facilitate timely monitoring of the health of the task and subsequent optimization.

Use kit project

The project structure of quartz-web-ui-kit is as follows:

quartz-web-ui-kit
  - quartz-web-ui-kit-core Core Package
  - h2-example H2 Demonstration of a database
  - mysql-5.x-example MySQL5.x Example of version
  - mysql-8.x-example MySQL8.x Example of version

If you just want to experience the kit functionality, download the project directly, launch club.throwable.h2.example.H2App in the h2-example module, and then visit http://localhost:8081/quartz/kit/task/list.

Based on the application of MySQL instances, here are some examples of MySQL 5.x with a large number of users to illustrate.Since the wheels have just been built and have not been tested for a while, they have not been delivered to Maven's repository for the time being, so you need to compile them manually:

git clone https://github.com/zjcscut/quartz-web-ui-kit
cd quartz-web-ui-kit
mvn clean compile install

Introduce dependencies (simply introduce quartz-web-ui-kit-core, and quartz-web-ui-kit-core depends on spring-boot-starter-web, spring-boot-starter-web, spring-boot-starter-jdbc, spring-boot-starter-freemarker, and HikariCP):

<dependency>
    <groupId>club.throwable</groupId>
    <artifactId>quartz-web-ui-kit-core</artifactId>
    <version>1.0-SNAPSHOT</version>
</dependency>
<!-- This is necessary, MySQL Driver package for -->
<dependency>
    <groupId>mysql</groupId>
    <artifactId>mysql-connector-java</artifactId>
    <version>5.1.48</version>
</dependency>

Add a configuration to implement QuartzWebUiKitConfiguration:

@Configuration
public class QuartzWebUiKitConfiguration implements EnvironmentAware {

    private Environment environment;

    @Override
    public void setEnvironment(Environment environment) {
        this.environment = environment;
    }

    @Bean
    public QuartzWebUiKitPropertiesProvider quartzWebUiKitPropertiesProvider() {
        return () -> {
            QuartzWebUiKitProperties properties = new QuartzWebUiKitProperties();
            properties.setDriverClassName(environment.getProperty("spring.datasource.driver-class-name"));
            properties.setUrl(environment.getProperty("spring.datasource.url"));
            properties.setUsername(environment.getProperty("spring.datasource.username"));
            properties.setPassword(environment.getProperty("spring.datasource.password"));
            return properties;
        };
    }
}

Because quartz-web-ui-kit-core was designed with the ImportBeanDefinitionRegistrar hook interface in mind of the loading order of some components, it is not possible to achieve attribute injection through @Value or @Autowired because these two annotations are processed in a later order, which is understood by the ApprScanner Configurer of MyBatis.A DDL script has been compiled in the quartz-web-ui-kit-core dependency:

scripts
  - quartz-h2.sql
  - quartz-web-ui-kit-h2-ddl.sql
  - quartz-mysql-innodb.sql
  - quartz-web-ui-kit-mysql-ddl.sql

Quatz-mysql-innodb.sql and quartz-web-ui-kit-mysql-ddl.sql need to be executed in advance in the target database.A relatively standard configuration file, application.properties, is as follows:

spring.application.name=mysql-5.x-example
server.port=8082
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
# This local is a pre-built local database
spring.datasource.url=jdbc:mysql://localhost:3306/local?characterEncoding=utf8&useUnicode=true&useSSL=false
spring.datasource.username=root
spring.datasource.password=root
# freemarker configuration
spring.freemarker.template-loader-path=classpath:/templates/
spring.freemarker.cache=false
spring.freemarker.charset=UTF-8
spring.freemarker.check-template-location=true
spring.freemarker.content-type=text/html
spring.freemarker.expose-request-attributes=true
spring.freemarker.expose-session-attributes=true
spring.freemarker.request-context-attribute=request
spring.freemarker.suffix=.ftl

Then you need to add a scheduling task class, just inherit club.throwable.quartz.kit.support.AbstractScheduleTask:

@Slf4j
public class CronTask extends AbstractScheduleTask {

    @Override
    protected void executeInternal(JobExecutionContext context) {
        logger.info("CronTask trigger,TriggerKey:{}", context.getTrigger().getKey().toString());
    }
}

Start SpringBoot's startup class, then visit http://localhost:8082/quartz/kit/task/list:

Add a timed task through the left button:

Current task expressions support two types:

  • CRON expression: The format is cron=your CRON expression, such as cron=*/20 * * * * *?.
  • Simple periodic execution expression: The format is intervalInMilliseconds=milliseconds value, such as intervalInMilliseconds=10000, which means 10000 milliseconds execution once.

Other optional parameters are:

  • repeatCount: Repeats a simple periodic task, defaulting to Integer.MAX_VALUE.
  • startAt: The timestamp of the task's first execution.

With regard to task expression parameters, neither very strict checks are considered nor strings are trim processed, requiring a compact, format-specific expression, such as:

cron=*/20 * * * * ?

intervalInMilliseconds=10000

intervalInMilliseconds=10000,repeatCount=10

Scheduling tasks also support input of user-defined parameters, which is currently simply a JSON string, which is processed once by Jackson and stored in the JobDataMap of the task, and is actually persisted to the database by Quartz:

{"key":"value"}

This can be obtained from JobExecutionContext#getMergedJobDataMap(), for example:

@Slf4j
public class SimpleTask extends AbstractScheduleTask {

    @Override
    protected void executeInternal(JobExecutionContext context) {
        JobDataMap jobDataMap = context.getMergedJobDataMap();
        String value = jobDataMap.getString("key");
    }
}

Other

With regard to kit, there are two design points that the author specializes based on the scenarios facing the project maintained by the team:

  1. AbstractScheduleTask uses the @DisallowConcurrentExecution annotation, which disables concurrent execution, meaning that only one service node can schedule tasks at the same trigger time when multiple nodes are involved.
  2. The Misfire policy is disabled for CRON-type tasks, which means that CRON-type tasks will not do anything if the trigger time is missed (see Quartz's Misfire policy).

If you cannot tolerate these two points, do not use this toolkit directly in production.

Summary

This paper simply introduces that the author has built a wheel of lightweight distributed dispatching service with the support of Quartz, which is easy to use and cost-saving.Unfortunately, since the services needed for scheduling tasks in the current team's projects are all internal shared services, the author did not spend much effort to improve the authentication, monitoring and other modules. This is also from the current business scenarios. If too many designs are introduced, it will evolve into a heavy scheduling framework such as Elastic-Job, which will violateSave deployment costs.

(This article finishes c-14-d e-a-20200410 recently too busy with this article for a long time...)

Tags: Spring MySQL FreeMarker SQL

Posted on Sat, 11 Apr 2020 17:03:00 -0700 by chuddyuk