DS 2.0 era API operation posture

DS 2.0 era API introduction

background

The much-anticipated Version of Apache Dolphinschedler 2.0 was finally released on December 17, 2021, with 70 percent of the code refactored and large JSON split planned for January 2021. It took 11 months until the release of 2.0.0 on December 5, 2021 (it was taken down due to unexpected bugs), and finally 2.0.1 was released on December 17, followed by 2.0.2 and the current version 2.0.3 two weeks later.

In ds version 2.0, the interface has been changed to has a Restful style way of request, the corresponding swagger document address is http://ip:port/dolphinscheduler/doc.html, from swagger documents as you can see, the interface classification is clear. We focus on workflow, task, and some interfaces of workflow and task relationship.

Workflow Definition Interface

The workflow definition has 25 interfaces in the current version. Here are some of the more commonly used interfaces:

Task definition interface

Workflow task relationship interface

Code calls interface mode

If you call the interface using code alone, you need a token, which can be generated using the admin user on the interface (creating a token) or obtained by calling the interface

Maven rely on

<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.6</version>
</dependency>
Copy the code

Code sample

private static String DOLPHIN_BASE_URI = "http://ip:port";
private static String token = "xxx";
private static String sendPost(String uri, List<NameValuePair> params) throws Exception {
    CloseableHttpClient httpclient = HttpClients.createDefault();
    CloseableHttpResponse response = null;
    try {
        UrlEncodedFormEntity formEntity = new UrlEncodedFormEntity(params, Consts.UTF_8);
        HttpPost httpPost = new HttpPost(DOLPHIN_BASE_URI + uri);
        httpPost.setEntity(formEntity);
        httpPost.setHeader("token", token);
        response = httpclient.execute(httpPost);
        return EntityUtils.toString(response.getEntity(), Consts.UTF_8);
    } catch (Exception e) {
        throw new Exception(String.format("[dolphin] The %s call failed", uri));
    } finally {
        try {
            if(response ! =null) {
                response.close();
            }
            httpclient.close();
        } catch(IOException e) { e.printStackTrace(); }}}private static String sendGet(String uri, List<NameValuePair> params) throws Exception {
    CloseableHttpClient httpclient = HttpClients.createDefault();
    CloseableHttpResponse response = null;
    try {
        HttpGet httpGet = new HttpGet(new URIBuilder(DOLPHIN_BASE_URI + uri).setParameters(params).build());
        httpGet.setHeader("token", token);
        response = httpclient.execute(httpGet);
        return EntityUtils.toString(response.getEntity(), Consts.UTF_8);
    } catch (Exception e) {
        throw new Exception(String.format("[dolphin] The %s call failed", uri));
    } finally {
        if(response ! =null) { response.close(); } httpclient.close(); }}Copy the code

Two ways to create workflows

Call createProcessDefinition (1)

    public static void main(String[] args) throws Exception {
        long projectCode = 4362891840832L;
        String uri = String.format("/dolphinscheduler/projects/%d/process-definition", projectCode);
        List<NameValuePair> params = new ArrayList<>();
        params.add(new BasicNameValuePair("name"."lee-test-04"));
        params.add(new BasicNameValuePair("projectCode", projectCode + ""));
        String taskDefinitionJson = "[{\"code\":4143298469059,\"name\":\"lee-test-4\",\"description\":\"\",\"delayTime\":0,\"taskType\":\"SHELL\"," +
                "\"taskParams\":{\"resourceList\":[],\"localParams\":[],\"rawScript\":\"echo 11333\",\"dependence\":{},\"conditionResult\"" +
                ":{\"successNode\":[],\"failedNode\":[]},\"waitStartTimeout\":{},\"switchResult\":{}},\"flag\":\"YES\",\"taskPriority\": \"MEDIUM\"," +
                "\"workerGroup\":\"default\",\"failRetryTimes\":0,\"failRetryInterval\":1,\"timeoutFlag\":\"CLOSE\",\"timeoutNotifyStrat egy\":\"WARN\"," +
                "\"timeout\":0,\"environmentCode\":-1}]";
        params.add(new BasicNameValuePair("taskDefinitionJson", taskDefinitionJson));
        params.add(new BasicNameValuePair("taskRelationJson"."[{\"name\":\"\",\"preTaskCode\":0,\"preTaskVersion\":0,\"postTaskCode\":4143298469059,\"conditionType\":0,\"conditionPa rams\":{}}]"));
        params.add(new BasicNameValuePair("tenantCode"."root"));
        params.add(new BasicNameValuePair("description".""));
        params.add(new BasicNameValuePair("globalParams"."[]"));
        params.add(new BasicNameValuePair("timeout"."0"));
        sendPost(uri, params);
    }
Copy the code

(2) call workflow createEmptyProcessDefinition, task, save, save the workflow task relations

ProcessDefinition createEmptyProcessDefinition api

TaskDefinition save api

ProcessTaskRelation save api

Recommended reading

Thread pool usage and parsing

Java Basics – Lock core

Process and optimization of a search performance

Kubernetes Scheduler source code parsing and custom resource scheduling algorithm practice

, recruiting

Zhengcaiyun Technology team (Zero) is a passionate, creative and executive team based in picturesque Hangzhou. The team has more than 300 r&d partners, including “old” soldiers from Alibaba, Huawei and NetEase, as well as newcomers from Zhejiang University, University of Science and Technology of China, Hangzhou Electric And other universities. Team in the day-to-day business development, but also in cloud native, chain blocks, artificial intelligence, low code platform system, middleware, data, material, engineering platform, the performance experience, visualization technology areas such as exploration and practice, to promote and fell to the ground a series of internal technical products, continue to explore new frontiers of technology. In addition, the team is involved in community building, Currently, There are Google Flutter, SciKit-Learn, Apache Dubbo, Apache Rocketmq, Apache Pulsar, CNCF Dapr, Apache DolphinScheduler, and Alibaba Seata and many other contributors to the excellent open source community. If you want to change something that’s been bothering you, want to start bothering you. If you want to change, you’ve been told you need more ideas, but you don’t have a solution. If you want change, you have the power to make it happen, but you don’t need it. If you want to change what you want to accomplish, you need a team to support you, but you don’t have the position to lead people. If you want to change the original savvy is good, but there is always a layer of fuzzy window…… If you believe in the power of believing, believing that ordinary people can achieve extraordinary things, believing that you can meet a better version of yourself. If you want to be a part of the process of growing a technology team with deep business understanding, sound technology systems, technology value creation, and impact spillover as your business takes off, I think we should talk. Any time, waiting for you to write something and send it to [email protected]

Wechat official account

The article is published synchronously, the public number of political cloud technology team, welcome to pay attention to