In the previous article, detailed introduction of azkaban service installation, now use SpringBoot to develop an Azkaban scheduling task, upload to the Web interface to run.

Azkaban Scheduling type

Azkaban supports a wide variety of task types, as explained on the official website, including the following:

  • Command: Uses Linux shell command line tasks.
  • HadoopShell: This is the same type of Command as Command, but can communicate with the Hadoop cluster
  • JavaTask: the Java
  • hadoopJava: is also a Java type that can communicate with hadoop clusters. You can create most Hadoop job types by running hadoopJava jobs, such as Pig, Hive, etc
  • Pig: Pig script task
  • Hive: Supports hiveSQL tasks

In the official website, each task type has response example, task configuration also has carried on the detailed instructions, the official website address: http://xiaoshuai.github.io/azkaban-gh-pages/#new-hive-type

Develop Java type tasks

Next, develop a Java type task that uploads Azkaban to run.

“Requirement: simply query user information from the database and print it out.“

Data preparation

First, prepare the data to be queried.

Construction sentences:

create table if not exists ts_userinfo

(

 name varchar(255) charset utf8 null.

 age int null.

 job varchar(255) charset utf8 null.

 address varchar(255) charset utf8 null.

 `desc` varchar(255) charset utf8 null

);

Copy the code

Table structure:


Insert data to query:

The task to develop

The project adopts SpringBoot for development, and the directory structure is as follows:


The structure is very simple, I believe you can understand the officer.

application.properties

The application. Properties configuration file is used to configure the mysql data source and mybatis configuration, as shown below:

# port

server.port=8080



# mybatis configuration file path

mybatis.mapper-locations=classpath*:mapper/*.xml



# mysql configuration

Spring. The datasource. Url = JDBC: mysql: / / 121.196.166.1 xx: 3306 / azkaban

spring.datasource.username=root

spring.datasource.password=123456

spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver

Copy the code

SqlSessionFactory/SqlSessionTemplate/SqlSessionTemplate/SqlSessionFactory/SqlSessionTemplate Property sqlSessionFactory or sqlSessionTemplate are required

MybatisConfig

Configure the data source, as shown below



@Configuration

@MapperScan(basePackages = {"com.tsmyk.azkaban.azkaban.dao"}, sqlSessionTemplateRef = "sqlSessionTemplate")

public class MybatisConfig {



    @Value("${mybatis.mapper-locations}")

    private String mybatisMapperLocations;

    @Value("${spring.datasource.url}")

    private String url;

    @Value("${spring.datasource.username}")

    private String username;

    @Value("${spring.datasource.password}")

    private String password;

    @Value("${spring.datasource.driver-class-name}")

    private String driverClassName;



    @Bean(name = "dataSource")

    public DataSource getDataSource(a){

        DruidDataSource dataSource =  new DruidDataSource();

        dataSource.setUrl(url);

        dataSource.setUsername(username);

        dataSource.setPassword(password);

        dataSource.setDriverClassName(driverClassName);

        return dataSource;

    }



    @Bean(name = "sqlSessionFactory")

    @Primary

    public SqlSessionFactory sqlSessionFactory(@Qualifier("dataSource") DataSource dataSource) throws Exception {

        SqlSessionFactoryBean bean = new SqlSessionFactoryBean();

        bean.setDataSource(dataSource);

        bean.setMapperLocations(new PathMatchingResourcePatternResolver().getResources(mybatisMapperLocations));

        return bean.getObject();

    }



    @Bean(name = "transactionManager")

    public DataSourceTransactionManager transactionManager(@Qualifier("dataSource") DataSource dataSource){

        return new DataSourceTransactionManager(dataSource);

    }



    @Bean(name = "sqlSessionTemplate")

    public SqlSessionTemplate sqlSessionTemplate(@Qualifier("sqlSessionFactory") SqlSessionFactory sqlSessionFactory){

        return new SqlSessionTemplate(sqlSessionFactory);

    }

}

Copy the code

TsUserinfoDao

Define the Mybatis operation interface as follows:

@Mapper

public interface TsUserinfoDao {

   List<TsUserinfo> queryAllUserInfo(a);

}

Copy the code

TsUserinfoDao.xml

Write mybatis SQL file as follows:

<mapper namespace="com.tsmyk.azkaban.azkaban.dao.TsUserinfoDao">

    <resultMap id="BaseResultMap" type="com.tsmyk.azkaban.azkaban.pojo.TsUserinfo">

        <result column="name" jdbcType="VARCHAR" property="name"/>

        <result column="age" jdbcType="INTEGER" property="age"/>

        <result column="job" jdbcType="VARCHAR" property="job"/>

        <result column="address" jdbcType="VARCHAR" property="address"/>

        <result column="desc" jdbcType="VARCHAR" property="desc"/>

    </resultMap>

    <select id="queryAllUserInfo" resultType="com.tsmyk.azkaban.azkaban.pojo.TsUserinfo">

        select * from ts_userinfo

    </select>

</mapper>

Copy the code

AzkabanServiceImpl

The main implementation of the query logic, as follows:

@Service

public class AzkabanServiceImpl implements  IAzkabanService  {

    

    private static final Logger  LOGGER = LoggerFactory.getLogger(AzkabanServiceImpl.class);

    @Autowired

    private TsUserinfoDao tsUserinfoDao;



    @Override

    public List<TsUserinfo> queryAllUserInfo(a) {

        LOGGER.info("Query user information........");

        List<TsUserinfo> userinfos = tsUserinfoDao.queryAllUserInfo();

        LOGGER.info("End Querying user information........");

        userinfos.forEach(System.out::println);

        return userinfos;

    }

}

Copy the code

AzkabanApplication

Modify the startup class as follows:

@PropertySource("classpath:application.properties")

@SpringBootApplication(exclude = {DataSourceAutoConfiguration.class})

public class AzkabanApplication {

    private static final Logger LOGGER = LoggerFactory.getLogger(AzkabanApplication.class);



    public static void main(String[] args) {

        LOGGER.info("Start the mission......");

        ApplicationContext context = new AnnotationConfigApplicationContext(AzkabanApplication.class);

        IAzkabanService azkabanService = context.getBean(IAzkabanService.class);

        List<TsUserinfo> userinfos = azkabanService.queryAllUserInfo();

        LOGGER.info("Task completed, result: \r\n");

        userinfos.forEach(System.out::println);

    }

}

Copy the code

Run tests:

After that, start AzkabanApplication and the log is printed as follows:


At this point, azkaban Java-type tasks are developed and then packaged and deployed to run on Azkaban.

Packaged deployment

The first step is to use Maven to create a JAR package for the project. You can use the tools that come with IDEA:


The second step is to export all the JAR packages that the project depends on. You can also use the tool of IDEA to export the jar packages as follows:

  • Click on the Maven navigation barExecute Maven Goal
  • The inputdependency:copy-dependencies -DoutputDirectory=D:\\“Command, and click OK

Step 3: Create a job task file with the suffix.job.

Set task type to Java task

type=javaprocess

# specify the startup class

java.class=com.tsmyk.azkaban.AzkabanApplication

Specify the configuration file and the dependent JAR package

classpath=resources,lib/ *

Copy the code

Of course, you can specify more parameters, such as JVM parameters, Xms, Xmx, and so on.

Finally, put the jar packages that the project depends on and the jar packages of the project into the lib folder, the lib folder, the Resources folder, and the job folder into the compressed package:


Then log in azkaban Web management terminal, project project, upload the compressed package.



After the upload is successful, run it.

At this point, an Azkaban simple Java type task is developed, and of course you can also set the scheduling period in the interface.

This article was first published in personal public number [Java technology programming], welcome to pay attention to.