BatchCompute is a distributed cloud service suitable for large-scale parallel batch jobs. BatchCompute supports massive concurrent jobs. The system automatically manages resources, schedules jobs, loads data, and charges based on actual usage. BatchCompute is widely used in film animation rendering, biological data analysis, multimedia transcoding, financial insurance analysis, scientific computing and other fields.

features

1. Submit homework

Users submit jobs to BatchCompute using tools (such as SDK and command line tools). BatchCompute starts the VM using the image specified by the user (such as Ubuntu), runs user programs on the VM, and releases the VM.

OSS is used as persistent storage in BatchCompute. You can save the resulting data to OSS when the program finishes running. In batch computing, data on OSS can also be accessed through the file interface, see THE OSS mount function.

The BatchCompute program runs on the VM by default and can also support Docker containers.

That is, you can customize the ECS image or use Docker to install any software you need to run any of your programs.

Job description

Users need to submit a Job (Job) describes the JSON file to batch computing services, the JSON file describes in detail what you need to perform program (can be multiple programs, running what what procedures need to start the machine, the machine’s specifications (such as memory and CPU), where run log printing, where the result after the completion of the output, etc.

A Job contains multiple tasks that are executed in the order described by the DAG you specify.

Each task defines which image to use, which instance specification to use, which program to run, how many machines to run it on, and where to store the results.

2. Manage my homework

You can use tools (console, command line tools, etc.) to view my submitted job and can stop, restart, or delete the job. View individual tasks, individual instances (VM instances), and logs. The following figure shows the job management interface of the console:

3. Use a cluster

The VM needs to be started before running the program each time, which takes a certain amount of time (usually several minutes). If you are busy, you may not be able to apply for resources (the VM is used by other customers). Therefore, you may need to wait for a period of time after submitting the job.

If you want to improve performance, you can first create a cluster, specify the number of VMS (for example, 5) and the image ID. BatchCompute assigns machines to you and starts them. These machines are always running, and can run directly once you submit the job.

4. Customize the mirror

When submitting a job or creating a cluster, you can use either the official images provided by batch Computing or a custom image. The advantage of custom mirrors is that you can install the required software yourself.

Course list: http://click.aliyun.com/m/51393/

Lesson 1: Docker application demonstration

课时2: console to submit and manage assignments

Lesson 3: Command line tool installation and configuration

Lesson 4: Command line tools to submit and manage assignments

Class 5: Demonstration of cloud rendering management system

Lesson 6: Custom mirror