Large computations and batch tasks can be done in Kubernetes by creating a workload resource Job. For example, Job transcoding files, obtaining partial files and directories, and training tasks in machine learning. In k8S, there are many types of cronjobs, such as job and cronjob.

Job creation

We can create a simple K8S Job using the API version Batch /v1

#new-job.yml
apiVersion: batch/v1
kind: Job
metadata:
  name: command-job
spec:
  template:
    spec:
      containers:
      - name: command-job
        image: busybox
        command: ["/bin/sh","-c","sleep 5;echo 'job one'"]
      restartPolicy: Never
Copy the code

The Job object will launch a pod to do our Job — sleep for 5s, and then output Job One

Apply the job definition to view the job status:

$ kubectl apply -f new-job.yml 
job.batch/command-job created
Copy the code

After the task is Completed, the POD state is set to Completed:

Logs to view the results of our task execution:

Job restart and failure identification

In our example above, the Job Pod does the job. When a POD is performing a job, the container may fail to start for a number of reasons, such as the process exiting with a non-zero code or exceeding the memory limit. In the pod template, you can use restartPolicy to control the restartPolicy of job pod.

RestartPolicy:

  • Never: If the POD fails to start, it will not restart but passjob-controllerRecreate the POD for node scheduling.
  • OnFailure: Pod will restart the node to perform the task.

BackoffLimit:

If the Job Pod fails to restart multiple times, it should be considered a failed Job. By default, the number of failed Job restarts is 6. We can change this by adding backoffLimit to the spec.

We adjust new-job. Yml as follows:

#new-job.yml
apiVersion: batch/v1
kind: Job
metadata:
  name: command-job-two
spec:
  template:
    spec:
      containers:
      - name: command-job-two
        image: busybox
        command: ["/bin/sh","-c","sleep 5;echo 'job two';exit 1"]
      restartPolicy: Never
  backoffLimit: 2
Copy the code

We use Describe to view the jobs created

After job-controller rebuilds the POD twice and reaches the threshold, the job-controller identifies this job as a failed workflow.

If the restart policy is set to Never, the failed Job will leave the POD on the node.

Job deadline and clearing

In addition to Job termination defined by Job execution completion and restart failure, you can automatically stop Job tasks by setting the activeDeadlineSeconds.

We can for the Job. The spec. ActiveDeadlineSeconds set a second value. This value applies throughout the life of the Job, regardless of how many pods the Job creates. Once the running time of a Job reaches the activeDeadlineSeconds seconds, all running pods are terminated, and the status of the Job is updated to Type: Failed and Reason: DeadlineExceeded.

Pay attention to the Job. The spec. ActiveDeadlineSeconds above its priority. Spec. BackoffLimit Settings. Therefore, if a Job is retrying one or more invalid pods, the Job will not deploy additional pods once it reaches the time limit set by the activeDeadlineSeconds, even if the number of retries has not reached the limit set by backoffLimit.

Adjust new-job. Yml as follows:

#new-job.yml
apiVersion: batch/v1
kind: Job
metadata:
  name: command-job-three
spec:
  template:
    spec:
      containers:
      - name: command-job-three
        image: busybox
        command: ["/bin/sh","-c","sleep 50;echo 'job three'"]
      restartPolicy: Never
  backoffLimit: 2
  activeDeadlineSeconds: 10
​
Copy the code

Although the Job is a 50s task, due to restrictions imposed by the activeDeadlineSeconds, the Job is terminated after 10 seconds

Cleaning job and termination are similar, we can add the spec. TtlSecondsAfterFinished makes the job a period of time after the completion of the mission is cleared, interested readers can begin to try.

Job Task type

  • The concurrent Job

    Normally, only one Pod is started, unless the Pod fails and the application in the Pod successfully completes, the Job task is considered to be in the completed state, which is the task we discussed above.

  • Parallel Job

    • Parallel jobs that specify the number of tasks

      Spec.com pletions specifies the number of tasks once all pods have successfully completed their tasks. Homework will be completed.

      We add a new-jobs.yml and specify completions to 3

      apiVersion: batch/v1
kind: Job
metadata:
  name: command-jobs
spec:
  template:
    spec:
      containers:
      - name: command-jobs
        image: busybox
        command: ["/bin/sh","-c","sleep 50;echo 'jobs '"]
      restartPolicy: Never
  backoffLimit: 2
  completions: 3
Copy the code

      When all three PODS are completed, the Job status is successfully executed.

    In this mode, the pod creation sequence is shown in the running process of Job Pod. That is, after a Job is completed, start the next Job.

    • Parallel jobs of work queue type

      Once one Pod terminates, all pods terminate, and the Job completes successfully.

      Modify new-jobs.yml and add Parallelism to make it 5

      apiVersion: batch/v1
kind: Job
metadata:
  name: command-jobs
spec:
  template:
    spec:
      containers:
      - name: command-jobs
        image: busybox
        command: ["/bin/sh","-c","sleep 50;echo 'jobs '"]
      restartPolicy: Never
  backoffLimit: 2
  parallelism: 5
​
Copy the code

      This type of Job Pod is created and terminated at the same time.

Cronjob Periodic task

CronJob Is used to perform periodic operations, such as backup, mail, and report generation.

The cron time configuration is similar to the Linux crontab.

# ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ time zone (optional) # | ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ minutes (0-59) # | │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ hours (0-23) # | │ │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ In one day (# 1-31) | │ │ │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ the month (# 1-12) | │ │ │ │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ one day of week (0 to 6) (Sunday to Monday; In some systems, # 7 is Sunday) | │ │ │ │ │ # | │ │ │ │ │ # | │ │ │ │ │ # CRON_TZ = UTC * * * * *Copy the code

Add cronjob. Yml as follows:

#cronjob.yml
apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: cronjob
spec:
  schedule: "*/1 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: cronjob
            image: busybox
            command: ["/bin/sh","-c","date"]
          restartPolicy: Never
Copy the code

We print the date every minute with cronJob.

View cronJob information:

View task results via logs:

[docker@localhost yml]$ kubectl logs cronjob-1635010680-n5gxj
Sat Oct 23 17:38:15 UTC 2021
Copy the code

Cronjob can automatically clean up tasks, the default reserve three successful mission, we can add. Spec. SuccessfulJobsHistoryLimit change historical task information is retained Pod.

Above, we have introduced most of the content related to Job and Cronjob in K8S.

Reference:

Job

Running Automated Tasks with a CronJob

I hope the essay is helpful to you. Please correct me if there is any mistake in the content.

You are free to reprint, modify and publish this article without my consent. Iqsing.github. IO