Let’s talk about how to build the simplest celery frame and how to use the native IDE to start server for data monitoring.

A, Celery

1, components,

The simplest and most suitable RabbitMQ is officially recommended. In this example, RabbitMQ is used to store middleware and result sets, rather than other components such as Redis, MySQL, etc.

2, configuration,

Celery_config stores Celery configuration. Celery_config stores Celery configuration. Celery_config stores Celery configuration. Tasks like celery_instance mainly instantiate celery main start celery server task_register tasks task_test tasks build file, carry out the test

# broker 
BROKER_URL = 'it: / / root: [email protected] / dispatch_server_host'
# backend
CELERY_RESULT_BACKEND = 'it: / / root: [email protected] / dispatch_server_host'
# time zone
CELERY_TIMEZONE = 'Asia/Shanghai'
# include
CELERY_INCLUDE = ['celery_test.task_register']
Copy the code

BROKER_URL Message middleware connection CELERY_RESULT_BACKEND task result data store CELERY_INCLUDE task registration path

Celery_instance. py instantiates celery

from celery import Celery
from celery_test import celery_config
app = Celery()
Load the configuration file into the instance
app.config_from_object(celery_config)
Copy the code

Main. py starts the celery server

from celery_instance import app
if __name__ == '__main__':
    app.start()
Copy the code

Task_register. py where we register three tasks add,sum, and prod

from celery_test.celery_instance import app
@app.task
def add(x, y):
    return x + y

@app.task
def sum(*args):
    """ :param args: ([2, 2], ) :return: """
    print args
    res = 0
    for i in args[0]:
        res += i
    print '<sum> res:{}'.format(res)
    return res

@app.task
def prod(*args):
    print args
    res = 1
    for i in args[0]:
        res *= i
    print '<sum> res:{}'.format(res)
    return res
Copy the code

Task_test.py starts with a simple Add test

# -*- coding:utf-8 -*-
from celery_test.task_register import *
from celery import group
from celery import chord

sig = add.s(1.1)
sig.apply_async()
Copy the code

Celery start

There are two ways to start celery here, Celery -A celery_test. Main worker -n work_test -c 1 -l info -q celery -A celery_test -q Specifies the name of the monitored queue, if not specified in the command it will monitor the queue configured in the configuration file, if not, it will default to the queue

1. Start RabbitMQ

Start celery

3. Submit the test task

Sig = add.s(1, 1) cyl = apply_async(

Third, summary

Simple build of distributed celery task scheduling system is very simple, for the overall task process is so go to see my other article, portal: blog.csdn.net/qq_35869630…

However, celery is not so simple in practical application, how to put different tasks into different queues, how to ensure the priority of tasks, how to ensure that tasks are not repeated etc. There are many problems needing attention in practical production.

In the next stop I will continue with the setup of celery queues and task priority in queues.

The best relationship is mutual achievement, if there is a little help for you, your triple link is the biggest motivation for my creation, thank you!

Note: If there are any mistakes or suggestions in this blog, please point them out. Thank you!