Original link:tecdat.cn/?p=8336

Original source:Tuo End number according to the tribe public number

 

 

introduce

As Web applications grow and use increases, use cases become more diverse. We are now building and using websites to perform more complex tasks than ever before. Some of these tasks can be processed and feedback forwarded to the user immediately, while others require further processing and result forwarding at a later time. The increasing adoption of Internet access and Internet-enabled devices has resulted in increased end-user traffic.

In this article we will explore the use of Celery in scheduling background tasks in a Flask application to reduce the burden of resource intensive tasks and prioritize responses to end users.

What is a task queue?

A task queue is a mechanism for allocating small units of work or tasks that can be performed without interfering with the request-response cycle of most Web-based applications.

Task queues help delegate work that would otherwise slow down the application while waiting for a response. They can also be used to handle resource-intensive tasks when a host or process interacts with a user.

demonstration

We will build a Flask application that allows users to set up reminders that will be delivered to their email at a set time.

We will also provide the ability to customize the time before a message or reminder is invoked and sent to the user.

set

As with other projects, our work will take place in a virtual environment:

$ pipenv install --three
$ pipenv shell
Copy the code

For this project we will need to install Flask and Celery packages to get started:

$ pipenv install flask celery
Copy the code

Our Flask application file structure looks like this:

.├ ─ Pipfile # Manage Our Environment ├─ Pipfile.Lock ├─ readme.md ├─ __init__. Py ├─ App.py # Flask Application main Implementation Of ├─ config.py # Managed configuration ├─ ├─ science. TXT # Store our requirements ├─ science. TXTCopy the code

Let’s start by creating the Flask application, which renders a form that allows the user to enter details of messages to be sent in the future.

We added the following to our app.py file:

from flask import Flask, flash, render_template, request, redirect, url_for app = Flask(__name__) app.config.from_object("config") app.secret_key = app.config['SECRET_KEY'] @app.route('/',  methods=['GET', 'POST']) def index(): if request.method == 'GET': return render_template('index.html') elif request.method == 'POST': email = request.form['email'] first_name = request.form['first_name'] last_name = request.form['last_name'] message = Request. form['message'] duration = request.form['duration'] duration_unit = request.form['duration_unit'] flash(" message Scheduled ") return redirect(url_for('index')) if __name__ == '__main__': app.run(debug=True)Copy the code

This is a very simple application that requires a single route to process GET and POST request forms. After submitting the details, we can hand over the data to the scheduled work function.

To tidy up the main application file, we put the configuration variables in a separate config.py file, and then load the configuration from the file:

app.config.from_object("config")
Copy the code

Our config.py file will be in the same folder as the app.py file and will contain some basic configuration:

SECRET_KEY = 'very_very_secure_and_secret' # More configurationCopy the code

Now, let’s implement the target page as index.html:

{% for message in get_flashed_messages() %} <p style="color: red;" >{{ message }}</p> {% endfor %} <form method="POST"> First Name: <input id="first_name" name="first_name" type="text"> Last Name: <input id="last_name" name="last_name" type="text"> Email: <input id="email" name="email" type="email"> Message: <textarea id="textarea" name="message"></textarea> Duration: <input id="duration" name="duration" placeholder="Enter duration as a number. for example: 3" type="text"> <select name="duration_unit"> <option value="" disabled selected>Choose the duration</option> <option value="1">Minutes</option> <option value="2">Hours</option> <option value="3">Days</option> </select> <button type="submit" name="action">Submit </button> </form>Copy the code

Now we can launch our application:

Use mail to send email

To send email from the Flask application, we will use the flask-mail library, which is added to our project as follows:

$ pipenv install flask-mail
Copy the code

With the Flask application and forms, we can now integrate flask-mail into our app.py:

from flask_mail import Mail, Message app = Flask(__name__) app.config.from_object("config") app.secret_key = app.config['SECRET_KEY'] # Flask-mail = Mail(app) def send_mail(data): """ Function to send emails. """ with app.app_context(): msg = Message("Ping!" , sender="admin.ping", recipients=[data['email']]) msg.body = data['message'] mail.send(msg)Copy the code

The send_main(data) function receives the message to be sent and the recipient of the E-mail, and then invokes it after a specified time to send the E-mail to the user. Config.py In order for flask-mail to work properly, we also need to add the following variables to us:

Flask- MAIL_SERVER = 'smtp.googlemail.com' MAIL_PORT = 587 MAIL_USE_TLS = True MAIL_USERNAME = 'mail-username' MAIL_PASSWORD = 'mail-password'Copy the code

integration

With our Flask application ready and equipped with email sending capability, we can now integrate Celery to plan email sending at a later time. Our app.py will be modified again:

Celery import tasks = tasks (app.name) Broker =app.config['CELERY_BROKER_URL']) client.conf.update(app.config) # Add this decorator to @client.task def in our send_mail function @app.route('/', methods=['GET', 'POST']) def index(): if request. Method == 'GET': return render_template('index.html') elif request.method == 'POST': data = {} data['email'] = request.form['email'] data['first_name'] = request.form['first_name'] data['last_name'] = request.form['last_name'] data['message'] = request.form['message'] duration = int(request.form['duration']) duration_unit = request.form['duration_unit'] if duration_unit == 'minutes': duration *= 60 elif duration_unit == 'hours': duration *= 3600 elif duration_unit == 'days': duration *= 86400 send_mail.apply_async(args=[data], countdown=duration) flash(f"Email will be sent to {data['email']} in {request.form['duration']} {duration_unit}") return  redirect(url_for('index'))Copy the code

The celery client is initialized in the Flask application using the URL of the attached message passing agent. In our example, we will use Redis as the proxy, so we add the following to our config.py:

CELERY_BROKER_URL = 'redis://localhost:6379/0'
CELERY_RESULT_BACKEND = 'redis://localhost:6379/0'
Copy the code

To make our send_mail() function perform as a background task we will add the @client.task decorator so that our Celery client will be aware of this.

After setting the Celery client, it will modify the main functions that also handle form input.

First, we wrap the input data for the send_mail() function in a dictionary. We then call the mail function through the Celery Task Calling API using the function apply_async which takes the parameters required by the function.

An optional countdown parameter is set that defines the delay between running the code and executing the task.

 

collection

To run our project we will need two terminals, one to start our Flask application and the other to start the Celery worker which will send messages in the background.

Start the Flask application in the first terminal:

$ python app.py
Copy the code

In the second terminal, start the virtual environment and then start the Celery worker:

Virtualenv $pipenv shell $celery worker -A app.client --loglevel=infoCopy the code

If all goes well we will get the following feedback in the terminal running the Celery client:

 

Now let’s navigate to http://localhost:5000 and fill in the details to plan for the email to arrive 2 minutes after submission.

At the top of the table, a message is displayed indicating the address to which the E-mail will be received and the duration of sending the E-mail. In our Celery terminal we will also be able to see a log entry indicating that our emails have been scheduled:

[2019-10-23 16:27:25,399: INFO/MainProcess] Received task: App. Send_mail [d65025c8 a291-40 d0 aea2 - e816cb40cd78] ETA: [the 2019-10-23 13:29:25. 170622 + 00:00]Copy the code

Part of the ETA entry shows when send_email() calls our function and when the email is sent.

So, let’s implement a monitoring solution for the background tasks so that we can look at the tasks and watch for problems and tasks that are not executed as planned.

Monitor our cluster

Installing Flower is very simple:

$ pipenv install flower
Copy the code

Previously we specified Celery client details in the app.py file. We need to pass this customer to Flower to monitor it.

To do this, we need to open a third terminal window, enter the virtual environment, and start the monitoring tool:

$ pipenv shell
$ flower -A app.client --port=5555
Copy the code

When starting the Flower we specify the Celery client by passing it to the Application (-a) parameter which specifies the port to use –port.

With our monitoring capabilities, let’s schedule another email to be sent on the dashboard and then navigate to http://localhost:5555, where we’ll welcome it:

On this page we see a list of workers in the Celery cluster which currently consists only of our machines.

To view the email we just planned, click the Tasks button at the top left of the dashboard, which takes us to the page where we can view the planned tasks:

In this section, we can see that we have scheduled two e-mails and that we have successfully sent one E-mail at the scheduled time. For testing purposes, plan to send emails one and five minutes later, respectively.

 

conclusion

We have successfully built the Celery cluster and integrated it into our Flask application, which allows users to plan to send emails after some time in the future.

---

Most welcome insight

1. How to solve the problem of SINGULAR FIT in linear mixed model

2. Install the R software package on the UBUNTU VM

3. Run R language commands on the CLI in WINDOWS

Error: OBJECT OF TYPE ‘SYMBOL’ IS NOT SUBSETTABLE

5. How to create R DATAFRAME from XML file

PACKAGE OR NAMESPACE LOAD FAILED IN dyn. LOAD(FILE, DLLPATH = DLLPATH,…) :

7.PYTHON installs TENSORFLOW 2, tf.keras, and deep learning model definitions

8.R language drawing does not show Chinese solution, how to use Chinese font

9. Write reproducible reports in R using MARKDOWN, KNITR, and PANDOC