Running a Flask Application as a Service with Systemd
When deploying an application on a server, you need to ensure that the application is running continuously. If the application crashes, you want it to restart automatically, and if the server loses power, you want the application to start as soon as power is restored. Basically, what you need is to monitor the application and restart it when you find that it is no longer running.
In the previous tutorial, I showed you how it is designed to be done using the container container container, a third-party utility written in Python. Today, I’ll show you a similar solution based on systemd, which is a native component in many Linux distributions, including Debian derivatives (such as Ubuntu) and RedHat derivatives (such as Fedora and CentOS).
Configure the service using Systemd
Systemd is configured through an entity called Unit. There are several types of units, including service, socket, device, timer, and so on. For services, the Unit configuration file must have a. Service extension. Below, you can see the basic structure of the service Unit configuration file:
[Unit]
Description=<a description of your application>
After=network.target
[Service]
User=<username>
WorkingDirectory=<path to your app>
ExecStart=<app start command>
Restart=always
[Install]
WantedBy=multi-user.target
Copy the code
The [Unit] section is common to all types of Unit configuration files. It is used to configure general information about Unit and any dependencies that help the system determine the boot order. In my template, I added a description of the service, and I also specified that I wanted my application to start after the network subsystem was initialized, because it was a Web application.
The [Service] section contains details specific to your application. I use the most common options to define the user to run the service, the starting directory, and the command to execute. The Restart option tells Systemd that in addition to starting the service at system startup, I also want to Restart the application if it exits. This can resolve crashes or other unexpected problems that could end the process.
Finally, the [Install] section configures how and when to enable the Unit. By adding the WantedBy=multi-user. Target line I tell Systemd to activate this unit when the system is running in multi-user mode, which is the normal mode for Unix servers at runtime. For more details on multi-user mode, see the discussion on Unix RunLevels.
Add the unit configuration file to the /etc/systemd/system directory for systemd to view. Every time a unit file is added or modified, systemd must be told to refresh its configuration:
$ sudo systemctl daemon-reload
Copy the code
You can then start, stop, restart, or get service status using the systemctl
command:
$ sudo systemctl start <service-name>
$ sudo systemctl stop <service-name>
$ sudo systemctl restart <service-name>
$ sudo systemctl status <service-name>
Copy the code
Note: You can use the service command to manage the service instead of systemctl. In most distributions, the service command maps to systemctl and gives the same result.
Write a system configuration file for the Flask application
If you want to create a Systemd service file for your own application, just use the template above and fill in Description, User, WorkingDirectory and ExecStart.
As an example, let’s say I want to deploy the Microblog application mentioned in the Flask Mega-tutorial on a Linux server, but I want to use Systemd to monitor the process, rather than the container supervisord.
For your reference, here’s the container configuration file I’m using for the tutorial it’s handling:
[program:microblog]
command=/home/ubuntu/microblog/venv/bin/gunicorn -b localhost:8000 -w 4 microblog:app
directory=/home/ubuntu/microblog
user=ubuntu
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
Copy the code
Systemd equivalent unit configuration file will be written to the/etc/systemd/system/microblog service, and will have the following contents:
[Unit]
Description=Microblog web application
After=network.target
[Service]
User=ubuntu
WorkingDirectory=/home/ubuntu/microblog
ExecStart=/home/ubuntu/microblog/venv/bin/gunicorn -b localhost:8000 -w 4 microblog:app
Restart=always
[Install]
WantedBy=multi-user.target
Copy the code
Notice how the boot command reaches inside the virtual environment to get executable Gunicorn. This is equivalent to activating the virtual environment and then running Gunicorn without a path, but this has the advantage of being done in a single command.
After adding this file to your system, you can start the service using the following command:
$ sudo systemctl daemon-reload
$ sudo systemctl start microblog
Copy the code
The environment variable
If the Flask application wants to set one or more environment variables in advance, they can be added to the service file. For example, if you need to set the FLASK_CONFIG and DATABASE_URL variables, you can define them using the Environment option as follows:
[Unit]
Description=Microblog web application
After=network.target
[Service]
User=ubuntu
WorkingDirectory=/home/ubuntu/microblog
Environment=FLASK_CONFIG=production
Environment=DATABASE_URL=sqlite:////path/to/the/database.sqlite
ExecStart=/home/ubuntu/microblog/venv/bin/gunicorn -b localhost:8000 -w 4 microblog:app
Restart=always
[Install]
WantedBy=multi-user.target
Copy the code
Please note that if you follow my tutorial style and use.env files for environment variables, there is no need to add them through the Systemd service file. In fact, I prefer to work with the environment through.env files because it’s a unified approach for development and production.
Access log
Systemd has a journaling subsystem called Journal, implemented by the Journald daemon, which collects logs for all running Systemd cells. You can use the Journalctl utility to view the contents of the journal. Here are some examples of common log access commands.
Check log of microblog service:
$ journalctl -u microblog
Copy the code
Check the last 25 log entries of microblog service:
$ journalctl -u microblog -n 25
Copy the code
Following the log of microblog service:
$ journalctl -u microblog -f
Copy the code
There are more options available. Run journalctl –help for a more complete summary of options.
Advanced usage: run Worker Pools using Systemd
If you run background processes with Celery it is easy to extend the above solution to your workers as Celery allows you to start the worker process pool with a single command. This is actually the same as with Gunicorn with multiple workers, so all you need to do is create a second.service file to manage the main Celery process, which in turn will manage the worker.
However, if you read the last few chapters of my Flask Mega-Tutorial, you will know that I have introduced an RQ-based task queue to perform background tasks. When using RQ, you must start workers separately, and there is no main process to manage workers pool for you. This is how I manage RQ workers using Supervisor in my tutorial:
[program:microblog-tasks]
command=/home/ubuntu/microblog/venv/bin/rq worker microblog-tasks
numprocs=1
directory=/home/ubuntu/microblog
user=ubuntu
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
Copy the code
In this case, the numprocs parameter allows you to start as many workers as you need. With this parameter, the Supervisor starts and monitors a specified number of instances from a single configuration file.
Unfortunately, there is no numprocs option in Systemd, so this type of service requires a different solution. The simplest approach is to create a separate service file for each working instance, but doing so can be cumbersome. Instead, what I’m going to do is create the service file as a template that can be used to launch all of these same instances:
[Unit]
Description=Microblog task worker %I
After=network.target
[Service]
User=ubuntu
WorkingDirectory=/home/ubuntu/microblog
ExecStart=/home/ubuntu/microblog/venv/bin/rq worker microblog-tasks
Restart=always
[Install]
WantedBy=multi-user.target
Copy the code
The odd thing you might notice in this file is that I added %I to the service description. This is the service parameter, a number to be passed to each instance. Including this %I in the description will help me identify the instance, because all output from the systemd command will be replaced with the instance number. In this particular case, I don’t actually need to use this parameter, but it is common to include %I in other fields, such as using the start command if necessary.
With regular service file another difference is, I will use the/etc/systemd/system/microblog – the tasks @. This service is service the name to write the file. The @ in the filename indicates that this is a template, so there will be a parameter after it to identify each instance derived from it. I’ll use the instance number as an argument, so the different instances of the service will be called microblog-tasks@1, microblog-tasks@2, and so on in Systemd.
Now I can start four workers in bash using the curly braces extension:
$ sudo systemctl daemon-reload $ sudo systemctl start microblog-tasks@{1.. 4} $ sudo systemctl status microblog-tasks@{1.. 4}Copy the code
If you want to handle an instance separately, you can also do this:
$ sudo systemctl restart microblog-tasks@3
Copy the code
This is almost as handy as the individual container configuration, but it has the small disadvantage of having {1.. 4}.
To truly treat the entire worker pool as one entity, I can create a new Systemd Target, which is another type of unit. I can then map all instances to the target, which will allow me to reference the target when I want to perform operations on all members of the group. Let’s start from the unit of the new target configuration files, I named it the/etc/systemd/system/microblog – the tasks. The target:
[Unit]
Description=Microblog RQ worker pool
[Install]
WantedBy=multi-user.target
Copy the code
Beyond the description, the only definition needed is the dependency on multi-user.target, which, as you recall, is the target of all the unit files defined above.
Now I can update the service file template to reference the new target, which is ultimately equivalent to the new target due to the transitive reference to the original multi-user.target.
[Unit]
Description=Microblog task worker %I
After=network.target
[Service]
User=ubuntu
WorkingDirectory=/home/ubuntu/microblog
ExecStart=/home/ubuntu/microblog/venv/bin/rq worker microblog-tasks
Restart=always
[Install]
WantedBy=microblog-tasks.target
Copy the code
The system can now reconfigure with the new Settings using the following command:
$ sudo systemctl daemon-reload
$ sudo systemctl disablemicroblog-tasks@{1.. 4} $ sudo systemctlenablemicroblog-tasks@{1.. 4}Copy the code
The disable and enable commands must be used to force Systemd to remove old targets and apply new ones to worker tasks. Woker pool can now be handled using target:
$ sudo systemctl restart microblog-tasks.target
Copy the code
If you decide to add a fifth worker later, you can do this:
$ sudo systemctl enable microblog-tasks@5
$ sudo systemctl start microblog-tasks.target
Copy the code
Of course, you can also reduce workers. Here’s how to reduce worker 4 and 5:
$ sudo systemctl stop microblog-tasks@{4.. 5} $ sudo systemctl disable microblog-tasks@{4.. 5}Copy the code
At this point, I think this solution surpasses supervisor’s Numprocs commands in terms of convenience and functionality, because I can not only control the entire worker process, but also add and remove workers without having to edit any configuration files!
