uWSGI with cookiecutter Pyramid Application Part 2: Adding Emperor and systemd

This guide will outline broad steps that can be used to add the uWSGI Emperor and systemd to our cookiecutter application that is being served by uWSGI.

This is Part 2 of a two-part tutorial, and assumes that you have already completed Part 1: uWSGI with cookiecutter Pyramid application Part 1: Basic uWSGI + nginx.

This tutorial was developed under Ubuntu 18.04, but the instructions should be largely the same for all systems, where you may adjust specific path information for commands and files.

Conventional Invocation of uWSGI

In Part 1 we used --init-paste-logged which got us two things almost for free: logging and an implicit WSGI entry point.

In order to run our cookiecutter application with the uWSGI Emperor, we will need to follow the conventional route of providing an (explicit) WSGI entry point.

  1. Within the project directory (~/myproject), create a script named wsgi.py with the following code. This script is our WSGI entry point.

    1# Adapted from PServeCommand.run in site-packages/pyramid/scripts/pserve.py
2from pyramid.scripts.common import get_config_loader
3app_name    = 'main'
4config_vars = {}
5config_uri  = 'production.ini'
6
7loader = get_config_loader(config_uri)
8loader.setup_logging(config_vars)
9app = loader.get_wsgi_app(app_name, config_vars)

    config_uri is the project configuration file name. It's best to use the production.ini file provided by your cookiecutter, as it contains settings appropriate for production. app_name is the name of the section within the .ini file that should be loaded by uWSGI. The assignment to the variable app is important: we will reference app and the name of the file, wsgi.py when we invoke uWSGI.

    The call to loader.setup_logging initializes the standard library's logging module through pyramid.paster.setup_logging() to allow logging within your application. See Logging Configuration.

  2. Create a directory for your project's log files, and set ownership on the directory.

    $ cd /var/log
$ sudo mkdir uwsgi
$ sudo chown ubuntu:www-data uwsgi

  3. Uncomment these three lines of your production.ini file.

    1[uwsgi]
2# Uncomment `wsgi-file`, `callable`, and `logto` during Part 2 of this tutorial
3wsgi-file = wsgi.py
4callable = app
5logto = /var/log/uwsgi/%(proj).log

    wsgi-file points to the explicit entry point that we created in the previous step. callable is the name of the callable symbol (the variable app) exposed in wsgi.py. logto specifies where your application's logs will be written, which means logs will no longer be written to STDOUT.

  4. Invoke uWSGI with --ini.

    Invoking uWSGI with --ini and passing it an .ini file is the conventional way of invoking uWSGI. (uWSGI can also be invoked with all configuration options specified as command-line arguments, but that method does not lend itself to easy configuration with Emperor, so we will not present that method here.)

    $ cd ~/myproject
$ sudo uwsgi --ini production.ini

    Make sure you call it with sudo, or your application will not be able to masquerade as the users we specified for uid and gid.

    Also note that since we specified the logto parameter to be in /var/log/uwsgi, we will see only limited output in this terminal window. If it starts up correctly, all you will see is this:

    $ sudo uwsgi --ini production.ini
[uWSGI] getting INI configuration from production.ini

  5. Tail the log file at var/log/uwsgi/myproject.log.

    $ tail -f /var/log/uwsgi/myproject.log

    and verify that the output of the previous step includes a line that looks approximately like this:

    WSGI app 0 (mountpoint='/') ready in 1 seconds on interpreter 0x5615894a69a0 pid: 8827 (default app)

    If any errors occurred, you will need to correct them. If you get a callable not found or import error, make sure that your production.ini properly sets wsgi-file to wsgi.py, and that ~/myproject/wsgi.py exists and contains the contents provided in a previous step. Also make sure that your production.ini properly sets callable to app, and that app is the name of the callable symbol in wsgi.py.

    An import error that looks like ImportError: No module named 'wsgi' probably indicates that your wsgi-file specified in production.ini does not match the wsgi.py file that you actually created.

    For any other import errors, it probably means that the package either is not installed or is not accessible by the user. That's why we chose to masquerade as the normal user that you log in as, so you would for sure have access to installed packages.

  6. Visit http://localhost in a browser. Alternatively call curl localhost from a terminal. You should see the sample application rendered.

  7. If the application does not render, follow the same steps you followed in uWSGI with cookiecutter Pyramid application Part 1: Basic uWSGI + nginx to get the nginx connection flowing.

  8. Stop your application. Now that we've demonstrated that your application can run with an explicit WSGI entry point, your application is ready to be managed by the uWSGI Emperor.

Running Your application via the Emperor

  1. Create two new directories in /etc.

    $ sudo mkdir /etc/uwsgi/
$ sudo mkdir /etc/uwsgi/vassals

  2. Create an .ini file for the uWSGI emperor and place it in /etc/uwsgi/emperor.ini.

    1# /etc/uwsgi/emperor.ini
2[uwsgi]
3emperor = /etc/uwsgi/vassals
4limit-as = 1024
5logto = /var/log/uwsgi/emperor.log
6uid = ubuntu
7gid = www-data

    Your application is going to run as a vassal. The emperor line in emperor.ini specifies a directory where the Emperor will look for vassal config files. That is, for any vassal config file (an .ini file) that appears in /etc/uwsgi/vassals, the Emperor will attempt to start and manage that vassal.

  3. Invoke the uWSGI Emperor.

    $ cd /etc/uwsgi
$ sudo uwsgi --ini emperor.ini

    Since we specified logto in emperor.ini, a successful start will only show you this output:

    $ sudo uwsgi --ini emperor.ini
[uWSGI] getting INI configuration from emperor.ini

  4. In a new terminal window, start tailing the emperor's log.

    $ sudo tail -f /var/log/uwsgi/emperor.log

    Verify that you see this line in the emperor's output:

    *** starting uWSGI Emperor ***

    Keep this window open so you can see new entries in the Emperor's log during the next steps.

  5. From the vassals directory, create a symbolic link that points to your applications's production.ini.

    $ cd /etc/uwsgi/vassals
$ sudo ln -s ~/myproject/production.ini

    As soon as you create that symbolic link, you should see traffic in the Emperor log that looks like this:

    [uWSGI] getting INI configuration from production.ini
Sun Jul 15 13:34:15 2018 - [emperor] vassal production.ini has been spawned
Sun Jul 15 13:34:15 2018 - [emperor] vassal production.ini is ready to accept requests

  6. Tail your vassal's log to be sure that it started correctly.

    $ tail -f /var/log/uwsgi/myproject.log

    A line similar to this one indicates success:

    WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x563aa0193bf0 pid: 14984 (default app)

  7. Verify that your vassal is available via nginx. As in Part 1, you can do this by opening http://localhost in a browser, or by curling localhost in a terminal window.

    $ curl localhost

  8. Stop the uWSGI Emperor, as now we will start it via systemd.

Running the Emperor via systemd

  1. Create a systemd unit file for the Emperor with the following code, and place it in /lib/systemd/system/emperor.uwsgi.service.

     1# /lib/systemd/system/emperor.uwsgi.service
 2[Unit]
 3Description=uWSGI Emperor
 4After=syslog.target
 5
 6[Service]
 7ExecStart=/usr/bin/uwsgi --ini /etc/uwsgi/emperor.ini
 8# Requires systemd version 211 or newer
 9RuntimeDirectory=uwsgi
10Restart=always
11KillSignal=SIGQUIT
12Type=notify
13StandardError=syslog
14NotifyAccess=all
15
16[Install]
17WantedBy=multi-user.target

  2. Start and enable the systemd unit.

    $ sudo systemctl start emperor.uwsgi.service
$ sudo systemctl enable emperor.uwsgi.service

  3. Verify that the uWSGI Emperor is running, and that your application is running and available on localhost. Here are some commands that you can use to verify:

    1$ sudo journalctl -u emperor.uwsgi.service # System logs for emperor
2
3$ tail -f /var/log/nginx/access.log /var/log/nginx/error.log
4
5$ tail -f /var/log/uwsgi/myproject.log
6
7$ sudo tail -f /var/log/uwsgi/emperor.log

  4. Verify that the Emperor starts up when you reboot your machine.

    $ sudo reboot

    After it reboots:

    $ curl localhost

  5. Congratulations! You've just deployed your application in robust fashion.

uWSGI has many knobs and a great variety of deployment modes. This is just one representation of how you might use it to serve up a cookiecutter Pyramid application. See the uWSGI documentation for more in-depth configuration information.

This tutorial is modified from the original tutorial Running a Pyramid Application under mod_wsgi.