Benjamin Smith

Recently I made the final step in converting my website over to nginx. I decided to manage my django application, byteflow, with supervisor. I have had great success with this over at AGI and thought I should advocate the success I’ve had to the public by extending it to my own site.

I first heard about supervisor at pycon last year and thought it could be useful in many ways, especially at the office. In a nutshell, supervisor ‘supervises’ processes, and allows you to manage them with a simple interface, but I’ll go into more detail about it later. Around the same time that I discovered supervisor I had also started experimenting with nginx and fastcgi to run my blog. I ended up going with lighty and fastcgi instead, however, mainly due to familiarity. Things change, and so do my opinions on technology. Nginx sold me on it’s performance and simplistic configuration, plain and simple. So now, here’s the meat and potatoes.

To break it down, here is what we’re looking at.

INTARWEBS -> nginx -> fastcgi -> django/byteflow(managed by supervisor)

Everything below assumes you have installed nginx, django, byteflow and supervisor.

So how about a look at the actual setup.

First, let’s take a look at the code to fire up the django app. I’ve dubbed it runserver.py and put it in my byteflow code tree.

#!/usr/bin/env python
if __name__ == '__main__':
    from flup.server.fcgi_fork import WSGIServer
    from django.core.handlers.wsgi import WSGIHandler
    WSGIServer(WSGIHandler()).run()

This keeps the django devs happy by not relying on manage.py to keep up with the server stuffs.

It’s pretty simple.. Starts up flups’ WSGIServer and uses django’s WSGIHandler to do the dirty work. Not much to it, really.

Now, lets take a look at the supervisor setup. You can get a lot of what you need by running:

# sudo echo_supervisord_conf > /etc/supervisord.conf

That sets up a basic (but verbose!) supervisor config. I’ve trimmed it down to this:

[unix_http_server]
file=/tmp/supervisor.sock   ; (the path to the socket file)

[supervisord]
logfile=/var/log/supervisord/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB       ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10          ; (num of main logfile rotation backups;default 10)
loglevel=info               ; (log level;default info; others: debug,warn,trace)
pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=false              ; (start in foreground if true;default false)
minfds=1024                 ; (min. avail startup file descriptors;default 1024)
minprocs=200                ; (min. avail process descriptors;default 200)
user=nobody                 ; (default is current user, required if root)
childlogdir=/var/log/supervisord/            ; ('AUTO' child log dir, default $TEMP)

[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL  for a unix socket

; Production setup
[fcgi-program:django_fcgi]
socket=tcp://127.0.0.1:8080  ; We reference this later in nginx
command = /home/bsmith/Dev/byteflow/runserver.py  ; Calls the above code
environment=PYTHONPATH=/home/bsmith/Dev/byteflow  ; Setup needed environment
environment=DJANGO_SETTINGS_MODULE=settings

; Development setup
[fcgi-program:django_dev_fcgi]
socket=tcp://127.0.0.1:8081
command = /home/bsmith/Dev/byteflow_new/runserver.py
environment=PYTHONPATH=/home/bsmith/Dev/byteflow_new
environment=DJANGO_SETTINGS_MODULE=settings

Simple enough, eh? Comments in the configuration should explain what’s going on.

Now you can crank it up by running:

$ sudo supervisord

and check the status…

$ sudo supervisorctl status
django_dev_fcgi:django_dev_fcgi_0 RUNNING    pid 15949, uptime 0:00:08
django_fcgi:django_fcgi_0        RUNNING    pid 15950, uptime 0:00:08

Could it be more simple? I doubt it. This barely begins to scratch the surface of what supervisor can do. Process groups/pools, XML-RPC interface for remote management, built-in web interface for process management(utilizing XML-RPC interface), tons of process management options (priority, umask, user/group, capture std* pipes, environment variables, auto restart/start, process naming), event listeners/handling and a simple configuration ta boot! All I’m sayin’ is, it’s awesome…I’m just sayin’.

Keeping with our simple but awesome theme, enter nginx…

My main config:

user www-data;
# Could vary by number of processors available.
worker_processes  1;

error_log  /var/log/nginx/error.log;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
    use epoll;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    access_log      /var/log/nginx/access.log;

    sendfile        on;

    keepalive_timeout  65;
    tcp_nodelay        on;

    gzip  on;

    include /etc/nginx/sites/*;

}

and my various site-specific configurations that I keep /etc/nginx/sites:

server {
        # We listen on port 80
        listen 80;
        server_name just-another.net;

        # access and error logs for our site
        access_log /var/log/nginx/my_site_access_log;
        error_log /var/log/nginx/my_site_error_log;

        # Configure redirect for our fastcgi server
        # The fastcgi server later runs on localhost Port 8080
        location / {
                expires 10d;
                # to fastcgi server could use socket..
                #fastcgi_pass unix:{project_location}/log/django.sock;
                fastcgi_pass 127.0.0.1:8080;
                fastcgi_param PATH_INFO $fastcgi_script_name;
                fastcgi_param REQUEST_METHOD $request_method;
                fastcgi_param QUERY_STRING $query_string;
                fastcgi_param CONTENT_TYPE $content_type;
                fastcgi_param CONTENT_LENGTH $content_length;
                fastcgi_param REMOTE_ADDR $remote_addr;
                fastcgi_pass_header Authorization;
                fastcgi_intercept_errors on;
        }

        location /images {
    # For my own images..
            alias /home/bsmith/public_html/images;
            expires 10d;
        }

        # Alias for static content like themes
        location /static {
            alias /home/bsmith/Dev/byteflow/static;
            expires 10d;
        }

        # Alias for python contrib.admin stuff, needed for admin interface
        location /admin-media {
    # Point to my most recent install of django.
            alias /home/bsmith/Dev/django_trunk/django/contrib/admin/media;
            expires 10d;
        }
        # Point to media
        location /media {
            alias /home/bsmith/public_html/media;
            expires 10d;
        }
        # Use feedburner for my feeds.
        rewrite ^/rpc(.*) http://feeds.feedburner.com/Just-anothernetBlogPosts$1 permanent;
}

Pretty simple, eh? I have the same server setup for my ‘dev’ site, only with a volatile code base that I can hack on.

Now when I make code changes all I need to do is reload the supervisor process like so:

$ sudo supervisorctl restart django_fcgi:django_fcgi_0

for the production site or replacing the process name with django_dev_fcgi:django_dev_fcgi_0 to restart the dev site. I could also restart everything by replacing a process name with all as an argument.

The possibilities here for process management endless and I couldn’t be happier with how simple it has become to run what I have.

Basically, this setup is easy to get running and easy to keep running. Highly recommended for any lazy django site maintainer like myself!