How To Serve Static Files in Django

Monday, January 14, 2013

I’ve been lamenting the state of static files in Django for a while now. The app has caused one problem after another for me. In one situation, after running `./manage.py collectstatic`, my entire development environment was wiped. In another, I found staticfiles was completely unreliable when syncing static media to remote storage backends, due to the way it calculates whether a file has changed (I posted this problem to the Django developers mailing list a couple months ago, but nothing came of it).

After reading a tweet that perfectly echoed my feelings on the subject, I decided it was time to write down how I solve the “staticfiles problem” in my own apps, in a way that works flawlessly on both development and production.

After following this tutorial, you will be serving all static files on your development machine from a folder called “static” in your Django project, and you will easily be able to sync static media to any storage backend quickly and easily.

This tutorial is for a base Django installation. I generally use my own project template that takes care of setting sane defaults for me (and in use in tens of production projects), but given the reality here, I know a lot of you reading this are probably reading this with legacy applications in mind. I hope these instructions work for you, but if they don’t, please send me an email (first name at this domain).

Serving Static Files on a Development Server

To keep it simple, let’s start a Django project using the standard template.

$ django-admin.py startproject statictest

Switch into the folder and start a virtual environment.

$ virtualenv .
New python executable in ./bin/python
Installing distribute..........................................................................................................................................................................................................done.
Installing pip................done.
$ . bin/activate

Set the variables in your `settings.py` file to the values below.

# This import belongs at the top of your settings file
import os

BASE = os.path.abspath(os.path.dirname(__name__))

STATICFILES_DIRS = (os.path.join(BASE, "static"),)
ADMIN_MEDIA_PREFIX = '/static/admin/'
STATIC_URL = "/static/"

Note: in a real-life situation, you will want to split out your local and production settings files, and symlink them as necessary for deployment. The above is purely for illustration purposes.

Here’s the magic step. To get all files in your “static” folder serving from the “/static/” URL on the development server, add these lines to your `urls.py` file.

from django.contrib.staticfiles.urls import staticfiles_urlpatterns

# ...

urlpatterns += staticfiles_urlpatterns()

To test it out, just create a folder called “static” in the base of your project directory. If you did everything correctly to this point, you should see the following in your “statictest” folder:

$ ls
bin        include    lib        manage.py  static     statictest

Start the development server, and everything in that folder will be accessible from “http://localhost:8000/static/”. Congrats. You rock.

Serving Static Files in Production

Frankly, it’s not super useful if all your static files just live locally. I tried to get staticfiles to play nicely with Amazon S3, but I just couldn’t do it.

Why? Well, firstly, `./manage.py collectstatic` will upload files that have already been uploaded if you use a modern DVCS and work with other developers (essentially, this means it will reupload files every time you pull down your code from the remote). It does this because it uses last modified times as the heuristic for deciding whether a file has changed. Unfortunately for every modern developer, we all know that Git and other tools set the last modified time of a time to whenever code was last pulled from the server, not when the code was actually last modified.

Secondly, the django-storages implementation for retrieving last modified time from Amazon S3 requires sending an HTTP request to AWS for each file you’re syncing. This, needless to say, is extremely slow.

Because my proposals for getting these issues fixes in Django core were denied, I wrote a Django application called statictastic. The app adds a new management command called `syncmedia` which, well, syncs your static media. It’s fast and works on all storage backends built on the Django API.

So, how is it so much faster than the collectstatic implementation? Well, it starts by creating a metadata file that contains the md5 checksums of the files that currently exist in your project. When you first run `./manage.py syncmedia`, statictastic creates this metadata file and uploads it to the remote backend. Then, every following time you run the command, statictastic compares your local md5 checksums to those on the remote backend. When statictastic notices that a checksum has changed, it re-uploads the outdated file and updates the remote metadata file as well.

If you’d like to use it, the first thing you’ll want to do is install it via pip:

pip install boto
pip install django-storages
pip install django-statictastic==0.6.1

Next up, in your settings.py:

INSTALLED_APPS = (
    # ...
    statictastic,
)

DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
STATICFILES_STORAGE = 'statictastic.backends.VersionedS3BotoStorage'

# Optionally change these to full CDN urls
UPLOAD_URL = "https://s3.amazonaws.com/uploads-statictest/"
STATIC_URL = "https://s3.amazonaws.com/static-statictest/"

# I like to store my uploads and static media on different buckets.
AWS_STATIC_STORAGE_BUCKET_NAME = 'static-statictest'
AWS_STORAGE_BUCKET_NAME = 'uploads-statictest'

# Access Keys, straight from Amazon
AWS_ACCESS_KEY_ID = 'XXXXXXXXXXXXXXXXXXXX'
AWS_SECRET_ACCESS_KEY = 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'

Now, go ahead and test it out.

$ python manage.py syncmedia
Updated favicon.ico
1 file updated

Awesome. You’re done. To reference static files in your HTML files, use the “static” template tag.

But wait! Not so fast. For some inexplicable reason, Django has two static template tags. The one that’s in your templates by default isn’t the one you want. The one you want you’ll have to manually import.

{% load static from staticfiles %}
<link rel='shortcut icon' href='{% static "favicon.ico" %}' />

The above template code would generate the following HTML:

<link rel='shortcut icon' href='https://s3.amazonaws.com/static-statictest/favicon.ico' />

Because I like to break my caches, I added a optional setting for statictastic called `COMMIT_SHA` that adds a querystring to all static files referenced using the `static` template tag. If you set it to `123456` in your settings.py file, you’d see this in your template:

<link rel='shortcut icon' href='https://s3.amazonaws.com/static-statictest/favicon.ico?123456' />

And that concludes my overview of how I make static files work on Django. If you have any questions, ping me on Twitter or send me an email.

All the code found in this post can be found in my company’s Django template hosted on GitHub.