arteme.fi

Introduction

Django web framework is famous for how easy it is to start developing
application with it. An installation of python and django is all you need to
get busy. Django provides you with all the essential gears and gadgets
including a simple development web server that you can use to run the
application you are developing, while you are developing it.

However, as your project grows more and more complex, you will discover the
limitations of the development server. The first one you are most likely to
face is the inability to serve static files (like site-specific media files).
To be frank, it is not so much an inability but reluctance — there is a
well-documented recepie of how to set up serving of static files, but it
has “do not use in production” written all over it. In fact, any use of the
development server in production environment is discouraged, as the following
disclaimer found in the documentation promptly states:

DO NOT USE THIS SERVER IN A PRODUCTION SETTING. It has not gone through
security audits or performance tests. (And that’s how it’s gonna stay.
We’re in the business of making Web frameworks, not Web servers, so
improving this server to be able to handle a production environment is
outside the scope of Django.)

Another, more serious limitation is its inherent single-threadedness. If you
are developing any kind of AJAX application that might need to have more than
one connection to the web server at any point in time you will run into
problems — the development server will only start processing the second
request once it’s done with processing the first one. Recently, the topic of
a more advanced, multi-threaded development server has been coming up in the
mailing lists more and more often. All the pleads to improve the default dev.
server were met with “won’t fix” comments pointing out that it is extremely
easy to hookup a custom CherryPy server to the django WSGI request handler
achieving this functionality.

Basic CherryPy setup

As of version 1.0, Django provides an interface for adding custom management
commands, which can be used to integrate Django with CherryPy.
http://oebfare.com/blog/2008/nov/03/writing-custom-management-command/ is a
good article on how to integrate the two, but for the sake of clarity and
completeness let’s reiterate.

A word of warning: The code presented here is for CherryPy
version 3.1 since it uses WSGIPathInfoDispatcher class. It may be
possible to back-port it to version 3.0, but I make no guaranties.

We will add a “runcherrypy” command, so that the server is started as:

./manage.sh runcherrypy [-h <address>] [-p <port>]

The following file tree will be added to the project directory:

devserver/
 |-- __init__.py
 `-- management/
      |-- __init__.py
      `-- commands/
           |-- __init__.py
           `-- runcherrypy.py

where runcherrypy.py is as follows:

from django.core.management.base import BaseCommand
from django.core.handlers.wsgi import WSGIHandler

from cherrypy.wsgiserver import CherryPyWSGIServer

from optparse import OptionParser, make_option

class Command( BaseCommand ):

    option_list = BaseCommand.option_list + (
            make_option( "-h", "--host", dest="host", default="127.0.0.1" ),
            make_option( "-p", "--port", dest="port", default=8000 ),
            )

    def handle( self, *args, **options ):

        app = WSGIHandler()
        server = CherryPyWSGIServer( (options['host'], options['port']),
                                     app )

        try:
            server.start()
        except KeyboardInterrupt:
            server.stop()

    def create_parser( self, prog, subcommand ):
        """
        Create and return the ``OptionParser`` which will be used to
        parse the arguments to this command.
        """
        return OptionParser( prog=prog,
                             usage=self.usage(subcommand),
                             version=self.get_version(),
                             option_list=self.option_list,
                             conflict_handler="resolve" )

Now, add 'devserver' to your settings.py and you’re ready to go! If
you have set up your static file serving in django, as specified earlier,
everything should “just work”. But why go through the network of django
resolvers just to arrive at basic static page serving if you can do it on
the CherryPy side?

Serving static files

A careful observer might notice that django actually serves the admin
interface static content quite happily. That is done by
django.core.servers.basehttp.AdminMediaHandler class. Steven Wilcox
at http://www.devpicayune.com has already taken AdminMediaHandler into
use in his article on django/CherryPy integration. If only one could use
AdminMediaHandler to serve static content outside of the
.../contrib/admin/media/ directory…

Why not? Here is a modified version of AdminMediaHandler applicable for
serving any static content. We will put it to devserver/mediahandler.py:

import os, stat, mimetypes
import django
from django.utils.http import http_date
from django.conf import settings

class MediaHandler( object ):

    def __init__( self, media_root ):
        self.media_root = media_root

    def __call__( self, environ, start_response ):

        def done( status, headers, output ):
            start_response( status, headers.items() )
            return output

        path_info = environ['PATH_INFO']
        if path_info[0] == '/':
            path_info = path_info[1:]

        file_path = os.path.join( self.media_root, path_info )

        if not os.path.exists( file_path ):
            status = '404 NOT FOUND'
            headers = {'Content-type': 'text/plain'}
            output = ['Page not found: %%s' %% file_path]
            return done( status, headers, output )

        try:
            fp = open( file_path, 'rb' )
        except IOError:
            status = '401 UNAUTHORIZED'
            headers = {'Content-type': 'text/plain'}
            output = ['Permission denied: %%s' %% file_path]
            return done( status, headers, output )

        # This is a very simple implementation of conditional GET with
        # the Last-Modified header. It makes media files a bit speedier
        # because the files are only read off disk for the first request
        # (assuming the browser/client supports conditional GET).

        mtime = http_date( os.stat(file_path)[stat.ST_MTIME] )
        headers = {'Last-Modified': mtime}
        if environ.get('HTTP_IF_MODIFIED_SINCE', None) == mtime:
            status = '304 NOT MODIFIED'
            output = []
        else:
            status = '200 OK'
            mime_type = mimetypes.guess_type(file_path)[0]
            if mime_type:
                headers['Content-Type'] = mime_type
            output = [fp.read()]
            fp.close()

        return done( status, headers, output )

CherryPy version 3.1 provides WSGIPathInfoDispatcher class which exposes
an interface to run multiple WSGI applications side-by-side, distinguished
by request path. We will utilize this to set up three applications:

• Django framework project, served by WSGIHandler: this will be placed
  at webserver root, ‘/’.
• Site media, served by MediaHandler: project settings file,
  settings.py, defines MEDIA_ROOT and MEDIA_URL, which are the
  path to the directory that hold the media and its URL on the server.
  We will serve site media from that URL.
• Admin media, served by MediaHandler: project settings file,
  settings.py, defines ADMIN_MEDIA_PREFIX, which is the URL prefix
  on the server. We will serve admin media from that URL.

This is how runcherrypy.py needs to be modified:

...
import django.contrib.admin

from cherrypy.wsgiserver import CherryPyWSGIServer, WSGIPathInfoDispatcher
from devserver.mediahandler import *
import os.path
...

    def handle( self, *args, **options ):

        app = WSGIHandler()

        path = { '/': app,
                 settings.MEDIA_URL: MediaHandler( settings.MEDIA_ROOT ),
                 settings.ADMIN_MEDIA_PREFIX:
                     MediaHandler(
                         os.path.join( django.contrib.admin.__path__[0],
                                       'media' )
                         )
               }
        dispatcher = WSGIPathInfoDispatcher( path )
        server = CherryPyWSGIServer( (options['host'], options['port']),
                                     dispatcher )

As you notice, we deduce the on-disk path of the admin media based on the
__path__ variable of the imported django.contrib.admin module.

Finally, a test application

First, a few practical thought on the layout of things. Django insists
that MEDIA_ROOT is an absolute path. If you are hosting your project
completely with CherryPy, you will probably keep all your media files
together with the project. I prefer to keep my media in a media/
directory within my project directory. settings.py features the
following code to resolve MEDIA_ROOT properly:

MEDIA_ROOT = 'media'

import os.path
MEDIA_ROOT = os.path.join(
               os.path.normpath( os.path.dirname(__file__) ),
               MEDIA_ROOT )

My applications start out from a simple structure:

alpha/
 |-- __init__.py
 |-- models.py
 |-- views.py
 |-- urls.py
 |-- media/
 |    `-- ...
 `-- templates/
      `-- ...

This way, I can create symbolic links in the project media/
directory like:

$ ls -l media/
total 0
lrwxrwxrwx 1 arteme users 13 Feb 27 09:34 alpha -> ../alpha/media/

which will make it easy to reference “alpha” application’s media in its
template files as “{{MEDIA_URL}}alpha/...“

Attached is a simple test application that put all these things together.
It features a project called cherrytest with a single application app.
The application has one view servers one HTML file with a 570×562 image
that is split into 35 100×100 tiles. Happy hacking!

• cherrytest.tgz