Object Publishing on the Web - Quixote

From a user's perspective This weekend it seems to be de rigueur to comment on web object publishing, inspired by Ben Bangert's article on 'best of breed' controllers for MVC web frameworks.

A long discussion about Zope and its ancestors ensued in the comments - Quixote borrows the key idea from Zope - simple object publishing via the web, and, in my opinion, succeeds because it kept it simple.

I'm taking a very quick stab at explaining how the Quixote web application development framework hangs together, from a mere users perspective (me) rather than a web framework developers perspective.

There are a number of other Quixote tutorials and documentation out there; in this effort I'm aiming to (but may miss completely) explain some Quixote basics, and perhaps impact enough information such that a prospective web framework shopper will themselves be able to put Quixote in context with some of the other web frameworks.

Knowing me, very quick will soon turn into long and convoluted, although as kids soccer is fast approaching on the clock, perhaps the time constraint will work to good effect. Nevertheless, expect revisions!

On Groovie, Ben takes a look at Best of breed Controllers for MVC web frameworks - its a worthwhile read that dives into some of the differences between the currently hot names, although I am surprised not to at least see a mention of Quixote.

I don't believe the good folks that wrote Quixote and its related tools are much into the self-promotion (some articles), but they certainly have written some fine web development software and it deserves a little talking up now and then. Perhaps other Quixote users like myself will jump into the fray.

Now at version 2.2, Quixote continues to evolve and progress, off a very fine base. Its in production use across a wide array of web platforms ranging from simple Python servers, to python Medusa, mod_python, AOL server, Apache, lighttpd, twisted and no doubt more. I use Apache (but perhaps soon lighttpd) in conjunction with scgi, which is a replacement for FastCGI that works particularly well. lighttpd supports scgi natively now too.

Carrying on from Ben's article, I'm at a loss to plug Quixote's object publishing method into his categories. In Quixote, objects are accessible according to URL (Ben calls this implicit mapping) but the application developer must take this into account when designing the UI (Ben calls this programmatic mapping). What app doesn't, I wonder?

I suspect most frameworks that allow something like http://someserver.net/calendar/new resolve to calling the new() method of a class called calendar use some sort of rule set (whether its by convention or explicit) to determine what gets called where.

Quixote's design allows your UI to naturally follow the url from root to end:

/ (root) will call
    calendar which will have one or more methods such as
        _index
        new
        edit
        delete

In reality, your typical Quixote application will have object classes and separate, but related, object UI classes, looking rather like this:

 / (root UI class) will call
    CalendarUI which will have one or more methods such as
        _index
        new         ... each of which will act upon a Calendar object)
        edit        ... and perhaps this also dives deeper and... deeper...
            CalendarEditUI
                add_resource
                book_resource
        delete      ...

We'll look later at how the hierarchy of UI classes resolves as you drive down, but lets first start off with a basic project skeleton - there is nothing written in stone in this regard; perhaps it might be useful if the Quixote package included a project skeleton builder.

Quixote encourages, but does not enforce, separation between objects and UI. A typical project might look like this:

myapp/
    bin/
        runapp.py      (this is your executable that runs the app)
    doc/
    obj/
        calendar.py
        contact.py
        resource.py
    ui/
        calendar.ptl*  (UI classes for object)
        contact.ptl    (UI classes for object)
        home.ptl       (perhaps an index class or function for home page
                        content)
        publisher.py   (typically a subclass of Publisher for your app)
        resource.ptl   (UI classes for object)
        root.ptl       (or call it driver.ptl or what turns your crank:
                        this defines the root web interface from which
                        all things flow. Q-fans might call theirs qslash.ptl)
        util.ptl       (UI helper classes and functions used elsewhere)
    __init__.py
    date.py            (various helper bits used by both obj and ui)
    utils.py

*don't worry about these ptl extensions for now. They more or less contain straight python code, no need to learn anything new here... move along.

We shall have a look at a Quixote application from the ground (from the driver or 'root' interface) on up (or down depending on how you view these things. Actually we'll start one level below the driver - every driver needs a car (or better yet, a bicycle!) - and in this case our car is a Publisher, which is the interface between your web server and your web application.

I'll use a simple python-based HTTP server for my example, but this is principally the same whether one uses Apache, Medusa, lighttpd, or AOL server, only the app to server interfaces change. In this case my app script is running both the web server (simple_server) and the application itself (create_publisher). bin/runapp.py:

from typicalapp.publisher import create_publisher
if __name__ == '__main__':
    from quixote.server.simple_server import run
    print 'creating demo listening on http://localhost:8080/'
    run(create_publisher, host='localhost', port=8080)

Ok, that's pretty straight forward. Now on to the heart of the matter, publisher.py:

from quixote.publish import Publisher
from typicalapp.ui.root import RootDirectory

def create_publisher():
    return Publisher(RootDirectory(),
                     display_exceptions='plain')

Of course a real application might subclass Publisher to do all sorts of other things, including registering a session handler, filtering output through HTMLTidy, custom logging, whatever...

Next, what are we publishing? Why, RootDirectory of course - from there all things flow. First, here's prototypical the web hello, world example. We are going to look at two different examples of this and explain why there are two different ways of pumping HTML out from your Quixote application. First - the base case:

class RootDirectory(Directory):

    _q_exports = ['', 'hello']

    def _q_index(self):
        return '''<html>
                    <body>Welcome to the Quixote demo.  Here is a
                    <a href="hello">link</a>.
                    </body>
                  </html>
                '''

    def hello(self):
        return '<html><body>Hello world!</body></html>'

    def goodbye(self):
        return '<html><body>Cheers! Salut!</body></html>'

Pretty straightforward, no?

Hey, guess what, there really is no step one. You don't have to design some complicated regex just to get to the root, or to any object no matter how complex your web applications url hierarchy may get to. To the regex challenged, this might be reason enough to consider Quixote!

The URL your client/user is resolved from start towards its end, from 'root' on down. This a call ro http://sometypicalserver.net/ will have Quixote dish up whatever has been defined as its root UI. In most cases this will be a class with one or more methods.

Ok, there are two things happening here that are not immediately obvious - what are _q_index and _q_exports? Before going there, let me interject: If there is one basic principle in Quixote its *there shall be no magic*. To a large extent this is true, there is no magic, however it wouldn't be a framework without a few basic rules and conventions.

_q_exports is a list containing all methods exposed by the class to the web. A '' name in exports allows the _q_index function to be exposed as a callable. Extra work? Perhaps, but the rule is be explicit in Quixote-land. This is a basic defense against exposing things that you don't want to, forcing the developer to think about what will be shown the world. This list can be dynamic of course, but more on that at another time.

When Quixote gets to the requested object along the url requested, it expects to find a callable which returns a result; in our example about the callable _q_index() is returning some simple html.

If the client had requested http://sometypicalserver.net/hello/, Quixote would resolve or dispatch the request to the hello method of RootDirectory. Had the client requested http://sometypicalserver.net/goodbye/, Quixote will return a 404 error, as the goodbye method was not listed in _q_exports. Clear as mud? Excellent, lets press on.

Wait! Before going on, lets look at one minor tweak to RootDirectory and introduce the concept of Quixote's Python Template Language (PTL), which is to traditional templating languages as the un-Cola is to regular colas. It spits out html, but it doesn't look like the other stuff you drink.

PTL is a hook to Python that allows you to write Python functions that integrate textual output more easily. An example will help. First, we have to enable PTL in our publisher script before PTL modules or functions can be called. That's easy, just add to runnapp.py:

from quixote import enable_ptl
enable_ptl()

enable_ptl() is called before create_pubilsher is called. Now we can re-write our prototypical hello, world RootDirectory class as such:

class RootDirectory(Directory):

    _q_exports = ['', 'hello']

    def _q_index [html](self):
        '''
        <html>
            <body>Welcome to the Quixote demo.  Here is a
            <a href="hello">link</a>.
            </body>
        </html>
        '''

    def hello [html](self):
        '''
        <html>
            <body>
            <ol>
        '''
        for n in range(1,101):
            '<li>Hello world, %d times!</li>\n' % n
        '''</ol>
            </body>
        </html>
        '''
# *Note*: you don't HAVE to use ptl extensions for all UI functions...
    def goodbye(self):
        return '<html><body>Cheers! Salut!</body></html>'

Look closely or you'll miss the difference, its pretty slight.

Doesn't look like much of a change, does it? On the surface, not much has changed other than the introduction of a [html] decorator-like extension to python, and the ability to intersperse text in among python code. This latter ability becomes very handy when your "template" is computationally complex, and allows you to lever what you already know - Python - rather than struggle with learning a new templating language and possibly force-fit your needs into whatever constraints the template language demands.

You do not need to use PTL! Many Quixote developers use other templating packages including Cheetah, Kid, or their own. Andrew Kuchling once described PTL in a whitepaper:

Without us ever realizing this while designing it, PTL turns the idea of HTML templating on its head: the default is program code, and there's an escape sequence into HTML. The escape sequence is just Python's notation for string literals, which is a compact and easily readable notation. Functions that actually contain HTML therefore can look a little messy, but most HTML only appears e lowest-level functions; the bulk of our PTL code simply calls other PTL templates and inserts the odd <br> or <table> here and there.

Greg Ward also discussed PTL in a comprehensive article on Quixote in Linux Journal. Other links to articles on Quixote can be found on the project web site.

PTL is not web designer friendly, but it is very programmer friendly. I'm not a web designer but have created some half decent xhtml/css based designs - my workflow is to prototype the design in straight xthml and css, and then decide how I want to chunk same up within my application. I don't find this to be problematic for the overall design of a site or web application.

Bits and pieces within the application I rarely find the need to prototype in xhtml first; generally I find I spend more time refactoring ptl code into smaller reusable widgets. Your mileage may vary, but for myself, I find it now hurts my eyes to look at html peppered with %this%, %some for loop that%. I'd rather look at plain python. Oh, by the way, vim has a ptl syntax add-on too.

PTL also delivers other benefits, chiefly automatic escaping of strings that are not explicitly safe. Read the project documentation on PTL for further information.

In installment two we'll look at how Quixote traverses a hierarchy of UI objects.