December 06 2008

Can We Have Some Obama Here?

I don't agree with everything President-elect Obama is planning for his economic stimulus package (building new roads and bridges is surely not the most important of America's needs) but I do agree fully and completely with this:

Third, my economic recovery plan will launch the most sweeping effort to modernize and upgrade school buildings that this country has ever seen. We will repair broken schools, make them energy-efficient, and put new computers in our classrooms. Because to help our children compete in a 21st century economy, we need to send them to 21st century schools.

As we renew our schools and highways, we’ll also renew our information superhighway. It is unacceptable that the United States ranks 15th in the world in broadband adoption. Here, in the country that invented the internet, every child should have the chance to get online, and they’ll get that chance when I’m President – because that’s how we’ll strengthen America’s competitiveness in the world. President-elect Barack Obama, December 6, 2008

Python 3 Performance: A Red Herring?

This entry was split from an overly long journal entry entitled Why Python 3?

Inside

Benchmark 1: pystone
Benchmark 2: web app
Benchmark 3: web framework template

I was once guilty of dwelling on raw language performance, but no longer. For the work I do, developer productivity is much more important than raw language speed. It doesn't trouble me at all that Python 3 is a little slower than the most recent 2.x release we have in production (2.5). History has shown the Python developer group are rather good at optimizing Python under the hood, so now that the language implementation has stabilized, we can look forward to plenty of optimizations coming to the 3 branch in time.

Yet Python 3 today is no slouch. For the work I do -- web application development -- I'm finding the first release of Python 3 performs roughly equivalent to Python 2.5.x, and that matters more to me than pystone or some other abstract measure.

Benchmark 1: pystone

Speaking of pystone, here is how this old single processor Unix workstation I'm using stacks up:

$ python2.5 /usr/local/lib/python2.5/test/pystone.py
Pystone(1.1) time for 50000 passes = 1.75781
This machine benchmarks at 28444.4 pystones/second

$ python2.6 /usr/local/lib/python2.6/test/pystone.py
Pystone(1.1) time for 50000 passes = 1.46094
This machine benchmarks at 34224.6 pystones/second

$ python3.0 /usr/local/lib/python3.0/test/pystone.py
Pystone(1.1) time for 50000 passes = 1.85938
This machine benchmarks at 26890.8 pystones/second

Benchmark 2: web app

Here is an obligatory (yet crude) web application benchmark - retrieving a wiki page - on the same old machine while gvim and a bunch of other stuff is also running:

ab -n 1000 http://127.0.0.1:8002/qwiki/foo/

             Req/second
Python 2.5   174.79
Python 2.6   204.29
Python 3.0   173.49

Benchmark 3: web framework template

The following benchmark was based off a templating benchmark included in the Genshi package and utilizes Python's timeit module, averaging 10 passes creating a large html table using the following Qpy template:

# this is to provide a Qpy template for use in comparison
# with the Genshi "bigtable" benchmark.

table = [dict(a=1,b=2,c=3,d=4,e=5,f=6,g=7,h=8,i=9,j=10)
          for x in range(1000)]

def qpy_template:xml():
    '<table>'
    for row in table:
        '<tr>'
        for col in row.values():
            '<td>%s</td>' % col
        '</tr>'
    '</table>'

You can see that Qpy isn't the only package to benefit from the improved performance in Python 2.6 and Python 3.0, and I'm sure we'll see need for speed type coptimization efforts in due course to bring some modules back to where they were:

$ python2.5 bigtable.py
ElementTree                         736.43 ms
cStringIO                            36.46 ms
StringIO                            141.10 ms
list concat                          22.73 ms
Qpy Template                        129.35 ms

$ python2.6 bigtable.py
ElementTree                         656.99 ms
cStringIO                            30.85 ms
StringIO                            131.95 ms
list concat                          16.37 ms
Qpy Template                         39.54 ms

$ python3.0 bigtable.py
ElementTree                         449.60 ms
StringIO                            138.03 ms
list concat                          21.70 ms
Qpy Template                         36.36 ms

Conclusion: There are valid reasons why one might choose Python 2.x today over Python 3.0, but raw performance is not likely to be one of them for many developers.

Python 2 and 3 Co-Existence Advice

David Binger of the MEMS Nanotechnology Exchange writes about their experience migrating their public and private code base such that it co-exists with either Python 2.x or 3.x deployment targets.

The Py3k team has a recommended migration strategy involving the use of their 2to3 conversion package and then maintaining separate releases for the two different flavors of Python. The prospect of maintaining different branches was unattractive, so we decided instead to try adjusting our code so that it would work with both Pythons. As others face this problem, perhaps it is worth describing our experience.

Worth a read. David concludes with "when we do decide to migrate our production systems to Python 3, we think it will be like jumping a creek, not the Snake River Gorge."