mike watkins dot ca : Python 2 and 3: Metaclasses

Python 2 and 3: Metaclasses

Carrying on with Python 2 to 3: Bridging the Gap, in this instalment lets cover one of the potentially thorny language changes in Python 3 - the new syntax for employing metaclasses.

Syntax changes in the language are especially problematic as one or the other version simply won't know how to compile your code. Fortunately there is a clean solution for implementing metaclasses in both 2.x and 3.x, with the same source. If pressed for time, skip ahead for the solution; there is also a source code file attached to this post which encompasses all that is discussed.

I've wrapped this discussion around a working example - Monty Python inspired of course - of a basic metaclass which might be helpful for those who haven't had any exposure to the concept yet.

Defining the metaclass

The definition of a metaclass remains the same in both Python 2 and 3. Here is a possibly topical example (if you like old British humour):

class SillyWalksMetaClass(type):
    """
    A metaclass describing a class that acts as a singleton and provides a
    registry of funny walking styles, something no Monty Python fan should
    be without.
    """

    def __init__(self, class_name, bases, namespace):

        if not hasattr(self, 'walk_types'):
            self.walk_types = []
        else:
            self.walk_types.append(self)

    def __str__(self):
        return self.__name__

    def get_walks(self, *args, **kwargs):
        """Returns a list ordered by the class name"""
        return sorted(self.walk_types, key=lambda x: x.__name__)

All in all fairly straightforward. Due to what we are doing in the constructor method this class will act as a singleton - more on that later. First lets deal with using the metaclass in both 2.x and 3.x versions of Python and then look at the simple cross version solution.

Using the metaclass in Python 2.x

In Python 2.x you base your metaclass defined class by using a special attribute called __metaclass:

class SillyWalk(object):
    """
    Subclasses implement a silly walk of stunning sillyness
    """

    __metaclass__ = SillyWalksMetaClass

Creating a subclass of our new SillyWalk type is simple:

class LurchAndJump(SillyWalk):
    pass

Using the metaclass in Python 3.x

Here's a potential gotchya when migrating code to Python 3 - while the __metaclass__ attribute remains valid, it does absolutely nothing. One assumes your unit tests will have picked up on any future, ahem, oversight, but forewarned is forearmed. In Python 3 we dispense with the magic __metaclass__ attribute and instead the class definition syntax grows a keyword ability:

class SillyWalk(metaclass=SillyWalksMetaClass):
    """
    Subclasses implement a silly walk of stunning sillyness
    """

(Of course you create a subclass in exactly the same way as you would in Python 2.x)

I like this just fine, except for the fact that the Python 2.x compiler will spit up all over you if you have this syntax sprinkled in your hoped-to-be cross compatible application. Because Python 3 introduces a syntax change (one which Python 2 knows nothing about) you can't simply surround the class definition in a if sys.version < '3': block.

But never fear, there has always been an approach and its dead simple.

Using the metaclass in Python 2.x and 3.x

What we need is a method which works for both Python 2 and 3 and the solution is:

SillyWalk = SillyWalksMetaClass('SillyWalk', (object, ), {})

# That's it. Really.

Yes, that's all there is to it. Want to know more? I've included a bibliography - in particular have a look at Michele Simionato's recent articles. I wish I had as I was puzzling this out, as he provides the answer (without highlighting it as a cross-version solution mind you) in the first several paragraphs of part 1.

Digging a little deeper we see the answer was always hiding in front of us, as is frequently the case. Fire up your interpreters!

% python3.0
>>> help(type)
Help on class type in module builtins:

class type(object)
 |  type(object) -> the object's type
 |  type(name, bases, dict) -> a new type
 | ...

The answer was there all along: type(name, bases, dict) -> a new type which is exactly what we want out of our SillyWalksMetaClass which, by the way, has the following properties:

  1. It acts as a singleton
  2. It provides a 'registry' of sorts - you can imagine building a plugin registry or some such thing with a similar object.

Lets see SillyWalks in all its glory:

SillyWalk = SillyWalksMetaClass('SillyWalk', (object, ), {})

# As you see, it was only a flesh wound. We have a simple
# and clean approach for metaclasses for both Python 2 and
# 3.

# Subclassing remains the same for Python 2.x and Python 3+,
# lets create four favorite silly walks:

class Skip(SillyWalk):
    pass

class LurchAndSkip(SillyWalk):
    pass

class CleeseSpecial(SillyWalk):
    'The quintessential silly walk'

class HopWeaveLurchShudder(SillyWalk):
    pass

# got all four?
assert len(SillyWalk.get_walks()) == 4
# right then, lets have a look
print(', '.join([str(walk) for walk in SillyWalk.get_walks()]))
# Being a singleton, you'd expect the following to be identical:
print(', '.join([str(walk) for walk in LurchAndSkip.get_walks()]))
# and it is...
assert SillyWalk.get_walks() == HopWeaveLurchShudder.get_walks()

Ok, Prove it

$ python3.0 meta2and3.py
CleeseSpecial, HopWeaveLurchShudder, LurchAndSkip, Skip
$ python2.6 meta2and3.py
CleeseSpecial, HopWeaveLurchShudder, LurchAndSkip, Skip
$ python2.5 meta2and3.py
CleeseSpecial, HopWeaveLurchShudder, LurchAndSkip, Skip

Further reading on metaclasses

And now for something completely different

http://64.21.147.48/tv-20081128-204328.gif

Click for a classic, very silly, youtube video. Go on, you know you want to!

meta2and3.py (2714 bytes, text/plain)
A demonstration of metaclass usage as well as a practical solution for using metaclasses in both Python 2 and Python 3 with the same source code.