New-Style Classes Thomas Wouters XS4ALL #python

Overview Old situation –types, classes –shape of Python Type unification –subtyping –descriptors and properties –class- and static-methods Metaclasses

Old situation: types Python types implemented in C –C structs represent types –C function pointers define functionality –No real class data –C structs represent objects –All instance data in object structs –'Manual' attribute retrieval –Explicit methods

Old situation: Python classes Implemented in terms of Python types –"bolted on top" –'classes' and 'instances' as distinct types –C function pointers delegated to 'magic' methods to define behaviour –tp_getattro function searches base classes –Implicit methods from Python functions –Class and instance data in '__dict__' attribute

Old situation: limitations Python and C as two distinct worlds No real accessors No class/static methods No immutable classes Simplistic inheritance order Less control over behaviour

Old situation: consequences Interface-based functionality –informal interfaces rather than inheritance Containment rather than inheritance –inheritance is not always the answer Simple inheritance trees Easy to use

Type unification Allow (better) mingling of Python and C types (or classes) –bring C and Python closer __dict__ and inheritance for C types descriptors, properties, class/staticmethods for Python classes –subclassing C types in C –subclassing C types in Python Correct 'warts' in classic classes Generalize special cases

Type unification (2) Metaclasses: a new type of type Explicit base class: object –container of generic functionality Old-style classes for compatibility –hidden metaclass Translation from C function-pointers to Python __methods__ (slots) __dict__ for C types and __slots__ for Python classes

Subclassing Subclass C types from Python in the expected manner: class mylist(list): def __getitem__(self, i): try: return list.__getitem__(self, i) except IndexError: return None Resricted multiple inheritance Not always a good idea!

Subclassing (2) __new__, Python's constructor –called to construct (allocate) the object –static method, called with class as first argument –may return an existing value __slots__: store data almost like C would –no __dict__, less memory consumption

Immutable Python types class tristate(int): __slots__ = [] nstates = 3 def __new__(cls, state=0): state %= cls.nstates return int.__new__(cls, state) def __add__(self, o): return tristate(int.__add__(self, o)) def __sub__(self, other): return self.__add__(-other)

Subclassing C types in C Make sure base type supports subclassing –Py _Check(), Py _CheckExact() –PyMethodDef, PyMemberDef, PyGetSetDef –PyObject_GenericGetAttr as tp_getattro –no type-object hardcoding Subclass's PyType object –leave unchanged behaviour up to base type –set tp_base to base class –call base class's tp_init Provide compatible object struct

Type checking Type-checking is a necessary evil (in C) PyObject_TypeCheck() for inheritance-aware PythonC type check Define Py _Check() in terms of PyObject_TypeCheck() Define Py _CheckExact() as type-pointer comparison Use Py _CheckExact() for internal optimizations

PyList_Check*() #define PyList_Check(op) \ PyObject_TypeCheck(op, \ &PyList_Type) #define PyList_CheckExact(op)\ ((op)->ob_type == \ &PyList_Type)

Py*Def Allow subclasses to extend/override parts PyMemberDef (tp_members) for instance data –maps C structs to Python attributes –tp_members in type struct PyMethodDef (tp_methods) for all methods –wraps C functions in Python objects –specifies argument style and type of method PyGetSetDef (tp_getset) for accessors –maps functions to attributes and vice versa

Subclass struct Include base class struct in subclass struct typedef struct { PyListObject list; PyObject * default; } defaultlistobject; No changes to original memory layout Multiple inheritance is only possible with 'compatible memory layouts' C subclasses subclassable in Python

Method Resolution Order Old MRO not suited to complex inheritance trees –base classes get queried before some of their derived classes –base classes get queried multiple times –No convenient way to access base classes hardcode base class names guess about attributes / methods

New MRO Published algorithm: C3 – ion-oopsla96.htmlhttp:// ion-oopsla96.html Relatively easy to explain –same order as before –eliminates all but the last occurance of the same class Same order for simple inheritance

Old-style MRO: D, B, A, C, A New-style MRO: D, B, C, A (see __mro__) A D(B, C) C(A) B(A)

A D(B, C) C(A) B(A) E(C, B) F(D, E) Old-style MRO: F, D, B, A, C, A, E, C, A, B, A New-style MRO (2.2): F, D, E, B, C, A

super() Proxy'object for accessing 'base' classes Continues MRO where it left off –requires current class and (derived) instance Somewhat inconvenient to use Very important for consistency Use it anyway

super() use class BStore(Storage): def __init__(self, state): Storage.__init__(self, state) class BStore(Storage): def __init__(self, state): super(BStore, self).__init__(state)

Descriptors Generalization of class-getattr magic and C tp_getattr tricks Trigger functioncalls when retrieved or stored from an object (getattr/setattr) –__get__() –__set__() –__delete__()

Properties Accessors for Python An application of descriptors class R(object): def _get_random(self): return random.random() random = property(_get_random) Also hold docstrings for attributes 'set' and 'delete' functions don't work with old- style classes

Caching Property class cachingprop(object): __slots__ = ["_name", "_fget"] def __init__(self, name, fget): self._name = name self._fget = fget def __get__(self, inst, type=None): if inst is None: return self v = self._fget(inst) inst.__dict__[self._name] = v return v

Special method types classmethods –Passes class as implicit first argument –can be called through class or through instance –allow for factory functions (or 'alternate initializers') that create subclasses dict.fromkeys tarfile.TarFile.open

Special Method Types (2) staticmethods –Passes no special arguments –Necessary for object.__new__ (or is it?) –Allows for regular (non-method) Python functions as attributes

Special Method Types (3) class Buffer(object): def __init__(self, data): self.data = data[:] def fromstring(cls, s): return cls(s.splitlines()) fromstring = classmethod(fromstring) def send(self): self._extern_send(self.data) _extern_send = staticmethod(sendmodule.send)

Metaclasses The class of class Usually derives from type Relate to classes like classes relate to instances Define class behaviour Allow for convenient post-processing of classes

Class/instance relation Creating the instance passes the contents (arguments) to the class __init__: class Send(object): def __init__(self, what, who):... Send("my data", him)

Metaclass/class relation class Meta(type): def __init__(self, name, bases, attrs): type.__init__(self, name, bases, attrs) class Impl(base1, base2): __metaclass__ = Meta X = 1 def method(self, it): return not it stat = staticmethod(...)

Metaclasses Behave like classes: –__new__ called for class creation –__init__ called for class initialization –inheritance Mixing metaclasses requires compatibility –derived classes must have same or derived metaclasses –metametaclasses can automatically derive metaclasses

Questions ? Slides will be on