Python Data Model and dunder Methods

Difficulty: Advanced

Question

What is the Python data model? How do dunder methods enable Pythonic class design?

Answer

The Python data model is the set of interfaces (dunder methods) that classes can implement to integrate with the language and built-in functions.

By implementing dunder methods, your class: - Works with len(), iter(), in, +, -, *, [], ==, <, etc. - Prints informatively with repr()/str() - Integrates with context managers (with) - Works with async for, await

Key dunder groups: - Representation: __repr__, __str__, __format__ - Comparison: __eq__, __lt__, __le__, __gt__, __ge__, __hash__ - Arithmetic: __add__, __radd__, __iadd__, __mul__, etc. - Container: __len__, __getitem__, __setitem__, __contains__, __iter__ - Context Manager: __enter__, __exit__ - Callable: __call__

Code examples

Custom Container Class

from collections.abc import MutableSequence

class FixedList:
    """A list with a maximum size."""

    def __init__(self, max_size, iterable=()):
        self._max = max_size
        self._data = list(iterable)
        if len(self._data) > max_size:
            raise ValueError(f'Too many items (max {max_size})')

    def __repr__(self):
        return f'FixedList({self._max}, {self._data!r})'

    def __str__(self):
        return str(self._data)

    def __len__(self):
        return len(self._data)

    def __getitem__(self, index):
        return self._data[index]

    def __setitem__(self, index, value):
        self._data[index] = value

    def __contains__(self, item):
        return item in self._data

    def __iter__(self):
        return iter(self._data)

    def append(self, item):
        if len(self._data) >= self._max:
            raise OverflowError(f'List is full (max {self._max})')
        self._data.append(item)

fl = FixedList(3, [1, 2])
fl.append(3)
print(repr(fl))           # FixedList(3, [1, 2, 3])
print(len(fl))            # 3
print(2 in fl)            # True
print([x * 2 for x in fl])  # [2, 4, 6]

__len__, __getitem__, __iter__, __contains__ make the class a proper sequence. for loops, list comprehensions, and in all work because of these dunder methods.

Arithmetic and Callable

class Vector:
    def __init__(self, x, y):
        self.x, self.y = x, y

    def __repr__(self):
        return f'Vector({self.x}, {self.y})'

    def __add__(self, other):
        return Vector(self.x + other.x, self.y + other.y)

    def __radd__(self, other):  # other + self
        if other == 0:
            return self
        return self.__add__(other)

    def __mul__(self, scalar):  # vec * 3
        return Vector(self.x * scalar, self.y * scalar)

    def __rmul__(self, scalar):  # 3 * vec
        return self.__mul__(scalar)

    def __abs__(self):
        return (self.x2 + self.y2)  0.5

    def __bool__(self):
        return bool(self.x or self.y)

    def __eq__(self, other):
        return self.x == other.x and self.y == other.y

v1 = Vector(1, 2)
v2 = Vector(3, 4)

print(v1 + v2)    # Vector(4, 6)
print(v1 * 3)     # Vector(3, 6)
print(3 * v1)     # Vector(3, 6)
print(abs(v2))    # 5.0
print(bool(Vector(0, 0)))  # False
print(sum([v1, v2], Vector(0,0)))  # Vector(4, 6)

__radd__ enables sum() which starts with 0 + v1. __mul__ and __rmul__ enable both vec*3 and 3*vec. __bool__ controls truthiness.

Key points

Prefer __repr__ over __str__ - repr() should return a valid Python expression if possible
__hash__ must be consistent with __eq__: a == b implies hash(a) == hash(b)
If you define __eq__, Python sets __hash__ to None (making instances unhashable)
Use functools.total_ordering to generate all comparison methods from just __eq__ and __lt__

Concepts covered

Data Model, Dunder Methods, Protocol, Operator Overloading, Custom Containers

Python Data Model and __dunder__ Methods