Descriptors Protocol in Depth

Difficulty: Advanced

Question

How does the Python descriptor protocol work? What is the difference between data and non-data descriptors?

Answer

A descriptor is an object that implements at least one of: __get__, __set__, or __delete__.

Data descriptor: implements __set__ (and usually __get__). Takes priority over instance __dict__. Examples: property, classmethod, staticmethod.

Non-data descriptor: implements only __get__. Instance __dict__ takes priority over it. Examples: functions (that is how method binding works).

Lookup order for obj.attr: 1. type(obj).__mro__ data descriptors 2. obj.__dict__ 3. type(obj).__mro__ non-data descriptors

This is how functions become bound methods: they are non-data descriptors with a __get__ that returns a bound method.

Code examples

Functions are Non-Data Descriptors

# This explains how methods work
class MyClass:
    def greet(self):
        return f'Hello from {self}'

obj = MyClass()

# greet is a function object (descriptor)
print(type(MyClass.__dict__['greet']))  # <class 'function'>

# Accessing via instance triggers __get__ -> bound method
print(type(obj.greet))   # <class 'method'>
print(obj.greet())        # Hello from <MyClass object>

# Manually: function.__get__(instance, class) returns bound method
bm = MyClass.__dict__['greet'].__get__(obj, MyClass)
print(bm())   # Same as obj.greet()

# Non-data descriptor: instance dict takes priority
class NonData:
    def __get__(self, obj, objtype=None):
        if obj is None: return self
        return 'from descriptor'

class Test:
    attr = NonData()

t = Test()
print(t.attr)           # from descriptor
t.__dict__['attr'] = 'from instance'  # Override!
print(t.attr)           # from instance (instance dict wins)

Functions implement __get__ to return bound methods - this is the descriptor protocol in action. Non-data descriptors (only __get__) can be shadowed by instance dict.

Data Descriptor: Lazy Property

class cached_property:
    """A data descriptor that computes and caches a property once."""

    def __init__(self, func):
        self.func = func
        self.attr = f'_cache_{func.__name__}'

    def __set_name__(self, owner, name):
        self.attr = f'_cache_{name}'

    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        if not hasattr(obj, self.attr):
            print(f'Computing {self.func.__name__}...')
            setattr(obj, self.attr, self.func(obj))
        return getattr(obj, self.attr)

    def __set__(self, obj, value):
        setattr(obj, self.attr, value)

class DataAnalyzer:
    def __init__(self, data):
        self.data = data

    @cached_property
    def statistics(self):
        import statistics
        return {
            'mean': statistics.mean(self.data),
            'stdev': statistics.stdev(self.data)
        }

da = DataAnalyzer([1, 2, 3, 4, 5])
print(da.statistics)  # Computing... (first time)
print(da.statistics)  # Cached (no recompute)

cached_property computes on first access, then stores result in instance __dict__. Python 3.8+ has functools.cached_property built-in.

Key points

• functools.cached_property is the built-in cached property descriptor (Python 3.8+)
• Data descriptors (with __set__) cannot be overridden by instance dict
• All Python classmethods, staticmethods, and properties are descriptors
• The lookup chain: data descriptors > instance dict > non-data descriptors

Concepts covered

Descriptor, __get__, __set__, __delete__, Data Descriptor, Non-data Descriptor