Iterators Protocol

Difficulty: Advanced

The iterator protocol is a fundamental concept in Python that defines how objects support iteration. Any object that implements the __iter__() and __next__() methods follows the iterator protocol and can be used in for loops, comprehensions, and anywhere an iterable is expected.

An iterable is any object that implements __iter__() and returns an iterator. An iterator is an object that implements both __iter__() (returning itself) and __next__() (returning the next value or raising StopIteration when exhausted). Lists, tuples, dicts, and strings are all iterables -- calling iter() on them returns an iterator.

To create a custom iterator, you define a class with __iter__() and __next__() methods. The __iter__ method returns the iterator object (usually self), and __next__ returns the next value in the sequence. When there are no more values, __next__ must raise StopIteration to signal that iteration is complete.

Understanding the iterator protocol clarifies how Python's for loop works under the hood. When you write `for item in obj`, Python calls iter(obj) to get an iterator, then repeatedly calls next() on that iterator until StopIteration is raised. This mechanism powers list comprehensions, unpacking, and all other iteration contexts.

Custom iterators give you fine-grained control over iteration behavior. Unlike generators (which are simpler to write), class-based iterators can maintain complex state, support reset operations, and implement additional methods. They're essential when you need reusable, configurable iteration objects.

Code examples

How for Loops Use the Iterator Protocol

numbers = [10, 20, 30]
iterator = iter(numbers)

print(next(iterator))
print(next(iterator))
print(next(iterator))

iter() calls __iter__ on the list to get an iterator. Each next() call invokes __next__ on the iterator, returning values one at a time.

Custom Iterator Class

class CountDown:
    def __init__(self, start):
        self.current = start

    def __iter__(self):
        return self

    def __next__(self):
        if self.current <= 0:
            raise StopIteration
        value = self.current
        self.current -= 1
        return value

for num in CountDown(5):
    print(num)

CountDown implements __iter__ (returns self) and __next__ (returns current value and decrements). The for loop calls these automatically and stops when StopIteration is raised.

Iterable vs Iterator

class EvenNumbers:
    def __init__(self, limit):
        self.limit = limit

    def __iter__(self):
        self.current = 0
        return self

    def __next__(self):
        if self.current >= self.limit:
            raise StopIteration
        value = self.current
        self.current += 2
        return value

evens = EvenNumbers(10)
print(list(evens))
print(list(evens))

Because __iter__ resets self.current to 0 each time, this object can be iterated multiple times. This is a common pattern for making reusable iterables.

Manual Iteration with StopIteration Handling

class TwoItems:
    def __init__(self, a, b):
        self.items = [a, b]
        self.index = 0

    def __iter__(self):
        return self

    def __next__(self):
        if self.index >= len(self.items):
            raise StopIteration
        value = self.items[self.index]
        self.index += 1
        return value

it = TwoItems("hello", "world")
print(next(it))
print(next(it))
try:
    next(it)
except StopIteration:
    print("No more items")

After yielding both items, __next__ raises StopIteration. We catch it to demonstrate what happens when a for loop encounters the end of an iterator.

Key points

• Every iterator is an iterable (has __iter__), but not every iterable is an iterator (lists have __iter__ but not __next__).
• Calling iter() on a list returns a list_iterator object -- the list itself is not the iterator.
• Resetting iteration state inside __iter__ (like self.current = 0) allows the object to be iterated multiple times.

Concepts covered

__iter__, __next__, custom iterators, StopIteration