Itertools & Functools

Difficulty: Advanced

Question

What are the key tools in itertools and functools? When do you use them?

Answer

itertools provides efficient iterator building blocks for looping patterns. All functions return iterators (lazy evaluation).

Key itertools: chain, product, permutations, combinations, groupby, islice, count, cycle, repeat, accumulate, zip_longest.

functools provides higher-order functions and operations on callable objects.

Key functools: lru_cache (memoization), partial (fix some args), reduce, wraps, total_ordering, singledispatch.

These modules are essential for writing clean, efficient, Pythonic code.

Code examples

itertools: Combinatoric Iterators

from itertools import product, permutations, combinations, chain

# product: Cartesian product
colors = ['red', 'blue']
sizes = ['S', 'M', 'L']
variants = list(product(colors, sizes))
print(variants)

# permutations: all orderings
print(list(permutations('ABC', 2)))  # All 2-length orderings

# combinations: unique selections (no repeats, order doesn't matter)
print(list(combinations('ABCD', 2)))

# chain: flatten multiple iterables
list1 = [1, 2, 3]
list2 = [4, 5, 6]
list3 = [7, 8, 9]
flattened = list(chain(list1, list2, list3))
print(flattened)  # [1, 2, 3, 4, 5, 6, 7, 8, 9]

# chain.from_iterable: flatten nested iterables
nested = [[1, 2], [3, 4], [5, 6]]
flat = list(chain.from_iterable(nested))
print(flat)  # [1, 2, 3, 4, 5, 6]

product gives all combinations across iterables. permutations gives ordered arrangements. combinations gives unordered selections. chain flattens multiple iterables.

itertools: Grouping and Slicing

from itertools import groupby, islice, accumulate, zip_longest

# groupby: group consecutive items
data = sorted(['apple', 'avocado', 'banana', 'blueberry', 'cherry'])
for key, group in groupby(data, key=lambda x: x[0]):
    print(f"{key}: {list(group)}")

# islice: slice an iterator
from itertools import count
infinite = count(10, 5)  # 10, 15, 20, 25, ...
first_five = list(islice(infinite, 5))
print(first_five)  # [10, 15, 20, 25, 30]

# accumulate: running totals
nums = [1, 2, 3, 4, 5]
running_sum = list(accumulate(nums))
print(running_sum)  # [1, 3, 6, 10, 15]

import operator
running_product = list(accumulate(nums, operator.mul))
print(running_product)  # [1, 2, 6, 24, 120]

# zip_longest: zip with fill value
names = ['Alice', 'Bob', 'Charlie']
scores = [95, 87]
result = list(zip_longest(names, scores, fillvalue=0))
print(result)  # [('Alice', 95), ('Bob', 87), ('Charlie', 0)]

groupby requires sorted input for correct grouping. islice works with infinite iterators. accumulate computes running aggregations.

functools: Caching and Partials

from functools import lru_cache, partial, reduce, singledispatch

# lru_cache: memoization decorator
@lru_cache(maxsize=128)
def fibonacci(n):
    if n < 2:
        return n
    return fibonacci(n - 1) + fibonacci(n - 2)

print(fibonacci(50))  # 12586269025 (instant with cache)
print(fibonacci.cache_info())
# CacheInfo(hits=48, misses=51, maxsize=128, currsize=51)

# partial: fix some function arguments
def power(base, exponent):
    return base  exponent

square = partial(power, exponent=2)
cube = partial(power, exponent=3)
print(square(5))  # 25
print(cube(5))    # 125

# singledispatch: function overloading by type
@singledispatch
def process(data):
    raise TypeError(f"Unsupported type: {type(data)}")

@process.register(str)
def _(data):
    return f"String: {data.upper()}"

@process.register(list)
def _(data):
    return f"List with {len(data)} items"

@process.register(int)
def _(data):
    return f"Integer: {data * 2}"

print(process("hello"))    # 'String: HELLO'
print(process([1, 2, 3]))  # 'List with 3 items'
print(process(42))         # 'Integer: 84'

lru_cache turns recursive fibonacci from O(2^n) to O(n). partial creates specialized functions. singledispatch enables function overloading by argument type.

Key points

• lru_cache requires hashable arguments - won't work with lists or dicts as parameters
• Use cache (Python 3.9+) instead of lru_cache(maxsize=None) for unlimited caching
• groupby only groups CONSECUTIVE items - sort first if you want all items grouped

Concepts covered

itertools, functools, chain, product, lru_cache, partial