Lessons and the learning game

Iterators and functional Python

Comprehensions, iterators, generators, map, filter, itertools and functools on a Pi, ending in a streaming pipeline from a knob to an LED.

The course

Iterators and functional Python is an intermediate Coding course in seven lessons. The first six run on a Raspberry Pi 4 with two LEDs (red on GPIO17, yellow on GPIO27); the finale uses the gallery's MCP3008 knob circuit, which is built on a Pi 5 (the same Python runs on both). Each step is checked against what the Pi prints or what its pins do.

  1. Comprehensions: list comprehensions over sonar pings, the if-else expression inside one, dict and set comprehensions over the planets, nested comprehensions that flatten and transpose a grid, and generator expressions with sum and any lighting a deep water alarm.
  2. Iterators: iter and next, a for loop written out with StopIteration, why zip and map work only once, a countdown class with __iter__ and __next__, iterables that start over, and the countdown flashing an LED.
  3. Generators in depth: lazy start, endless generators, send for a running average, yield from with a return value, close and finally, and a pipeline that shows when each value moves.
  4. map and filter: functions as values, lambda, map over two lists, a dict of converters, key functions for sorted and max, and a lazy filter that blinks once per hot reading.
  5. itertools: count and islice, cycle for a watch rota, accumulate and pairwise, groupby and why it needs sorted data, combinations and product, and chain, takewhile and dropwhile.
  6. functools: reduce with operator, partial for an ADC conversion, lru_cache taking 21,891 calls down to 21, decorators, and a compose of your own.
  7. Project: a streaming sensor pipeline: the finale.

The finale

A knob on channel 0 of an MCP3008 (a 10-bit ADC on SPI) becomes an endless generator of readings. Stages are functions that take a stream and return one: volts (map with partial), a moving average over a deque of four, and a filter that passes only moves of 0.2 V or more. The sink sets a PWMLED on GPIO18 to the voltage, so the green LED follows the knob. The last step turns the nested calls into a list of stages folded together with reduce, and stops after five changes. Nothing in the pipeline ever holds more than four readings.

What the Pi's engine does differently

The Pi runs MicroPython 1.29 with a CPython-style standard library. Checked while writing the course:

  • itertools has every function CPython's has, written as Python generators, so it is slower. Its islice reads one value more from its source than it hands out before it stops (CPython's stops at the last one), so the finale stops its live stream with break, which works the same everywhere.
  • functools has reduce, partial, lru_cache, cache, cached_property, total_ordering, cmp_to_key and wraps. There is no singledispatch; a dict from type to function does the job.
  • wraps is there but copies nothing: MicroPython functions cannot hold attributes, so a wrapped function's __name__ stays wrapper. A class-based decorator keeps the name, and the course shows one.
  • iter(function, stop_value) is missing; a while True loop with next and StopIteration does the same.
  • A map, filter or generator prints differently from CPython (<map> rather than <map object at ...>), and a dict does not keep the order keys went in, so the lessons sort keys or read them by name.