How it works
Image processing on a thermal camera runs every lesson on the gallery's thermal camera: an Arduino Uno, an AMG8833 Grid-EYE on I2C that sees an 8x8 grid of temperatures ten times a second, and an ST7735 color screen. The camera part shows its scene above the lens: click a square to move the warm spot (a crew member), click it again to take it away, and drag the sliders for the room's and the person's temperatures.
Every lesson opens a working sketch. You run it, read what the serial monitor prints (the frame as numbers, text art, masks and labels), then change a line and run again. The checks read the serial output, so every number a lesson quotes is one the board really printed.
The lessons
- Images as grids: sixty-four temperatures kept as quarter degrees in 128 bytes, index against row and column, contrast-stretched text art, moving the crew member, and mirroring the picture with index arithmetic.
- Interpolation: bilinear blending in whole numbers from 8x8 up to 29x29, streamed a pixel at a time because the 1682-byte picture would not fit, the paint time, nearest-pixel for contrast, and changing the scale.
- Thresholds: a mask of crew and background, a fixed threshold that breaks when the airlock warms up, an adaptive one that follows the room's mean, and a difference test that also finds a crew member in a cold suit.
- Blobs: connected-component labeling with a flood fill that keeps its own 64-byte stack instead of recursing, a test frame with a known answer, 4 against 8 neighbors, a size filter for specks, and a blob cut off by the edge of the view.
- Edges: the Sobel filter, why the hottest pixel has no edge, why edges ignore the room temperature, an outline threshold, and max plus half the min as a cheap stand-in for a square root.
- Tracking: blob centers matched to tracks by distance, a direction from the motion, a gate that refuses a teleport, and patience for someone who drops out of view.
The boss: an airlock crew counter
The last lesson puts the whole pipeline together. The camera looks down on the airlock, space on the left and the ship on the right, and counts each track that crosses the line down the middle: in one way, out the other. A crew member who dithers on the line gets counted three times, so you add a dead band (hysteresis) and they count once. Then the counter tells the ship when everyone on the roster is aboard and the hatch can close. It ends with ideas to make it yours: two lines for a corridor, per-direction totals on the screen, or a warning when someone stays in the airlock too long.
What the Uno does differently
- 2 KB of RAM is the whole budget: variables, library buffers and the stack.
One 8x8 frame of
int16_tis 128 bytes; a 29x29 interpolated picture would be 1682, so the course streams it instead of storing it. - There is no float hardware, so temperatures stay in the sensor's own unit, quarter degrees, and positions in hundredths of a pixel. Floats only appear as text for people to read.
- Recursion is dangerous on a 2 KB stack, so the flood fill keeps its own small stack of pixels to visit.
- The screen is the slow part: a full 29x29 paint takes about a second, so the sketches repaint only what changed.
- The gallery circuit is an Uno with an ST7735. Faster chips with more RAM (an ESP32 and a bigger ILI9341 screen, say) run the same pipeline with room to spare; the ideas do not change.
- The simulated camera has no frame-to-frame noise. A real AMG8833 wobbles by about a degree, which is what the size filter, the gate and the dead band are for.
After this course
Next is Machine learning on a microcontroller, where features like blob size and position become inputs to a model, or Fixed-point and integer math for the number tricks this course leans on.