Lesson plan

Image processing on a thermal camera lesson plan

Vision from raw sensor words on an Uno with 2 KB of RAM: images as grids, interpolation streamed a pixel at a time, thresholds, blobs, edges and tracking. Ends with a crew counter for a spaceship airlock.

4 chapters, 7 lessons, about 6 hours, on the Arduino Uno R3. Every lesson runs in the browser, with nothing to install.

The unit

Objectives and materials

By the end, students can

  • An 8x8 frame in 128 bytes, then smooth it without the RAM to hold it.
  • Thresholds that follow the room, then blobs you can count.
  • Where the picture changes, then who went where.
  • Count the crew in and out, and know when everyone is aboard.

Materials for the whole unit

  • Arduino Uno R3
  • Full-size breadboard
  • TFT display, 1.8 inch ST7735
  • Thermal camera, AMG8833

Every part is already on the simulator’s bench, so there is nothing to buy or hand out. The counts are the most any one lesson uses, if you build it on real breadboards too.

Chapter 1. About 1 h 50 min

Pictures made of numbers

An 8x8 frame in 128 bytes, then smooth it without the RAM to hold it.

LessonTimeWhat students do
Images as gridsA thermal camera sees sixty-four temperatures. Store them as quarter degrees in 128 bytes, find a pixel by row and column, shade the grid as text art, move the warm crew member, and mirror the picture.55 min
  1. Run it: the airlock camera
  2. Row and column from one number
  3. Quarter degrees, not floats
  4. Text art: shade the grid
  5. Move the crew member
  6. Mirror it
InterpolationTurn the blocky 8x8 into a smooth 29x29 with bilinear interpolation in whole numbers, streamed a pixel at a time because the big picture would not fit in 2 KB, then compare nearest-pixel and change the scale.55 min
  1. Run it: blocky to smooth
  2. Bilinear: blend the four around it
  3. Why stream: 2 KB
  4. Paid for in time
  5. Nearest pixel, for contrast
  6. Change the scale

Materials

  • Arduino Uno R3
  • Full-size breadboard
  • TFT display, 1.8 inch ST7735
  • Thermal camera, AMG8833

Check for understanding

Each student finishes "Interpolation" on their own, without opening the hints. The simulator checks 6 of its 6 steps as they go.

Then ask each one to explain, in a sentence or two, the last thing it asked of them: "Change the scale".

Chapter 2. About 1 h 30 min

Find the crew

Thresholds that follow the room, then blobs you can count.

LessonTimeWhat students do
ThresholdsSplit the frame into crew and background: a fixed threshold, why it breaks when the airlock warms up, a threshold that follows the room, and a difference test that also finds a crew member in a cold suit.45 min
  1. Run it: a mask
  2. Lower the bar
  3. The airlock warms up
  4. A threshold that follows the room
  5. A crew member in a cold suit
BlobsCount the crew in a mask with connected-component labeling: a flood fill with its own 64-byte stack, a test frame with a known answer, 4 against 8 neighbors, a size filter for noise, and a blob cut off by the edge.45 min
  1. Run it: test the labeler first
  2. Flood fill with its own stack
  3. 4 neighbors or 8
  4. Throw away the specks
  5. Half out of the frame

Materials

  • Arduino Uno R3
  • Full-size breadboard
  • TFT display, 1.8 inch ST7735
  • Thermal camera, AMG8833

Check for understanding

Each student finishes "Blobs" on their own, without opening the hints. The simulator checks 5 of its 5 steps as they go.

Then ask each one to explain, in a sentence or two, the last thing it asked of them: "Half out of the frame".

Chapter 3. About 1 h 50 min

Edges and motion

Where the picture changes, then who went where.

LessonTimeWhat students do
EdgesWhere the temperature changes: the Sobel filter in whole numbers, why the hottest pixel has no edge, why edges ignore the room temperature, an outline threshold, and a better cheap stand-in for the square root.55 min
  1. Run it: a ring of edges
  2. The Sobel kernels
  3. The hottest pixel has no edge
  4. Edges ignore the room
  5. An outline
  6. A better square root
TrackingFollow each crew member from frame to frame: blob centers matched to tracks by distance, a direction from the motion, a gate that refuses a teleport, and patience for someone who drops out of view.55 min
  1. Run it: a track starts
  2. Walk them across
  3. And back
  4. The gate: no teleporting
  5. Out of view, back again
  6. Give the track some patience

Materials

  • Arduino Uno R3
  • Full-size breadboard
  • TFT display, 1.8 inch ST7735
  • Thermal camera, AMG8833

Check for understanding

Each student finishes "Tracking" on their own, without opening the hints. The simulator checks 6 of its 6 steps as they go.

Then ask each one to explain, in a sentence or two, the last thing it asked of them: "Give the track some patience".

Chapter 4. About 55 min

Boss: the airlock counter

Count the crew in and out, and know when everyone is aboard.

LessonTimeWhat students do
Boss: a person counterThe airlock crew counter: thresholds, blobs and tracks count crew in and out across a line, a dead band stops a hesitant crew member being counted three times, and the hatch knows when everyone is aboard.55 min
  1. Run it: the counter
  2. One crew member comes aboard
  3. In, then back out
  4. The hesitant crew member
  5. A dead band
  6. Close the hatch

Materials

  • Arduino Uno R3
  • Full-size breadboard
  • TFT display, 1.8 inch ST7735
  • Thermal camera, AMG8833

Check for understanding

Each student finishes "Boss: a person counter" on their own, without opening the hints. The simulator checks 6 of its 6 steps as they go.

Then ask each one to explain, in a sentence or two, the last thing it asked of them: "Close the hatch".

Your class

Teach this unit as a class

Start a class from this course and its 7 lessons are assigned in one go, in order. Students join with a six-character code or printed login cards, and each one’s work comes back to you.

The class tools are free for 30 days for one class of up to 20 students, with no card. The lessons themselves are free either way.