Build a database from scratch lesson plan
Write a real storage engine in plain Python on a Raspberry Pi: records on disk, an append-only log, an index, queries, compaction and crash safety, then run it as a Mars weather station's history store on a live sensor.
Objectives and materials
By the end, students can
- Records as bytes, and a log that only ever grows.
- An index of byte offsets, then real queries over it.
- Take the space back, and survive the power dying at the worst moment.
- SolDB on a live sensor: logging, queries, crashes and retention.
Materials for the whole unit
- Eight-channel ADC, MCP3008
- Half breadboard
- LED x 2
- Potentiometer
- Raspberry Pi 4 Model B
- Raspberry Pi 5
- Resistor x 2
Every part is already on the simulator’s bench, so there is nothing to buy or hand out. The counts are the most any one lesson uses, if you build it on real breadboards too.
Storage
Records as bytes, and a log that only ever grows.
| Lesson | Time | What students do |
|---|---|---|
| Records on diskStart SolDB, the history store of a Mars weather station: turn readings into bytes and back, find out why fixed-size records let you jump straight to one, pack them in binary with struct, and see what this Pi does with files between Runs. | 45 min |
|
| Append-only logTurn SolDB into a key-value store whose only write is "add a line at the end": updates as new records, deletes as tombstones, the cost of reading it back, and the free history an append-only log gives you. | 45 min |
|
Materials
- Half breadboard
- LED x 2
- Raspberry Pi 4 Model B
- Resistor x 2
Check for understanding
Each student finishes "Append-only log" on their own, without opening the hints. The simulator checks 5 of its 5 steps as they go.
Then ask each one to explain, in a sentence or two, the last thing it asked of them: "History for free".
Finding things
An index of byte offsets, then real queries over it.
| Lesson | Time | What students do |
|---|---|---|
| An indexMake reads as cheap as writes: a hash index that maps each key to the byte offset of its newest record, rebuilt from the log at startup the way Bitcask does it, and the dead records it leaves behind. | 35 min |
|
| QueriesAsk SolDB real questions: a full scan with a filter, range queries with sorted keys and binary search, why zero padding makes text sort like numbers, aggregates over a range, and a tiny query language over the Serial Monitor. | 45 min |
|
Materials
- Half breadboard
- LED x 2
- Raspberry Pi 4 Model B
- Resistor x 2
Check for understanding
Each student finishes "Queries" on their own, without opening the hints. The simulator checks 5 of its 5 steps as they go.
Then ask each one to explain, in a sentence or two, the last thing it asked of them: "A tiny query language".
Keeping it healthy
Take the space back, and survive the power dying at the worst moment.
| Lesson | Time | What students do |
|---|---|---|
| CompactionTake the space back: copy only the live records into a new file and swap it in with one atomic rename, prove a crash halfway through loses nothing, and make SolDB compact itself when garbage passes half the log. | 35 min |
|
| Crash safetyPull the plug on SolDB, on purpose: a torn last record that silently corrupts the store, recovery that trusts only records ending in a newline, checksums that catch a flipped bit, and commit markers that make a batch all-or-nothing. | 45 min |
|
Materials
- Half breadboard
- LED x 2
- Raspberry Pi 4 Model B
- Resistor x 2
Check for understanding
Each student finishes "Crash safety" on their own, without opening the hints. The simulator checks 5 of its 5 steps as they go.
Then ask each one to explain, in a sentence or two, the last thing it asked of them: "What a real disk adds".
Boss
SolDB on a live sensor: logging, queries, crashes and retention.
| Lesson | Time | What students do |
|---|---|---|
| Boss: a sensor history storePut SolDB to work on live data: log a real sensor (the MCP3008 knob is the station thermometer), answer stats and recent-history queries over the Serial Monitor, survive a power cut on command, and keep the log small with a retention policy. | 55 min |
|
Materials
- Eight-channel ADC, MCP3008
- Half breadboard
- LED
- Potentiometer
- Raspberry Pi 5
- Resistor
Check for understanding
Each student finishes "Boss: a sensor history store" on their own, without opening the hints. The simulator checks 6 of its 6 steps as they go.
Then ask each one to explain, in a sentence or two, the last thing it asked of them: "What you built".
Teach this unit as a class
Start a class from this course and its 7 lessons are assigned in one go, in order. Students join with a six-character code or printed login cards, and each one’s work comes back to you.
The class tools are free for 30 days for one class of up to 20 students, with no card. The lessons themselves are free either way.