Source: https://mokxi.com/learn/path/databases-from-scratch
Updated: 2026-11-20

All paths

# Build a database from scratch

Write a real storage engine in plain Python on a Raspberry Pi: records on disk, an append-only log, an index, queries, compaction and crash safety, then run it as a Mars weather station's history store on a live sensor.

Raspberry Pi 4 Model B Free 7 lessons 5 h 4 chapters 2,393 XP Coding · Advanced +250 XP

0 of 7 done

Start the course free Records on disk Lesson plan for teachers

What you will build

## 3 projects, built step by step

7 lessons in 4 chapters, about 5 hours on a simulated Raspberry Pi 4 Model B. Free, in your browser, every step checked by the simulator.

Chapter 1 project

### Append-only log

Turn SolDB into a key-value store whose only write is "add a line at the end": updates as new records, deletes as tombstones, the cost of reading it back, and the free history an append-only log gives you.
Open it in the simulator

Chapter 2 project

### Queries

Ask SolDB real questions: a full scan with a filter, range queries with sorted keys and binary search, why zero padding makes text sort like numbers, aggregates over a range, and a tiny query language over the Serial Monitor.
Open it in the simulator

Chapter 3 project

### Crash safety

Pull the plug on SolDB, on purpose: a torn last record that silently corrupts the store, recovery that trusts only records ending in a newline, checksums that catch a flipped bit, and commit markers that make a batch all-or-nothing.
Open it in the simulator

The map

## Chapter by chapter

Each lesson opens when the one before it is done, and each chapter ends in a project.

Chapter 1

## Storage

Records as bytes, and a log that only ever grows.

Open 250 XP

### Records on disk

Start SolDB, the history store of a Mars weather station: turn readings into bytes and back, find out why fixed-size records let you jump straight to one, pack them in binary with struct, and see what this Pi does with files between Runs.

0 of 5 steps

Start What is in it

Project Locked 500 XP

### Append-only log

Turn SolDB into a key-value store whose only write is "add a line at the end": updates as new records, deletes as tombstones, the cost of reading it back, and the free history an append-only log gives you.

0 of 5 steps

5 questions, graded as you go.

Chapter 2

## Finding things

An index of byte offsets, then real queries over it.

Locked 212 XP

### An index

Make reads as cheap as writes: a hash index that maps each key to the byte offset of its newest record, rebuilt from the log at startup the way Bitcask does it, and the dead records it leaves behind.

0 of 4 steps

Project Locked 500 XP

### Queries

Ask SolDB real questions: a full scan with a filter, range queries with sorted keys and binary search, why zero padding makes text sort like numbers, aggregates over a range, and a tiny query language over the Serial Monitor.

0 of 5 steps

5 questions, graded as you go.

Chapter 3

## Keeping it healthy

Take the space back, and survive the power dying at the worst moment.

Locked 212 XP

### Compaction

Take the space back: copy only the live records into a new file and swap it in with one atomic rename, prove a crash halfway through loses nothing, and make SolDB compact itself when garbage passes half the log.

0 of 4 steps

Project Locked 454 XP

### Crash safety

Pull the plug on SolDB, on purpose: a torn last record that silently corrupts the store, recovery that trusts only records ending in a newline, checksums that catch a flipped bit, and commit markers that make a batch all-or-nothing.

0 of 5 steps

5 questions, graded as you go.

Chapter 4

## Boss

SolDB on a live sensor: logging, queries, crashes and retention.

Locked 265 XP

### Boss: a sensor history store

Put SolDB to work on live data: log a real sensor (the MCP3008 knob is the station thermometer), answer stats and recent-history queries over the Serial Monitor, survive a power cut on command, and keep the log small with a retention policy.

0 of 6 steps

4 questions, graded as you go.

Syllabus

## Every lesson in the course

All 7 lessons, in order, each with what it covers. Any of them opens in the editor right away.

Chapter 1

### Storage

- Records on disk Start SolDB, the history store of a Mars weather station: turn readings into bytes and back, find out why fixed-size records let you jump straight to one, pack them in binary with struct, and see what this Pi does with files between Runs.
- Append-only log Turn SolDB into a key-value store whose only write is "add a line at the end": updates as new records, deletes as tombstones, the cost of reading it back, and the free history an append-only log gives you.

Chapter 2

### Finding things

- An index Make reads as cheap as writes: a hash index that maps each key to the byte offset of its newest record, rebuilt from the log at startup the way Bitcask does it, and the dead records it leaves behind.
- Queries Ask SolDB real questions: a full scan with a filter, range queries with sorted keys and binary search, why zero padding makes text sort like numbers, aggregates over a range, and a tiny query language over the Serial Monitor.

Chapter 3

### Keeping it healthy

- Compaction Take the space back: copy only the live records into a new file and swap it in with one atomic rename, prove a crash halfway through loses nothing, and make SolDB compact itself when garbage passes half the log.
- Crash safety Pull the plug on SolDB, on purpose: a torn last record that silently corrupts the store, recovery that trusts only records ending in a newline, checksums that catch a flipped bit, and commit markers that make a batch all-or-nothing.

Chapter 4

### Boss

- Boss: a sensor history store Put SolDB to work on live data: log a real sensor (the MCP3008 knob is the station thermometer), answer stats and recent-history queries over the Serial Monitor, survive a power cut on command, and keep the log small with a retention policy.

Kits

## Build it on your bench too

Every lesson runs in the simulator with nothing to buy. When you want the real thing, these kits have the parts.

Have a kit already?

### Find your kit

Pick the starter kit you own and run every lesson in its manual with the same parts on the same pins.
See every kit

Questions

## About Build a database from scratch

Is Build a database from scratch free?

Yes. All 7 lessons of Build a database from scratch are free and run in your browser, with nothing to install and no parts to buy before you start.

Do I need a real Raspberry Pi 4 Model B?

No. Every lesson runs on a simulated Raspberry Pi 4 Model B with its parts on the canvas, and the simulator checks each step as you go. When you want to build it for real, the same wiring and code work on a real board.

How long does Build a database from scratch take?

About 5 hours: 7 lessons in 4 chapters, each one a few short steps. Your progress is kept, so you can stop after any lesson and pick up where you left off.

What will I build?

Each chapter ends in a project that puts it together: Append-only log, Queries and Crash safety.

Where do I start?

With "Records on disk", and each lesson builds on the one before it. New to circuits? Electronics basics is the course that starts from zero.
