Starting with an empty Cargo.toml

One rule for the whole project — no crates, in the database or in its tests. It is not asceticism. It is the only way I know to actually learn this.

The first commit is an empty binary and a Cargo.toml with nothing under [dependencies]. That stays true for everything after it.

The reason is simple. If I pull in a crate for the on-disk format, I will learn the crate's API and nothing about the format. Every interesting decision in a storage engine — how a record is framed, when a write is safe to acknowledge, what a delete even is — is exactly the decision a good library makes for you and then hides. Hiding it is the right call in production and the wrong call here.

What it costs

Plenty. No serde, so records get framed by hand with length prefixes. No crc32fast, so checksums are a thing I will have to write. No async runtime, so concurrency is threads and locks from the standard library. Some of this is slower than the crate would be, and some of it I will get wrong at least once before I get it right.

That is the point. The bugs are the curriculum. A durability bug you introduced yourself and then had to reason your way out of teaches you more about fsync than any amount of reading the man page.

What it buys

Every byte on disk is a byte I chose to put there, and I can explain why. When something breaks there is no layer I cannot open. And the whole thing builds with cargo build on a clean machine with no network — which turns out to be a genuinely nice property for a database.

The plan is bottom-up and strictly ordered: log, then memtable, then on-disk tables, then the thing that ties them together. Nothing gets started before the layer under it works end to end.