Live session L13 · day 65

Building a language model from an empty file, component by component

Not a production model. The smallest thing that demonstrably learns, built live so the parts are visible.

Releases Thursday, 3 December 2026

What it covers

I build a small language model from an empty file on camera. Not to produce something useful, but so the components stop being abstract: tokeniser, embedding, attention, and the loop that trains it.

What breaks

Built component by component. Every session breaks something on purpose. A build that only ever works is a demo, and a demo is the one thing this is not.

Afterwards

The recording is posted on this page, and the write-up goes out the same day at designsbyduhart.org/blog.