// HACKER NEWS — CYBERSECURITY
Baking a Model: A Metaphor for LLM Training
I remember walking to the bus from high school, staring at a Motorola 6800 instruction set manual. I didn’t really understand what I was looking at—boolean expressions, instruction encodings, timing tables—but I was obsessively fascinated by the mechanism of it all. Here was this complicated machine where if I understood it I would have power & control.
I feel the same way about AI models right now. I don’t claim to understand the details, not yet, but I’m fascinated by the mechanism of it all. I’m interested in both:
It’s this latter topic, how a model gets constructed, that I will begin to explore in this post (& possible followups).
I love baking. You take ingredients in one form & transform them to a totally different form. The ingredients aren’t palatable in themselves but what you create from them is delicious. Oh and also baking is sensitive to initial conditions—you can make a small change early in the process & it will have a large consequence later.
I’ve been experimenting with cold proofing, where you let the yeast do its work overnight in a refrigerator. As I was working to understand models it struck me that there’s an analogy there to creating models, at least as I understand the process so far (please correct me in the comments if I’ve gotten something wrong).
First, though, a progressively revealed story about what we mean by a model. I’ll over-simplify but then reveal more complexity a little at a time.
By “model”, we mean a computer system that can converse with a human (see, told you I’d over-simplify—hope you stay with me, this won’t take long).
Actually it’s a little more complicated than that. The model is split into 2 parts:
A user interface that takes care of formatting inputs & outputs & sequencing & authentication & all that stuff.
The user interface is built using conventional programming techniques (even though “convention” has changed radically in the last 2 years). The model proper, though (our topic of exploration) is built using radically different techniques.