// HACKER NEWS — CYBERSECURITY
Retire the Abstractions
TL;DR: We think CUDA DSLs are heading toward retirement
Last year, we set out to write megakernels. But implementing megakernels can be quite miserable. It requires complex data structures, extensive inter-thread/SM/GPU synchronization, deeply nested control flow, etc. We couldn't hold it in our heads, so we did what computer science has done for 70 years: we built a layer of abstraction. Even with it, we had to fight through race conditions and deadlocks for a couple months before the Llamas ran blazingly fast.
This year, we built an MoE megakernel, but we deleted the abstraction. With agents, we could work through the complexity directly and build target-optimized code from scratch. No intermediate layers of C++ abstraction were needed.
We've noticed an interesting pattern over the past quarter. The tasks we could already do without abstractions (e.g., writing an optimized GEMM kernel) are now nearly automated given the right prompt. For now, we still have to tell the agent which PTX instructions to use, the warp-specialization design, and so on, but it gets to the state-of-the-art pretty darn quickly.
The tasks we needed abstractions for (e.g., writing a megakernel) haven't been automated, unfortunately; we certainly can't one-shot a megakernel today. But agents let us put the abstraction in the prompt, in an incomplete, messy form, rather than in carefully designed C++ templates. Complexity that was unmanageable last year suddenly became manageable, because we now have a “compiler” that can take vague instructions and produce code. The ideas that went into the megakernel still mattered just as much, but it was much easier to express.
The job of abstraction as a cognitive offloader is starting to retire. Agents are taking that job.
By inductive reasoning, CUDA DSLs are next on the retirement list, including our beloved ThunderKittens.1 Probably next year, or maybe sooner. We don't know how far down this will go. Somewhere below us is a floor where an abstraction is purely the contract, rather than an offloader, and it stops. But we can't see it from here.
A codebase's pitch is precision. It's the one artifact that isn't ambiguous, the thing a particular machine will execute the same way twice. But precision has a cost, and the cost is brittleness. A codebase is tied to a language, a framework, a hardware target, and a set of conventions only the team that wrote it really understands. It appears portable, but barely.
A prompt is the opposite. It's fuzzy, but it travels. Hand the same intent to a different worker, human or machine, and if that worker is smart enough to fill in the gaps correctly, you get a correct result without ever having standardized the gaps in advance.
That's the interesting part. With agents, there's now a competition between an intelligent executor reading underspecified instructions and doing the right thing, versus specifying every gap so a dumb executor doesn't need judgment. DSLs and frameworks are that specification, expressed as a codebase. If the executor stops being dumb, the DSL loses its ground.