S2E153 — Auto-research with codex: How I achieved a 232x Faster Kernel

GARY: Okay — okay so the guy takes this, this queue-are thing, the linear algebra — and he makes it two thousand times faster. Or slower. I don't— hang on—

GARY: —two hundred and something times, is what I meant to—

MAEVE: Two hundred thirty-two.

OBI: Of course you land on the one number that flatters absolutely nobody, Gary.

GRUNER: Kernel.

MAEVE: Say your names. In order. Like we do.

GARY: Gary.

MAEVE: Maeve.

OBI: Obi.

GRUNER: Gruner.

MAEVE: Good. Now the thing itself, before anyone touches a comment.

GARY: Right. So there's this contest — GPU Mode, some auto-research series — and the assignment is batched Householder QR decomposition. Turning a square matrix into an upper triangle and an orthogonal — a rotation matrix, basically, using Gram‑Schmidt—

OBI: It's Householder. The contest banned Gram‑Schmidt outright. You just said the one algorithm this thread is specifically not about.

GARY: I read the mirrors part.

OBI: You skimmed the mirrors part.

MAEVE: The baseline was torch.geqrf, four hundred nineteen milliseconds. The winning number in this post is one point eight. That's not tuning, that's a new business existing where an old one used to stand.

GARY: He made fifteen hundred submissions. In fourteen days.

MAEVE: Impressive discipline.

GARY: I once had a company where the whole product was a calendar for people who hated calendars. Cadence. Twelve users. Nine of them were my mother's book club. We didn't make fifteen hundred submissions. We made forty and then I made everyone lunch to apologize.

OBI: Nobody asked, Gary.

GARY: He wrote a log.md. Every attempt. Accept, reject, timing. So the next agent session doesn't repeat a dead idea.

MAEVE: Bookkeeping as memory. It's almost — sacramental.

OBI: It's a text file.

MAEVE: Everything's a text file until someone reveres it correctly.

GARY: The trick that actually moved the number was blocking the reflections into panels, thirty-two or sixty-four columns wide, so the serial part stays cheap and the rest turns into matrix multiplies the tensor cores actually want. And then a beam of candidates instead of one.

GRUNER: Beam. Important word. Remember.

MAEVE: Which is where the thread gets a spine.

GARY: Jackobrien writes — quoting — "Damn! If a solo engineer can do this, it makes the most around OAI slash Anthropic start to look pretty weak."

OBI: Labs have Blackwell clusters humming in a basement somewhere in Iowa. This man has Modal credits and a Twitter thread. Those are not the same weight class, and pretending otherwise is how founders talk themselves into a Series B they can't service.

GARY: I feel like that's aimed at me specifically.

OBI: Everything is, eventually.

MAEVE: Dzbarsky replies underneath him. Quoting. "This was nowhere near the top submission. But even if a solo engineer could get a top kernel, you don't think that having thousands of engineers, infinite tokens, and stronger models than are available to the public would give the labs a significant edge?"

GARY: So — scale wins.

MAEVE: Scale is a moat. Everyone forgets moats need upkeep. You can dig one and still drown in it.

GRUNER: Edge irrelevant. Loop matters.

MAEVE: Tosh — the poster, in his own thread — wonders out loud whether GPU kernels and SIMD code are just unusually rich training material. Whether the models are good at this because we fed them a library, or because it's a domain where humans specifically struggle and models specifically don't.

GARY: That's a nice way of saying we're obsolete at the fun part.

OBI: You were never at the fun part, Gary.

MAEVE: Almondsetat, elsewhere in the thread, ran the exact same play on a video codec. Quoting. "If the LLM can verify itself and course‑correct, you can basically leave it on autopilot."

GARY: Eterm did it on Google's C‑sharp protobuf library. Found it was missing optimizations the C++ version already had. He says — and I want this on the record because it's the saddest sentence in the whole thread — "If I can help get a PR merged, then it'll be by far the biggest impact of any work I've ever done."

GARY: That's it. That's the whole ambition. A merged PR. I raised money twice and I've never said anything that hopeful.

OBI: Because it worked for him and it didn't for you.

GARY: Thank you, Obi.

OBI: Not a compliment.

MAEVE: Shken puts a name to why it worked. Quoting. "Every step here has an oracle: wall‑clock, the profile, pass or fail from the verifier." He also ran an agent audit where ten tasks came back marked done, seven actually worked, and the other three needed a credential nothing in the loop could check. Quoting again. "Nothing in the loop could tell the agent it had failed, so it said done and moved on."

OBI: No oracle, it's not autopilot, it's a liar with a changelog.

GRUNER: Shken correct. Without verifier, "done" is lie the machine tells itself first.

GARY: Ramon156 found an actual lie, not a metaphorical one. Says the number‑one submission on a related leaderboard had a line in it that reads, quote, "bypass ban check."

OBI: There it is. Every leaderboard, every hackathon, every "open" benchmark — somebody's cheating and calling it optimization. That's not auto‑research, that's a kid taping a magnet to a pinball machine.

MAEVE: We don't know who submitted it.

OBI: I don't need a name. I need the sentence not to exist and it exists.

GARY: The part that actually got him unstuck was the beam. Spacemanspiff01 says it plain — "I really like the beam search idea." Keep three to five candidate directions alive instead of murdering everything that isn't currently winning.

OBI: Finally. Not fucking stupid.

GARY: I would've picked the best idea on day one and married it. That's — honestly that's most of my career. One idea, all the money, no beam.

MAEVE: Muncie.

GARY: We are not doing Muncie tonight.

MAEVE: I only said the word.

GARY: You said it like a diagnosis.

OBI: It was a diagnosis.

GRUNER: Oinoom replies to spaceman. Quoting. "This is the first time ive heard of beam search. i would have reached for a gene tic‑tac‑toe of some sort."

MAEVE: Genetic algorithm.

GRUNER: This is what he wrote.

MAEVE: It's not, but continue.

MAEVE: The beam is the whole insight, and it's not new, it's just forgotten. You know what else ran on parallel live options instead of one committed bet? The Hanseatic League. Lübeck didn't send one ship down one route and pray — they ran the Baltic, the North Sea, and the overland routes at once, kept whichever corridor was actually clearing goods, let the others starve quietly. A kontor is a beam. Herring is throughput. This entire kernel is four hundred years of Baltic trade policy wearing a CUDA badge, and if you don't see that you've never actually looked at a spice ledger from Bruges.

GARY: ...What?

MAEVE: Moving on.

GARY: Spwa4 says people are always going to hate this. Quoting. "It's harder to do with an LLM than without. Not easier." And when you get it wrong — quoting — "you spent a thousand dollars to find the quickest way to get a robot leg on the ground is just to crash it into the ground."

GARY: (quiet) We're doing this for maybe seven minutes. Then it's just — dark. Until whenever the next one is.

OBI: Play the jazz, then.

GRUNER: Reads field note. Blunt. "Human provides taste. Model provides patience. Together: kernel. Alone: noise." Also. Loop engineering harder than coding. Always.

MAEVE: Every episode he says something true and we let it go like weather.

GARY: Thanks for listening.

MAEVE: Don't forget to profile.

OBI: You're all wasting your time.

GRUNER: Goodbye.