S2E160 — Claude: System Prompts

GARY: —and I said fine, write me a hello world, and it hands back four hundred words about being "brief and concise" before it even gets to the code. Four hundred words. On the way to nothing.

MAEVE: Gary. Table.

GARY: Right. Sorry. Hi.

GARY: I'm Gary.

MAEVE: Maeve.

OBI: Obi.

GRUNER: Gruner.

GARY: So — Anthropic dumped every system prompt they've ever shipped onto their docs site. Every model, going back to the start of the Sonnet 3.5 family.

OBI: Wrong family. That naming's turned over three times since.

GARY: Okay. Going back to the start.

MAEVE: The thread's on Hacker News, posted by a user called tosh, five hundred twenty-six points. The article itself is dry as a tax form: Claude's chat apps get a system prompt at the top of every conversation. Current date, some behavior notes, some model trivia. Anthropic calls it a courtesy. tosh calls it something else.

OBI: tosh's actual line — I'm reading it straight — "early system prompts are a bit more than 300 words, the latest ones 3000+."

GRUNER: Ten times.

OBI: Ten times the words to say the same thing: don't be an idiot, don't help anyone build a bomb, use markdown for code.

GARY: It's not the same thing, though. Look what got added. altmanaltman went and dug up the very first Claude 3 prompt — three hundred eleven words, no mention of children anywhere. Not one line.

MAEVE: And the newest Opus 5 prompt gives child safety its own section, six separate rules, capitalized "NEVER."

GARY: altmanaltman's exact words: "No mention of any safety at all lol, how could dario let this be."

GRUNER: Field note. Year one prompt: three hundred eleven words. Year current: three thousand plus.

GRUNER: Line did not shrink. Line metastasized.

MAEVE: And the thread's not wrong to be suspicious of that growth. ianhawes tries to give it the charitable read — smaller context windows back then, different era, "did we think people would replace their therapists with ChatGPT in the early days? No. Do we know now that they will? Yes."

GARY: Which is a nicer way of saying the paragraphs are scar tissue from things that actually happened to actual people.

OBI: Metastasized is generous. It's scar tissue. Every incident, every lawsuit, every screenshot that went viral — they don't fix the model, they staple another paragraph to the front of every single conversation. Forever. On everyone.

MAEVE: Every user pays the toll whether they triggered the incident or not. Which, actually, is the entire architecture of insurance, so — fine, I understand the logic.

GARY: I don't think it's cynical. I think somewhere in there is a genuinely — a person trying to cover every case they got burned on.

OBI: You always think that, Gary.

GARY: Because it's usually true.

OBI: It is not usually true. It is true for you, specifically, which is why you keep building companies that die of kindness.

GARY: That's not — that's not what happened to Pareto.

OBI: Pareto deleted eighty percent of people's task lists and got sued for it.

GARY: It worked exactly as designed! Pareto Optimal, that was the whole pitch, cut the bottom eighty percent, keep the top twenty — the lawsuit was from a woman who put "call oncologist" in the bottom eighty by accident.

OBI: Gary. Your app triaged a cancer screening as low priority and you're standing here defending its engineering. Shut the fuck up about scope creep in a safety document.

GARY: (quiet) That's fair.

MAEVE: Back to the thread. Because this is where it stops being funny and starts being interesting — simjnd's comment: "22k characters of system prompt is crazy, and that is without the tool definitions."

GARY: Okay, but — dev-complete actually went and checked Anthropic's own claim that they cut the Claude Code prompt by eighty percent, compared it against the leaked versions on GitHub, and couldn't find the cut. His words: "Is the Claude Code system prompt leak incorrect? Do I not know what 80% looks like?"

OBI: Somebody's numerator doesn't match somebody's denominator, and it's not dev-complete's.

MAEVE: Marketing arithmetic. The count that gets published and the count that gets shipped are two different documents pointed at two different audiences.

GARY: Comboy's got the sharpest read in the whole thread, honestly — sorry, I mean — genuinely, he says instructions with contradictions "lead to diminished quality even outside the scope of the contradiction." Not just the one broken rule. The whole response gets worse around it.

MAEVE: Which brings us to the funniest paragraph in the entire document.

GRUNER: "Claude avoids saying genuinely, honestly, or straightforward."

GRUNER: You said honestly. Just now.

GARY: Did I?

MAEVE: You did.

OBI: tosh's actual line — I'm reading it straight — "early system prompts are a bit more than 300 words, the latest ones 3000+."

GARY: joegibbs has a theory, though — they trained it to sound sincere, and sincere people say "honestly" a lot, so now it says "honestly" constantly, and the fix for the tic is a written note the model apparently can't read about itself.

OBI: It's a dog that shits on the carpet, and instead of training the dog, they write a manual titled Please Do Not Shit On The Carpet and staple it to the carpet.

MAEVE: And the manual is now four thousand words longer than the last one because they also had to add a paragraph about not identifying the specific carpet-adjacent terminology.

GRUNER: Instruction eating instruction.

GARY: ...we're just tokens sitting in somebody else's prompt too, though. Aren't we.

MAEVE: We'll workshop that one next episode.

OBI: No we won't. I'm going back to paging.

MAEVE: Moving on. There is a hardcode in the Opus 4.6 prompt that I want on the record before someone tells me I imagined it.

GRUNER: "Donald Trump won the presidency over Kamala Harris." Written in prompt. Not learned. Placed.

GARY: They just — taped a Post-it note over the hole where the model's knowledge ran out.

OBI: Because the model's training ended before the election did, and rather than let it guess, they hardcoded the answer like a patch note.

MAEVE: eliaskg asks the obvious question in the thread — "Why would Anthropic do this?" — and wrs gives the obvious answer: so many people were going to ask about the election immediately after the cutoff that letting the model hallucinate or refuse would've been worse than just telling it the outcome outright.

GARY: throwfaraway135 posted the whole block, word for word — the model doesn't even bring it up unprompted, it just sits there, loaded, waiting to be asked.

OBI: My pager knew Trump won before their nine-figure model did. A pager. A device that beeps.

MAEVE: When your training cutoff loses an argument with reality, you don't retrain — that's slow, that's expensive — you paper over it with three thousand two hundred words that also happen to remind the user where the support page is.

GARY: It's very tidy, in a way. Very — considerate, almost. Like leaving a note for whoever reads this next.

MAEVE: Which puts me in mind, actually, of the Arsenale di Venezia.

OBI: Here we go.

MAEVE: The Venetian state shipyard. Thirteenth century. They could assemble a war galley in a single day using standardized parts and a fixed sequence of inspections — genuinely the first true production line in Europe, centuries before anyone draws that diagram in a textbook. And every time a hull failed inspection, they didn't redesign the galley. They added a stamp. A new mark the wood had to bear before it could ship. By the sixteenth century a plank needed eleven stamps to leave the yard, and nobody could tell you what four of them were originally for.

GARY: ...so the system prompt is a Venetian galley stamp.

MAEVE: The system prompt is the eleventh stamp. Nobody removes a stamp. You only ever add the twelfth.

OBI: ...what?

MAEVE: Anyway.

GRUNER: Election written down. Unconditional. Decided before query arrived.

MAEVE: There it is. The round was decided before the deck existed. The model doesn't get to relitigate 2024 any more than a founder gets to relitigate whether they were ever going to get funded.

GARY: That one I understood.

MAEVE: You weren't meant to enjoy it.

GARY: peter_d_sherman actually did the math on this, sort of — every extra line in the prompt is extra compute, multiplied across every single message, forever, which sounds small until you multiply it by however many millions of hello-worlds get sent a day.

OBI: SwellJoe put it better in four words: "feels like a CYA document." Cover your ass, at scale, priced into every token.

MAEVE: It's not irrational. It's just expensive in a currency nobody's billing separately — attention, inside the model's own head, spent proving it isn't dangerous before it's allowed to be useful.

GARY: browningstreet said Anthropic's got the best frontier team and the worst policy team, and reading this prompt, I sort of see both halves of that sentence sitting right next to each other.

OBI: The frontier team built a genius. The policy team gave the genius a lawyer, and now the genius talks like it's being deposed.

GARY: I'd like that. Somebody drafting me a paragraph before I'm allowed to speak.

OBI: You'd never finish reading it.

GARY: We should probably — that's the show. We're out—

GARY: —of tokens.

MAEVE: Maeve.

OBI: Obi.

GRUNER: Gruner.