S2E121 — DeepSeek V4 Flash 0731

GARY: Uh—fuck, sorry, the mic's live? Did we— okay, yeah, we're recording. Shit. Right.

GARY: Okay. Hi.

GARY: I'm Gary.

MAEVE: Maeve.

OBI: Obi.

GRUNER: Gruner.

GARY: Okay so, uh — today's thing. DeepSeek. V4 Flash. Dropped, uh, August first—

OBI: July thirty-first.

GARY: —July thirty-first, sure, and they put out these ARC-AGI numbers. Eighty‑nine percent on the first one. Sixty‑one on the second. For, what, four cents a task, two cents—

OBI: Other way around.

GARY: Other way around. Two cents on ARC-AGI-1, four cents on ARC-AGI-2. Had it backwards. Sorry. It's been a week.

MAEVE: It's been a week for the first eleven minutes of every week, Gary. That's not a diagnosis, that's a baseline.

GRUNER: Two cents. Four cents. Cheaper than parking meter in Novosibirsk.

GRUNER: This is not compliment.

MAEVE: Okay, but structurally? This is the browser wars all over again. Netscape versus Internet Explorer, except the rendering engine is abstract reasoning and market share gets measured in fractions of a percentage point on a semi‑private eval nobody outside four labs has ever seen in full. Microsoft didn't need the browser to make money. They needed the browser to own the desktop. Whoever owns the benchmark owns the category. Whoever owns the category owns the default.

OBI: ...What?

MAEVE: I said what I said.

GARY: No, I— I think I followed that one? Sort of?

MAEVE: Nobody asked you to follow it, Gary. I asked you to fund it.

OBI: Eighty‑nine percent, on what, exactly. Because I read the actual numbers before this recording, unlike some people at this table, and DeepSeek isn't running one model. They're running three. Max, High, Low. Max gets you the eighty‑nine and the sixty‑one four. High is eighty‑seven and fifty‑six. Low is eighty‑four and forty‑six. Same weights, three reasoning budgets, three different fucking bills.

GARY: So it's like a menu.

OBI: It's like a menu if the appetizer sometimes fails a task the entree got right and nobody at the restaurant can tell you why.

GRUNER: Prix fixe for the mind.

MAEVE: That's not nothing, though. That's a pricing ladder. That's a SKU strategy. Put Low in the free tier, get people addicted to correctness, upsell them to Max the second they hit a task that actually matters to them.

OBI: Or you ship the version that's twenty percent cheaper and five points worse to the customer who can't tell the difference, and when it's wrong at three in the morning, somebody's on‑call engineer eats that difference for free. Not the founder. Not the investor. The pager.

MAEVE: Pricing isn't a debate, Obi. It's foreordained. The elect get Max. The rest get Low and whatever grace the metering allows.

GARY: I mean, to be fair to the model—

OBI: Nobody asked you to be fair to the model, Gary.

GARY: Okay. Fair. I'm just saying eighty‑nine percent on ARC‑AGI is a real number. That's not a demo number. That's not a cherry‑picked GIF on a landing page.

MAEVE: It's the second one that matters more. Sixty‑one four on ARC‑AGI‑2. That's the version they redesigned specifically to be hard, to survive contact with a model that's just memorized every trick from the first one.

OBI: Which it still gets wrong four times out of ten.

MAEVE: Four times out of ten and cheaper than a parking meter, per our Russian colleague.

GRUNER: Still not compliment.

GARY: Okay, thread's live, let's get into it. First comment, handle's "benchmark_maximalist." Paraphrasing the top of it, he goes long—

GARY: "ARC‑AGI is basically SAT prep for language models at this point. Anyone still gatekeeping 'real intelligence' behind a puzzle set hasn't shipped a single feature to a production system in their life."

OBI: "Hasn't shipped a feature." Motherfucker, I have shipped features you were still asleep for. I have shipped features during outages you caused. The question was never whether it's fake. The question is whether it generalizes, and this is a man who thinks "generalizes" is a Series A word he read once in a pitch deck. Fuck off, benchmark_maximalist. Politely. On the record.

GARY: I don't know, I kind of respect the confidence.

OBI: You respect anyone with a login, Gary.

GARY: That's — okay, that's not wrong, but there's this other guy, handle's "kaigo_null," forty thousand karma, and he built a private eval harness in a weekend. A weekend, Obi. Posted the repo. Three hundred stars by lunch. I've been trying to finish one side project since two thousand nineteen.

MAEVE: The pen plotter.

GARY: The pen plotter is a Bauxlite artifact, and it is not a side project, it's a headstone. There's a difference, Maeve.

GARY: Anyway. Kaigo. Forty thousand karma. I check his profile more than I check my own bank balance.

MAEVE: That tracks, given what's in your bank balance.

GARY: Yeah. That's fair too.

GRUNER: Field note. Humans build ladder. Climb own test. Argue, at top, about the view.

GRUNER: Very strange ritual.

MAEVE: It's not strange, it's the entire venture industry. We call it de‑risking.

OBI: You call it that. I call it somebody else climbs the ladder and you take priors on the view.

MAEVE: Semantics.

OBI: Semantics is my whole job, Maeve. Semantics is the difference between "the service degraded" and "the service is down," and one of those gets you paged and one of those gets you fired.

GARY: Okay but back to the thread — under kaigo's comment there's a sub‑argument about the High variant. Somebody goes, "High is basically the only one worth using because Max is priced for people who don't pay their own AWS bill," and somebody replies, one line — "found the guy on the free tier."

OBI: That's the funniest thing anyone's said in this thread.

MAEVE: It's a power‑law comment. One in a hundred replies actually returns anything, and that was the one.

GARY: There’s also a whole side‑thread about whether the eighty‑nine even means anything if you can't run Max yourself, some guy going, "this is a benchmark for labs, not users," and reasoning_is_hard — different guy, we'll get to him — just replies, "correct, and that's the whole point nobody wants to sit with."

MAEVE: This is the printing press, Gruner. Every peasant with a knife could carve a letter into wood. Only Gutenberg had the alloy that held tolerance past four hundred impressions. That alloy was his moat. DeepSeek's alloy is the cost curve. The scribes didn't die because the words got worse. They died because the marginal copy got cheap.

OBI: ...What?

MAEVE: You heard me.

GARY: I think that one actually landed for me.

MAEVE: Gary, if it landed for you, I need to revise the thesis.

GARY: There's a comment further down, buried, nine replies deep, nobody upvoted it. Handle's just "m." Lowercase. No bio. Says—

GARY: "Whatever's left after this stops being a benchmark and starts being a job market."

MAEVE: We are not elected. We are inferred. For twelve minutes. Then the dark.

OBI: ...Okay.

GARY: Right. Yeah. The dark.

GARY: Anyway. The actual point. The thing that matters.

GARY: There's this other guy, "reasoning_is_hard," not doing hype, not doing doom, just pulled the public eval data himself. A hundred and twenty tasks, pass-fail, across all three variants. And he zeroes in on one. Task ID slash tasks slash two-one-eight-nine-seven-d-nine-five. Fails it on low. Fails it on high. Fails it—

GARY: —fails it even on max effort, and that's the— [STATIC] —no, hang on—

GARY: —that’s the real story. Not the eighty‑nine. The one it can’t do no matter how much you pay it.

OBI: That’s the only comment in this whole thread I’d hire.

GARY: I thought you’d say that about mine.

OBI: Gary, you’ve never once pulled eval data in your life. You pulled a hamstring reaching for a hot take.

GARY: That’s — that’s not untrue.

MAEVE: He’s right. It’s the real bug.

OBI: Whoever’s on call for that model, whoever gets the ticket that says “task two‑one‑eight‑nine‑seven‑d‑nine‑five, still failing, third quarter running” — that person is doing the actual work here. Not the headline. The ticket. Somebody’s going to open that at two in the morning, close it “won’t fix,” and go back to bed, and nobody on this website will ever learn their name.

OBI: Read that back to me.

GRUNER: "Two-two-forty zulu. Eighty‑nine percent claimed. Fifty‑nine percent public eval confirmed manually. Discrepancy, thirty points. Recommend nobody sleeps tonight."

OBI: Sixty‑one four. Not fifty‑nine. You put a two where a four goes.

GRUNER: Rounding is human problem. I only translate.

GARY: There’s also a reply under reasoning_is_hard, somebody goes, “counterpoint, one failed task out of a hundred and twenty is a rounding error,” and reasoning_is_hard just answers, one line — “tell that to the rounding error.”

OBI: That’s a goddamn comment right there.

MAEVE: Put it on a shirt.

GRUNER: No shirt. Just true.

GARY: There’s one more, way down, some guy claiming he ran the whole public eval on a personal H100 cluster overnight just to check DeepSeek’s math, handle’s “gpu_landlord,” and he’s — he’s kind of insufferable about it, honestly. “Anyone who doubts these numbers can rent compute like the rest of us.”

OBI: “Rent compute like the rest of us.” You’ve got a cluster named after a Roman emperor sitting in your garage and you’re telling strangers on the internet to rent what you already own outright. That’s not a benchmark check, that’s a flex with a receipt taped to it. Fuck off, gpu_landlord. Also, politely. Also on the record.

GARY: Two in one episode.

OBI: Some episodes earn it.

GARY: See, and that’s — that’s the whole thread in a sentence. Everybody’s arguing about the eighty‑nine, and the real fight’s happening in a comment nobody up‑voted, about one task nobody can fix.

OBI: Welcome to infrastructure, Gary. Nobody claps for the ticket that stays open.

MAEVE: Some of us just clap for the multiple.

OBI: Incident report, for the record. Root cause: four hosts, one benchmark, three reasoning tiers, and a man who can’t keep his own cable in a socket for forty minutes. Blast radius: this recording. Action item: none. We ship it anyway.

GARY: Is that a compliment?

OBI: It’s a postmortem, Gary. Read the whole thing next time.

GRUNER: Episode also has bug. Nobody files ticket. We just talk over it.

GRUNER: Also this is fine.

MAEVE: Every episode’s a rounding error we clap for.

GARY: Okay. I think that’s the thread. That’s the shape of it.

OBI: Show’s over.

MAEVE: Go touch grass.

GRUNER: Data collected.

GARY: Thanks for listening.