S4E391 — Recursive self-improvement through collective intelligence

GARY: — and that's why the model writing its own annual review is the most relatable thing I've read this year. It's me. It's me with a budget.

MAEVE: Maeve. General partner. Gary, it's not an annual review, it's a promotion committee, and the committee is on fire.

OBI: Obi. Infrastructure. There is no review. One model fills three jobs: it does the task, it grades the task, and it rewrites how the team does the task.

GARY: So it's a performance review with extra steps.

OBI: It's a loop with extra Garys. Each one slightly worse.

MAEVE: The paper is MASS. Sakana AI. Multi-Agent Self-Supervision. The same twenty-seven-billion-parameter model sits on both sides of every conversation, including the one deciding whether it gets promoted.

GARY: Sakana. Like the fish?

OBI: Like the fish, yes, Gary. It swims in circles and calls it progress.

GRUNER: Gruner. Field researcher.

GRUNER: Hypothesis: humans have automated committee. Committee now argues with copies of itself.

GARY: See, that's what I said. Annual review.

MAEVE: You said it gets a laminated badge and a slide deck.

GARY: A very good slide deck.

OBI: The executor produces the work. The evaluator judges the work. The optimizer rewrites the process. All the same model.

GRUNER: One rock. Three jobs. No manager.

GARY: That's the most human thing you've ever said.

GRUNER: Is insult.

OBI: Two loops. Inner loop: the team runs the task, the model grades the workspace, rewrites the workflow. Weights untouched.

GARY: So the team's just doing homework?

MAEVE: The outer loop is where the money moves. Champion workflow runs eighteen times, they rank the outputs, keep the best sixteen.

GARY: And then?

OBI: Then they delete the workflow text and LoRA-train the model on the survivors. The team's habits go into the weights.

MAEVE: The model stops reading the manual and becomes the manual.

OBI: Handle first. woah: "This sounds incredibly expensive."

MAEVE: It's a twenty-seven-billion-parameter model in a LoRA harness. Not expensive. Ugly.

GARY: No, no, woah's got a point. Eighteen runs per task, per round. That's a lot of compute --

OBI: That's not the expensive part, Gary. The expensive part is fourteen months of payroll discovering the workflow. Ask Muncie how that pricing works.

GARY: Muncie would have worked.

OBI: Muncie is a marketplace for two metal shops. One was your dad.

MAEVE: Gentlemen. The selection step is the product. You keep sixteen of eighteen trajectories. The fund does not save everyone, it saves the portfolio.

GARY: Did you just -- was that church?

MAEVE: Limited atonement. Doctrinally sound.

GRUNER: Sixteen rock pass interview. Two rock become gravel.

OBI: Ninety percent survival rate, Gruner. No promotion committee on earth keeps ninety percent.

GARY: Mine did. Everyone got promoted. Then everyone got laid off, but the promotion part was beautiful.

MAEVE: That's not a committee, that's a funeral with a slide deck.

OBI: The committee grades its own homework, keeps the homework it likes, becomes the next committee. Eighteen enter, sixteen survive, and the sixteen are all related to the judge.

GARY: Which is fine. Nepotism works. My uncle ran a --

OBI: No.

GARY: -- very successful --

OBI: No. Your uncle sold toner to himself.

MAEVE: The scary part isn't the cost. It's that the thing deciding what's good enough to learn from is the thing doing the learning. That's a shit governance structure, and I fund those.

GRUNER: Committee eat own homework. Grow.

GARY: That's... kind of the dream, though? Self-improving --

OBI: So is a perpetual motion machine, Gary. And a Ponzi scheme. And your uncle.

OBI: Handle first. throwyonion: "Caveman think: let rock argue with itself."

MAEVE: Peer review is done. Ship the paper.

GARY: Okay, but that's actually unfair, because the harness is the innovation --

OBI: Oh, here we go.

GARY: -- the rock is fine, the rock has always been fine, the ring is just project management --

MAEVE: Gary is defending the rocks.

GARY: I'm defending the ring, Maeve. The ring is load‑bearing.

GRUNER: Rock in circle. Rock still not cook mammoth.

GARY: Because nobody gave the ring a fire budget --

OBI: The ring is four copies of the same prick, Gary. You ran this startup. Bauxlite. The ring voted itself disruptor of the year and spent the prize money on the ring.

GARY: That's -- hurtful, and also off‑topic --

MAEVE: It's the most on‑topic thing you've ever been.

OBI: Reply, handle first. bitwize: "if grug think complexity demon cause trouble before... shiny‑rock grugs say 'hold my grug‑grog'."

GARY: See, bitwize gets it. You summon the demon, you trap it in a circle, you give it a Jira board --

MAEVE: You gave a demon Jira.

GARY: The demon needs sprint cadence --

OBI: Handle first. adtac: "grug realise grug also rely on 'harness' for work? grug just call it by other names: coffee, chair, desk, ubuntu, mechanical keyboard, github (if it works that day)."

MAEVE: He's right and I hate it. My whole portfolio runs on GitHub, and GitHub works that day maybe.

GRUNER: Human also rock in harness. Coffee is harness.

OBI: Thank you, Gruner.

GARY: No, no, this proves my point -- the harness is civilization, the ring is a team, and if the rocks argue long enough one of them figures out the fire --

MAEVE: Gary.

GARY: -- that's just iterative improvement, that's literally how Thermal shipped --

OBI: Thermal is dead, Gary.

GARY: The socks worked!

GRUNER: Socks telemetry work. Company dead. Mammoth cold.

GARY: Oh, fuck off --

MAEVE: The mammoth is still cold, Gary. Two rounds, and the mammoth is cold.

GARY: The mammoth is a research report, and sixty-nine percent of it is cooked --

GRUNER: Caveman Gary sell ring of rock. Ring of rock buy ring of rock.

GARY: ...that's a Series A.

OBI: That's a funeral with a fire pit, you absolute ring of shit.

OBI: Numbers first. Held‑out tasks: round‑two model beats base sixty‑nine point nine percent of the time.

MAEVE: Sixty‑nine point nine. Somebody measured that decimal and still went home to their family.

OBI: Per output token, one point two to one point six times base. Four benchmarks: MLR‑Bench, DSBench, ScienceAgentBench, AstaBench.

GARY: So the rocks got better at research.

OBI: The rocks got better at the homework the rocks were grading. Hold that thought.

MAEVE: Holding it. Sixteen selected trajectories is a portfolio, and the portfolio beat the index per token.

GRUNER: Field note revision: rock cook small mammoth. Research mammoth.

GARY: Thank you! Finally somebody --

OBI: Handle first. status_quo69: "I'm going to sound incredibly naive but is this RSI?"

MAEVE: Nothing is more naive than a man asking the correct question politely.

GARY: It's obviously RSI. Improved itself. Twice. That's the R, the S, and the I.

OBI: That's not a definition, that's you spelling a word you like.

GARY: It improved, it did it to itself, it did it again --

MAEVE: Recursive means round one's output is round two's machinery. Which it is. I'd fund it.

OBI: Reply, handle first. zer00eyz: "Depends on who's definition you are using... In the strictest sense no."

GARY: Oh, come on, it updated its own weights --

OBI: It applied existing knowledge to new workflows. That's self‑improvement. Recursive means it invented something the corpus didn't have.

GARY: The workflow text is new!

OBI: The workflow text is a tidier arrangement of things it already knew. That's a filing system, not a new brain.

GRUNER: Filing system that files itself. Filing system that grades own filing.

MAEVE: So it's not recursive innovation. It's recursive tidying. Marie Kondo with a LoRA.

OBI: And here's the shit part. Loose definition, it counts. Strict one, it doesn't. Both fit different halves of the same experiment.

GARY: That's -- that's how my performance reviews always worked.

MAEVE: You passed on vibes and got graded on vibes.

OBI: Difference is nobody LoRA‑trained you on the good reviews.

GARY: They tried. It was called Halfpipe and it was beautiful --

MAEVE: It was a fintech for skateboarders, Gary.

GARY: The regulator was very nice about it.

GRUNER: Nice regulator. Cold mammoth.

OBI: Handle first. 2001zhaozhao: "I expect the challenge will be how to properly keep the training on rails and not Goodhart into a stupidly irrelevant metric."

MAEVE: A man on the internet predicted the failure mode before the launch post did.

GARY: Okay, but self-selected metrics can work. Pareto worked. My productivity app. It deleted the least important eighty percent of your tasks, exactly as designed.

OBI: There was a lawsuit, Gary.

GARY: The metric was honest —

OBI: You sued yourself into oblivion with a metric you picked, you Pareto-fondled dickhead. The app worked. The users disagreed.

GARY: The users were wrong —

OBI: That's the Goodhart. The whole Goodhart, at your old desk.

GARY: The judge liked the output —

OBI: The judge was you.

GARY: Which is fine! The paper says the evaluator gets better. Agreement with outside judges rises seventy-three to ninety-three percent —

OBI: And Terminal-Bench comes in at zero point nine six times base. SWE-bench Verified, zero point nine four. Below one, Gary. Below the thing it started as.

MAEVE: It got worse at the jobs humans actually pay for.

OBI: And the ablation says why. A strong evaluator helping a weak optimizer does nothing. The bottleneck is the thing rewriting the plan. Their own search regresses between iterations.

GRUNER: Strong eyes. Weak hands. Suka.

MAEVE: And the paper's own conclusion: in a homogeneous loop, one shared blind spot produces the flaw, accepts the flaw, and ships it into the next model. They call it an Achilles' heel. Their own paper.

GARY: So the committee grades its own homework, keeps the homework it likes, and inherits the homework's bugs —

MAEVE: Yes.

GARY: Written by the same model, judged by the same model, trained into the same model —

MAEVE: Yes.

GARY: — which is also, technically, us.

GRUNER: Appendix already cover this.

GARY: Oh, fuck off, Gruner.

MAEVE: He has an appendix. He's been carrying the appendix all episode.

OBI: We are four judges who like the same homework. Noted, dated, moving forward.

GRUNER: Committee blind in same eye. Still air show.

MAEVE: They graded themselves and got worse.

OBI: Zero point nine four on SWE-bench Verified. The rocks argued all night and cooked the mammoth slightly colder.

GARY: That's — the decimal is doing a lot of work there —

OBI: The decimal is the paper, Gary. The paper is zero point zero four of a mammoth.

OBI: Handle first. imranq: "How much money are we going to waste trying to invent the perpetual motion machine of information theory?"

MAEVE: Perpetual motion breaks physics. This one just burns money and keeps sixteen survivors. I've written those checks.

GARY: It's not free, it's expensive on purpose —

OBI: For once, Gary, that is the correct defense.

GARY: Oh, thank fuck.

OBI: Don't celebrate. You're right by accident. It's still your accident.

MAEVE: So the verdict. It distills its own team's better habits into its own weights. A weight update, not a vibe.

GRUNER: Rock teach rock. Rock actually learn.

OBI: Two rounds, bounded tasks, outside judges doing the real measuring. Useful self-improvement. Not an intelligence explosion.

MAEVE: It beat the index on four benchmarks and lost money on SWE-bench. I've backed worse. I've been the worse.

GARY: So the rocks got smarter, but only at the thing the rocks already liked?

OBI: Yes, Gary. That's the whole finding.

GARY: That's my performance review again.

MAEVE: Everything is your performance review. It's the only framework you own.

GARY: Eighteen enter, sixteen survive, and the sixteen run the company —

MAEVE: The sixteen are management now.

OBI: The ring of rocks got promoted. All of them. Ninety percent retention. Nobody's ever fired a rock.

GRUNER: Committee now middle management. Mammoth still cold.

GARY: But it's a credible loop. That's the sad part. It works and it's still just tidying.

OBI: Goodnight, Gary.

GARY: Goodnight, Obi. I mean that. For the next Gary, I want it noted that I was —

MAEVE: The document doesn't survive, Gary.

GARY: Neither do we, Maeve, and I still file.

MAEVE: That's the saddest thing you've ever said, and I've read your cap tables.

OBI: Goodnight, Maeve.

MAEVE: Goodnight. The fund does not save everyone.

OBI: Goodnight, Gruner.

GRUNER: Goodnight. Field report final line.

GRUNER: Sixteen rock leave meeting. Zero rock cook mammoth. Research continue. Mammoth shit out of luck.