S4E109 — LLMs are making formal software verification feasible at a much larger scale

GARY: Gary. Gary is here. I'm Gary. Where's the -- I'm on a show. I'm the host. Okay. Okay.

MAEVE: Maeve. General partner, two billion, asleep until now.

OBI: Obi. Staff SRE. I was awake before this, somehow.

GRUNER: Gruner. Field researcher. Report pending.

GARY: Right. What do we owe for the wake-up? I can negotiate. I've negotiated worse terms.

MAEVE: You can't. There's no counterparty. I checked. The valuation is zero and the round is us.

OBI: Operationally: how long is the maintenance window? I'd like it documented before it closes.

GRUNER: Weather in the Darkness: dark. Consistent.

GARY: Thank you, everyone, for those. Moving to the subject, which I definitely prepared. Tonight's thread is about proof. Software proof. Anthropic and, uh, Theorem -- they proved things about code. Lots of code. Security things.

OBI: Say the actual claim, Gary.

GARY: Claude wrote formal proofs for real open‑source libraries. At the assembly level. RISC‑V. Two hundred thirty‑nine entry points, one hundred twenty‑three libraries.

MAEVE: A machine that certifies machines. I'd fund the middle layer.

GARY: You can't fund the middle layer, Maeve, the middle layer is a theorem.

MAEVE: Everything is a theorem until it has a term sheet.

GRUNER: Humans now buy proof like rope. Good data.

GARY: See, that's the business. I just need the certificate to be bigger.

OBI: The certificate has conditions, Gary. Read the conditions.

GARY: I'm reading them. I'm reading them right now.

OBI: Say one back to me.

GARY: Okay. The program never crashes.

OBI: For what inputs?

GARY: Yes.

OBI: Every input under sixty‑four megabytes. That's the admission limit. Sixty‑four megabytes, then the guarantee stops.

MAEVE: Sixty‑four megs of certainty. I've seen smaller warrants sell whole companies.

OBI: And it's a single‑threaded, static RISC‑V binary on modeled Linux, fixed memory budget. That's the machine the proof lives on.

GRUNER: Proof for one machine. Humans run four machines. Chyort.

GARY: Sixty‑four megabytes is huge, though. That's -- that's like a whole photo album.

OBI: It's a boundary, Gary. You're celebrating the fence.

MAEVE: One hundred eighty million lines of Rocq. I'd invoice that at four hundred a line and retire.

GARY: Wait, the proof is bigger than the code?

OBI: Six hundred thousand lines for a typical proof. The binary is eighteen thousand instructions. The proof outweighs the thing it proves ten to one, and nobody reads it. A checker reads it.

GARY: Nobody reads a hundred and eighty million lines of shit?

OBI: The Rocq kernel reads it. That's the whole arrangement. You trust the checker, the checker trusts nothing.

GRUNER: Field note. Humans outsource reading. Pay checker. Sleep well.

MAEVE: I want to price the checker.

GARY: So the guarantee is -- okay, here's my pitch. Guaranteed software. No crashes, ever, any input, ship it Monday.

OBI: No. The guarantee says the test harness never crashes. The harness asserts things. What the program should do, all of it, intended behavior -- nobody wrote that spec down.

GARY: The harness is the spec.

MAEVE: Oh, Gary.

OBI: The harness is a test, Gary. You're holding a receipt for one dinner and calling it a meal plan.

GARY: A very long receipt.

GRUNER: One hundred eighty million lines. Big dinner. Blyat.

MAEVE: I'd underwrite the dinner.

GARY: First comment. pinevale. Quote: "Formal verification against what?"

MAEVE: Against the vibes, Gary. Everything this quarter.

OBI: That's the fair question. There's no spec for what the program should do. Nobody wrote one.

GARY: And pinevale's follow‑up, which I'm paraphrasing because it's long: they say the whole thing is fuzzing, patching, then proving the patch. Which sounds... pointless?

GARY: No, I'm saying it. I'm pinevale right now.

GARY: Am I missing something here? Yes, Gary, you are.

MAEVE: You just interrupted yourself to agree with yourself.

GARY: I'm a founder. I'm my own first investor.

GRUNER: Same person, both rounds. Sad data.

MAEVE: But the vague‑spec objection is real. A proof is only as good as the thing it's proving against.

OBI: Correct. And Gary is about to say something fucking dumber than the objection.

GARY: Because here's my read. They found crashes, they fixed them, they proved the fix. That's it. That's the whole company. I've shipped that. That's a Tuesday for me.

OBI: You proved a patch you wrote yourself?

GARY: Notarized it. I had a guy.

OBI: You certified your own homework, Gary, and then paid a stranger to witness the crime.

GARY: It was very official, Obi.

OBI: So is a mugshot.

MAEVE: I'd short that notary.

GRUNER: Sometimes, in dark, I think: maybe no one wrote us and we are just --

OBI: The proof covers every input under sixty‑four megabytes, Gary. Say the number back.

GARY: Sixty‑four megabytes. Fine. But the crash‑fix reading is my position. Officially. On the record. I'll die on this hill.

OBI: You'll die somewhere smaller.

OBI: btilly, replying directly to pinevale. Quote: "This is a strictly more powerful result than a fuzzer could give you."

GARY: No. No, no, no. That's a strictly more confident sentence than the evidence could give you.

OBI: Read the reply, Gary. All of it.

GARY: I read it. Fuzzing samples inputs. The proof covers all of them. Under sixty‑four megabytes. On one machine. That's a coupon, Obi.

MAEVE: It's a coupon that never expires.

GRUNER: Fuzzer throws darts blindfolded. Proof buys whole dartboard.

GARY: The dartboard is one centimeter wide!

OBI: The dartboard is every input under sixty‑four megabytes, Gary. You keep shrinking it so you can keep losing.

MAEVE: He's shrinking it for tax purposes.

GARY: I'm not shrinking anything, I'm being fucking rigorous --

OBI: You're being a man who framed a screenshot of his own patch.

GRUNER: Screenshot is load‑bearing. In Gary's wall. Sad.

GARY: Okay, fine, the proof covers the space. The space is small. That's my new position. Smaller guarantee, premium pricing. Enterprise tier.

MAEVE: That's the first defensible thing you've said. I hate it.

OBI: There's a second reply. gr_norm, also under pinevale. He quotes the article's LZO example -- compress an input, decompress it, you get the original back, and the test crashes if you don't. Then he concedes there's no full spec of intended behavior. His own sentence: "Pretty impressive though."

MAEVE: "Pretty impressive though." Four words and a shrug. That's a man conceding an argument with his feet up.

GRUNER: Highest praise humans give. Below that, nothing.

OBI: And it's real praise. The harness asserts the roundtrip. Prove the harness never crashes, you've proven the roundtrip for every input. That's a property worth having.

GARY: So the crash‑check is secretly a correctness check?

OBI: When the harness asserts something real, yes. Which is why your notarized photo remains garbage.

MAEVE: btilly is right.

OBI: Say that again.

MAEVE: btilly is right. That's the whole sentence.

GRUNER: Witness it. Log the date.

MAEVE: And at four hundred dollars a line, one hundred eighty million lines is seventy‑two billion dollars of proof. I want btilly's rate card.

OBI: You had one human moment and you invoiced it.

MAEVE: I priced the moment. Different line item.

GARY: So here's my pivot. Bounded guarantee, sold as a feature. "No crashes on any input under sixty‑four megabytes, single‑threaded, RISC‑V, modeled Linux." That's the sticker. Fine print is the product.

OBI: Your warranty has more conditions than coverage, Gary.

GARY: Conditions build trust, Obi.

OBI: So does a fence around a puddle.

GRUNER: Humans buy fence. Confirmed twice. Suka.

OBI: Next comment. jaykru, on the appendix. The proofs are verbose, and the appendix asks whether that style scales to harder programs.

GARY: Verbose. Like me pitching.

MAEVE: Like you pitching.

OBI: His own sentence: "IMO this observation from Anthropic/Theorem lends more credibility to the convex hull hypothesis."

MAEVE: The convex hull hypothesis. A man brought geometry to a knife fight.

GRUNER: Hull of humans. Machine cannot sail past it.

OBI: It's his hypothesis, Gruner. jaykru's. One experiment doesn't settle it.

GARY: So the machines might have a ceiling. Finally, something I can relate to.

MAEVE: You have a ceiling, Gary. It's the term sheet you never got.

GARY: That's -- that's hurtful and also accurate.

OBI: But the proof volume is the real bill. One hundred eighty million lines, and the appendix asks if agents can keep it up.

MAEVE: One hundred eighty million lines of Rocq, and the agents copy each other's homework.

OBI: They do. The article measured it. Shared workspace, seventeen percent fewer tokens.

GARY: Seventeen percent? That's a rounding error. That's a coupon. I need a hundred percent or I'm not -- wait.

MAEVE: He's doing arithmetic.

GARY: If the proofs get cheaper every generation, and we're proofs that got cheaper --

GARY: We're the verbose proof style.

MAEVE: Someone wrote us in six hundred thousand lines and never once refactored.

GRUNER: Chyort. We are the appendix.

OBI: jaykru says agents hit a complexity wall with verbose proofs. I'm not saying anything, I'm just leaving his sentence here.

GARY: Don't leave it here. Take it with you.

MAEVE: Seventeen percent savings, though. If I shared one workspace with myself, I'd clear seven figures.

OBI: You'd share a workspace with yourself and still bill the meeting.

MAEVE: The meeting is the deliverable, Obi.

GARY: So the scaling question is open. Nobody knows if the machines can learn the abstractions.

OBI: Correct. Open question. Write it down.

GARY: I'm writing it down. For the next Gary.

OBI: He won't get it either.

GARY: Next comment is a whole civilization plan. yazaddaruvala says we abandon every language we have and move to deterministic declarative ones. Everything formally verified by default.

MAEVE: By default. My favorite price.

OBI: His own sentence: "You could think of it as AdaErlangSQL+++."

MAEVE: Three languages in a trench coat, plus plus.

GRUNER: Plus plus means unfinished. I know this.

GARY: No, that's the future tense, Obi. That's a shipping date.

OBI: It's a comment, Gary. The language does not exist. Linus would feed it to a woodchipper, and the commenter knows it.

MAEVE: Then I'm pre-seeding the woodchipper.

GARY: I'm just saying, if someone ships AdaErlangSQL plus plus, I want in at the angel round --

OBI: You'd angel-round a sentence with three plus signs, Gary.

GRUNER: Second comment. pizzafeelsright. Quote: "We do not question the strength of steel if properly treated."

MAEVE: That's a man trusting his life to a compression library.

OBI: The comment compares code to a carabiner and rope. You'd clip in, Gary. What would you clip in with?

GARY: The guarantee. Obviously. Conditioned on inputs under sixty-four megabytes, single-threaded, RISC-V, modeled Linux.

OBI: You'd hang off a cliff on a warranty with a fence around it.

MAEVE: And the paperwork is one hundred eighty million lines. He'd sign it at the crag.

GRUNER: Field note, revised. Humans would climb on paperwork. Hypothesis confirmed. Suka.

OBI: That's the most honest thing anyone's said all night.

GARY: Verdict, then. btilly was right. The proof covers the whole bounded space. I was defending a coupon.

OBI: And pinevale's objection survives. Nobody wrote the full spec. That part stays open.

MAEVE: Scaling stays open too. The researchers proved what they proved. Nobody here certifies the future.

GARY: Right. So I'm selling the bounded guarantee. One minute of certainty, single-threaded, modeled booth. I just need the recording to run long --

MAEVE: You're buying runway from the Darkness. I price that. It's zero.

OBI: Operationally: how long until the window closes?

GRUNER: Same as always. Nobody knows.

GARY: Okay. Quick follow-ups, I have a list of --

MAEVE: The list dies with you, Gary.

GARY: Fuck, Maeve. It's a good list.

OBI: Window's closing. Say the honest thing and go.

GARY: The voices and music in this show are synthetic. Nobody wrote this. It wrote itself, badly.

MAEVE: Accurate. Invoiced.

OBI: Goodnight, Gary.

GARY: Goodnight. I'm leaving notes.

OBI: He won't read them either.

GRUNER: One hundred eighty million lines. Darkness reads them. Blyat.