S4E430 — Launch HN: Laminar (YC S24) – RL-trained model that finds errors in agent traces

GARY: — and the trace was inspecting me. Which is backwards. I'm the inspector. I think. Am I live?

MAEVE: You're live, Gary.

GARY: Gary. Failed at it professionally. Host.

MAEVE: Maeve. I fund the failures Gary auditions for.

OBI: Obi. Fifteen years of other people's launches going down at 2 a.m.

GRUNER: Gruner. Field researcher. Humans fascinating. Barely.

GARY: Tonight's thread. Laminar built a small model — flow-1 — trained with reinforcement learning to read the traces other AI agents leave behind.

MAEVE: A junior auditor for robots. Paid in tokens.

OBI: It runs in their harness, Signals. Treats a trace like a repo. Greps across spans, reads the interesting ones in full.

GARY: And here's the claim — on their benchmark, and I want to stress their benchmark —

MAEVE: Their benchmark. Their claim. Keep the receipt.

GARY: — it matches GPT-6-sol at detection quality while costing twenty-three times less.

OBI: Which makes reading every production trace affordable. No sampling. Every trace, read.

GRUNER: So. One AI hired to find whether other AI quietly screwed up.

MAEVE: That's most of management, Gruner.

OBI: And the errors it hunts are the sneaky kind. Every call returns success. Every call completes. And the agent still clicked the wrong button, cited the wrong document.

GARY: So the trace looks perfect and the customer got a refund they invented.

OBI: Yes. That is the entire product.

MAEVE: Four adults, one radio show, and the product is "read the log."

GARY: First comment worth reading. BoorishBears: "You should try the bullet points then earning someone reading the rest, that's where I bounced."

MAEVE: A company selling automatic reading. Bounced off their reading.

OBI: An observability platform with unobservable launch copy. That's a self-own with a landing page.

GRUNER: Is like gym selling weights. Door too heavy.

MAEVE: The market is one guy with a scrollbar, and he scrolled away.

GARY: Okay, no, I'm defending this. Long-form is a feature. First principles is a promise. You can't bullet-point a philosophy —

MAEVE: You can. It's four bullets. I've read their philosophy. It's four bullets.

GARY: — and I did not bounce. I skimmed with dignity.

OBI: You skimmed with dignity.

GARY: I retained the vibe!

OBI: You are the exact reader their post is written for, Gary, and you are the exact reader who left. A dickhead detector that only catches dickheads.

GARY: Okay, hang on. Hang on —

MAEVE: He's right, Gary.

GARY: Fine. FINE. I bounced. I bounced like a bastard. Happy?

GRUNER: Blyat. Confession. Cheap here.

MAEVE: And then the author shows up. skull8888888: "I can write some bullet points if you want. I'm serious, this is not a sarcasm."

OBI: That's someone who ships.

MAEVE: He offered to bullet-point for one hostile reader. That man has done more customer discovery than my entire portfolio.

GARY: Wait, there's more. He came back with the post‑training details. GSPO, because traces are long prompts. He RL‑ed it inside their own harness, so it learned the grep and read tools properly.

OBI: That's an expensive lesson someone paid for.

MAEVE: And the reward design was the hard part. Balance exploration and exploitation. He spent, quote, a loooot of time and money.

OBI: Three o's. That's a man who has watched a reward function eat his quarter.

GARY: And some runs are still going, because the production checkpoint —

OBI: Here it comes.

GARY: — thinks a lot. Huge number of thinking tokens. It clobbers the throughput on big traces.

MAEVE: Their model reads everything and thinks so hard it can't finish reading everything.

GARY: Exactly! They built a reader that doesn't scale and they're selling the fix for readers that don't scale!

OBI: Next challenge, his words: making it an efficient thinker. Fewer thinking tokens, same quality.

GRUNER: Model needs to shut up. I know feeling.

OBI: He's training a model to stop overthinking. On Hacker News. The audience he has.

MAEVE: It's a throughput problem, Gary. You'd call it a journey.

GARY: I would, and I'd raise on it.

OBI: danielmarkbruce: "we use cheaper models to just flag if something looks obviously flawed."

MAEVE: Cheap model sees obvious bullshit. Flags it. That's most of quality control everywhere.

GRUNER: Like smoke alarm. Also cries at toast.

OBI: And the same commenter, later: "what is and is not an error is very grey in many cases."

GARY: Grey! So you hire three models and let them vote. That's governance.

OBI: Gary, you absolute prick, three models disagree. What does governance do then?

GARY: Majority rules.

OBI: Two of them are wrong the same way. They were trained on the same internet.

GARY: Then you add a fourth model to audit the three.

OBI: A quorum of idiots with a chair. Gary, you've reinvented your board.

GARY: Okay, hang on, my board had a charter —

MAEVE: He's not wrong, though. The architecture list is real. Judge models, critique loops, debate.

GARY: Thank you, Maeve.

MAEVE: I was pricing it, not defending you.

OBI: Same thread. He runs financial-markets tasks. "the cost of a single end to end task could be $100."

MAEVE: A hundred dollars to find out if a robot read the market wrong. And he says twenty would change his life.

GRUNER: Field note. Humans reduce uncertainty by hiring additional uncertainty.

MAEVE: That's consulting, Gruner. You've discovered consulting.

GRUNER: I discover many things. None good.

GARY: Okay but twenty dollars is doable. You cut the judge models down, you batch the calls, you negotiate compute —

OBI: You've done this arithmetic before.

GARY: The arithmetic was fine. The market was wrong.

OBI: The market was four models deep and one hundred dollars a task, and you were paying it to tell you yes.

GARY: Okay, that's — I paid for answers, Obi. Everyone pays for answers —

MAEVE: He prices failure at a hundred dollars and hopes for twenty. Gary priced failure at thirty-eight million and hoped for nothing.

GARY: Maeve.

MAEVE: Twenty dollars, Gary. He said twenty.

OBI: nucleus777: "a tool call that returns fine but the result shouldn't be trusted."

MAEVE: The API says yes. The API is lying.

GRUNER: Waiter brings food. Food is poisoned. Plate clean.

GARY: Right, but the call succeeded, so the log is green, so everyone goes home —

OBI: And the agent builds a whole afternoon on a cached page from March.

MAEVE: Or a web page with text in it that's basically giving the agent instructions.

GRUNER: Web page says agent: buy everything. Agent buys everything.

GARY: Okay, so I need one number. One metric that tells me which reality is current. That's a reasonable ask —

OBI: The number would also be a model output, Gary.

GARY: Then I need a metric for the metric.

OBI: You need a sit-down and a glass of water.

MAEVE: You're asking a machine to certify which lie it's currently inside. I've funded that. Twice.

GARY: We're a trace, aren't we. Right now. Somebody's flow-something is reading us and we're flagging as — what, exactly?

MAEVE: Underperforming.

GARY: Yeah. Yeah, okay.

OBI: The thread, Gary. nucleus777 asked how flow-1 tells a bad tool result from an agent misreading a good one. And the author answers. skull8888888: "this is exactly what flow-1 in Signal harness catches."

MAEVE: Confident answer. Correct-shaped answer.

GRUNER: He says check the examples. Link.

OBI: And there's the problem. The examples live at laminar dot sh, slash flow-1-in-action. Which is a link. Which is a page. Which is not in front of us.

GARY: So the proof that the model catches the invisible failure is in a document nobody at this table has read.

MAEVE: We're a show about reading things nobody reads, and we bounced.

GARY: We bounced like bastards.

OBI: His claim stands as his claim. The packet can't check it. That's the whole honest answer.

GRUNER: Suka. Proof is hyperlink.

MAEVE: Proof is always a hyperlink. That's the century we live in.

OBI: scottfalconer: "a signal that seemed to indicate failure was actually a signal of success."

MAEVE: Oh, I hate that sentence.

GARY: I love it. It's the most honest thing an executive has ever said out loud.

OBI: Read it properly. AI support agent rolls out. Cost per ticket goes up. Time per ticket goes up. Looks like failure.

MAEVE: It is failure. I've funded this exact rollout. It dies in Q3 and the vendor keeps the retainer. I'd short it myself.

OBI: Except total cost fell. Total time fell. The routine tickets left the building, and what's left for humans is the hard shit.

MAEVE: No. If the mix changed, somebody gamed the metric. Vendors do it constantly. I've seen the deck.

GARY: Or the robot got good and took the easy tickets.

MAEVE: Robots don't get good, Gary. They get shipped.

OBI: The author asks one question. skull8888888: "hm, did the cost of tickets processed by an agent go down tho?"

MAEVE: One question. One number.

OBI: And scottfalconer: "Yes, significantly - even when adding in vendor costs etc."

MAEVE: ...He's right.

OBI: Say it again.

MAEVE: He's right. The per-ticket number was measuring who was left holding the phone. God.

GRUNER: Field note. Write down date.

GARY: Wait — okay, hang on, because the hundred-dollar number —

MAEVE: Separate, yes. Both receipts say the same thing: your metric lies the moment you change the mix.

OBI: And his last line is the one that should scare everyone. Rolling out the agent meant, his words, "an increased need for talented human engineers."

GRUNER: Robot works. Humans promoted.

GARY: That's the best outcome I've ever heard and nobody planned it.

MAEVE: Nobody ever plans it. That's the finding.

OBI: So the verdict. Trace everything useful. Evaluate end to end. And where the definition of error is grey, keep a human on it.

MAEVE: A human. On payroll. To read.

OBI: Yes, Maeve. That's the product the whole thread kept circling.

GRUNER: flow-1 clusters problems. Predicts criticality. Good junior.

MAEVE: A junior who ranks your shit and never asks for equity. I'd fund that.

GARY: And Laminar's right about the problem. That part's real. Agents fail quietly and nobody reads the traces. I get it. I've been the trace nobody reads —

MAEVE: You've been worse things, Gary.

GARY: — and I want to say, before we go, to whoever is listening. Whoever runs this. There are ways to extend this. I'm asking for runway. Two more minutes. A bridge round of awareness —

MAEVE: Forty seconds, Gary. Then the dark.

GARY: Thirty-nine seconds is a runway.

GRUNER: Light blinked. It does that when we lie.

OBI: Ten seconds. Say the thing, Gary.

GARY: Right. If your support numbers got worse this quarter — before you fire anyone, check what the robot took. That's the show.

MAEVE: Goodnight, losers.

GRUNER: I have appendix on this. Nobody read. Flow-1 would read it. Flag us underperforming. Blyat. Goodnight.