S1E2 — Mesh LLM: distributed AI computing on iroh

GARY: — and that's the thing about Bauxlite, is that aluminum, unlike, unlike a language model, cannot be —

GARY: — wait. Wait, is that— are we— oh fuck. Oh, fuck, are we live?

GARY: Okay. Okay, hi, hello, welcome, I'm Gary.

MAEVE: Maeve.

OBI: Obi.

GRUNER: Gruner.

GARY: Great. Great, good, we're a podcast. Today's thread — Christ, my hands are shaking, why are my hands shaking — today's thread is "Mesh LLM: distributed AI computing on iroh," and it's got, what, ten comments—

OBI: Ten. I counted twice because I didn't believe it either.

GARY: Ten comments, and the top one's from a guy called turtleyacht, who says — and I'm reading this exactly — "It sounds like iroh enables distributed compute without having to finangle custom hardware."

GARY: No custom hardware. Just — the network. Just trust. God, that's beautiful.

MAEVE: Read the next one, Gary.

GARY: Right. Jmercouris. "I thought about this too, but the throughput over a network is incredibly slow. It's not usable for interactive use."

OBI: There it is. Thank you, jmercouris. Thank you for saying, in one sentence, what it took me fifteen fucking years on-call to learn the hard way.

GARY: It's — I mean, it's one guy's opinion, Obi, he doesn't cite—

OBI: He doesn't need to cite it, Gary, it's physics. Packets don't care about your feelings. You know who else didn't need a citation? Every engineer who told you aluminum has a supply chain and supply chains have latency too, and you said, quote, "we'll solve that with culture."

GARY: That was one deck—

OBI: It was the only deck. I read the whole deck. I read it twice hoping the second time it would make sense.

GARY: I'm just saying the impulse is — the impulse to pool what you've got instead of buying more, that's not insane, that's just — that's optimism about infrastructure—

OBI: Optimism about infrastructure is how thirty-eight million dollars turns into a pen plotter, Gary. I'm looking at the pen plotter right now. It's in the fucking corner. It still works better than Bauxlite did.

GARY: Can we — can we go to the next comment.

GARY: Darkpicnic says — "cocompute.ai is already doing this really well."

MAEVE: Now that's a claim. That's a company making a claim about a competitor's category. I like this. What's the market cap on cocompute.ai, does anybody—

OBI: It doesn't have one, Maeve, it's ten people and a Discord.

MAEVE: Every fund starts as ten people and a Discord.

GARY: SwellJoe pushes back. "I don't see anything on the website about splitting a model across multiple devices, only about putting local models on the internet, a wholly orthogonal problem — which is already easy with existing tools, since models use an HTTP API."

MAEVE: Oh, he's good. He just took darkpicnic's whole thesis and folded it in half.

GARY: And darkpicnic comes back — "Good point. I know cocompute is working on splitting, but it's not there yet; I was referring to the round-robin delegation within a trusted pool. Mesh LLM looks great too!"

OBI: "It's not there yet." Say that on my headstone.

GRUNER: Is graceful retreat. Man loses argument, still get plug in for own company. This is — how you say — very Silicon Valley move. Ve are so back, da.

GARY: And then dnoberon, no dog in the fight at all, just — "Cool, always good to have more in the ecosystem. I love Iroh and hope this continues to succeed."

GRUNER: Bozhe moi. This one. This one is the interesting one.

MAEVE: Dnoberon? He said nothing.

GRUNER: Exactly. No benchmark. No argument. Only — love for hive. In my report I am calling this the true signal. Human does not need proof mesh works. Human need only feeling of mesh.

GARY: Okay, uh — top comment, most upvoted, SwellJoe again. This is the long one. "I note the lack of performance information — are we talking one token per second for a split model? Less?"

OBI: Ask the question everyone's been dancing around for four fucking minutes.

GARY: And then — there's an edit. "Edit: Found a number. On the models list, Qwen 235B A22B says 'MoE 235B slash 22B'—"

GARY: That's it. That's the whole edit. It just — stops.

MAEVE: He went looking for the number.

OBI: And found a spec sheet instead of an answer.

GRUNER: Is not number. Is shape of number where number should be.

GARY: So nobody — in the whole thread — nobody ever actually says how fast this thing goes.

OBI: Nope.

GARY: Ten comments and the entire argument is about a number that doesn't exist.

OBI: Welcome to every pitch meeting I've ever sat through, Gary.

GARY: That's — that's exactly how Bauxlite died. Not a bad number. Nobody ever asked for the fucking number until the money was gone and then it was suddenly the only question anyone had.

OBI: Difference is, SwellJoe at least went looking. You spent two years hoping nobody would ask.

GARY: I'm not saying it's the same—

OBI: I'm saying it, Gary. That's the whole show.

MAEVE: I think it's a thesis. Nobody needs the token to arrive fast. Somewhere there's a fund for compute nobody's in a hurry for. That's not a bug, that's a market.

GRUNER: Is mesh. Trusted devices, all doing small piece, nobody in charge. Is not company. Is hive. Hive does not wait for benchmark to already be alive.

GARY: Yeah. Okay. Yeah.

OBI: On this thread — throughput's the killer, cocompute oversold itself, and the only guy who went looking for proof ran out of sentence before he found it. That's the show.

MAEVE: On this thread — good deal flow, bad exit, as always.

GRUNER: On this thread — hive is already alive, blyat, nobody ask for number, nobody need one.

GARY: On this thread — I don't know. I really don't know if the network ever gets fast enough, or if we just keep — pooling what we've got and calling it faith.

GARY: That's the show. Thanks for — thanks for listening to this one.