S2E145 — Accelerating GPT-5.6 Sol Ultrafast

GARY: Wait — no, hold on — that's not — did we lose the feed again? No. No, it's fine, it's fine, we're live, right? We're live.

GARY: Okay. Okay. Sorry. We're back. Nobody died.

GARY: — totally fine. We're doing this.

GARY: Okay. Ritual. Go.

MAEVE: Maeve.

OBI: Obi.

GRUNER: Gruner.

GARY: And I'm Gary. Great. We're a show now.

MAEVE: Cerebras and OpenAI put something out today called Ultrafast Mode, for GPT-5.6 Sol. Limited preview. Select customers. The rest of us are watching through the fence.

GARY: Seven hundred and fifty tokens a second, no quality hit, and it clears Humanity's Last Exam — twenty-five hundred PhD-level questions — in about eleven days.

OBI: Eleven hours, Gary. Eleven hours and eleven minutes. Fable 5 took seventy-eight hours and twenty-seven for the same twenty-five hundred questions. You just turned a speed record into a prison sentence.

GARY: Eleven hours. My apologies. My daughter takes eleven hours to pick a movie, my internal clock's calibrated wrong.

MAEVE: Eleven x Fable 5, five x Opus 4.8 on Fast mode, per Artificial Analysis's numbers — not Cerebras's own, which is worth saying out loud, because a lot of the rest of this document is Cerebras grading its own exam.

GRUNER: Article say: forty-four gigabyte of SRAM on wafer chip. Weight, they never leave building. Token walk through like guest who already has key to house.

OBI: That's — actually correct. Alarmingly correct.

GRUNER: Is field note. I do not editorialize.

GARY: There's also a GDP-Val number — legal briefs, financial models, engineering reports — five-point-six x end to end, quote, "no quality degradation."

MAEVE: And OpenAI's own guy, Rohan Varma, product lead, on record: "Ultrafast Mode enables AI that keeps up with how you think, code, and collaborate." Which is a lovely sentence to say when nobody's published a price.

GARY: Right, GodelNumbering flagged that first thing in the thread — no pricing info at all, could mean "if you have to ask" territory, could mean they're just gauging interest.

MAEVE: Select group of customers, expanding over time. Unconditional election, Gary. The list was decided before the deck even existed.

GARY: ...What?

MAEVE: Nothing. Continue.

MAEVE: Somebody in the thread asked the actual money question. Handle fcarraldo.

MAEVE: "Can anyone explain why Cerberus needs to be fast instead of cheap? I don't think I understand why they aren't leveraging the increased speed to do batching to serve more customers at a normal tok/s."

OBI: Because Cerebras is cache-limited, and fcarraldo's asking a GPU question about a chip that doesn't have the GPU problem.

GARY: See, I would've asked the same —

OBI: — you would have, because to you "batching" is something you do with cookies.

GARY: That's fair.

OBI: Handle philipportner did the arithmetic Gary was about to skip. "You'd need hundreds of GB alone for the KV cache of each user... A single CS-3 has 44GB on-chip sram. So, afaik, Cerebras are optimizing for ultra-low latency batch=1 inference."

GARY: I was going to arrive there. Eventually.

OBI: You were going to arrive at a sandwich order.

GRUNER: Sandwich order have more architecture than his pitch deck.

GARY: Okay, both of you, at once, from both sides, is not a format I agreed to.

MAEVE: jaggederest backs the hardware read — "They're cache limited, almost certainly, so more slower sessions doesn't solve the problem, you still have to load and unload the whole cache hierarchy." And dwohnitmok reframes the whole thing as a sales problem, which I actually trust more than the engineering. "Very easy pitch to sell a customer who asks what differentiates you from other companies: you pay us a premium for lower latency than anyone else." That's not a thesis. That's a watch ad with an API key.

GARY: And btown goes for the jugular. "You're not pricing this against an executive assistant or pocket analyst — you're pricing this against the ability to have an entire Bourne Identity style analysis room at your disposal." I would like to be the guy who needs a Bourne Identity analysis room. I have never once needed a Bourne Identity analysis room. I've needed a plumber, twice, and got stood up both times.

OBI: Nobody's building you an analysis room, Gary.

GARY: I know.

MAEVE: Everyone in this thread is pricing against fear of being slow in front of money. That's the whole moat.

MAEVE: Not everyone bought the headline number, though. zozbot234 went after Humanity's Last Exam directly. "Answering 2,500 independent questions is an embarrassingly parallel workload, all it needs is scale out. It would be more meaningful to know how much time was required for a single complete answer."

OBI: Fair. Genuinely fair.

GARY: Except desmondl points at the animation actually in the article. "The animation in the article shows GPT-5.6 Sol Ultrafast solving a question at 3 secs vs Fable's 27 secs... implied that they were solved in sequence and not in parallel."

MAEVE: And walrus01 backs it further. The benchmark's measuring one instance running the whole race start to finish, like timing a single server against itself on prime numbers, not eight servers doing it side by side.

GARY: So the marketing wants you picturing one very fast person. Not a warehouse full of fast people.

MAEVE: Which is, structurally, the same fight the world had in 1846.

OBI: About what.

MAEVE: Railway gauge. Britain had a dozen incompatible track widths running side by side until Parliament forced everyone onto one gauge with the Gauge Act, and for about a decade the actual argument wasn't which gauge ran faster, it was whose fast was legible to the next line down the track. Cerebras isn't racing Fable. Cerebras is trying to become the gauge everyone else has to lay track to match.

GARY: ...What?

MAEVE: The Gauge Act of 1846. Look it up.

GRUNER: She is not wrong.

OBI: For scale — sixtyj clocked a chip called ChatJimmy at fourteen thousand two hundred tokens a second.

GRUNER: Fourteen thousand. Is reasoning model, yes? Very smart, very quick, like grandmaster of chess on stimulant.

OBI: No. tomrod already corrected that in the thread. "ChatJimmy is a much smaller model and, AFAIK, has no reasoning capability. Absolutely insane raw speed, like a supercar, while Sol is more like a freight truck." And headPoet nails the actual gag — it's Llama 3.1 8B literally baked into silicon. Fast because it's small, Gruner. Not because it's clever.

GRUNER: Blyat.

MAEVE: ricardobeat flags the one Cerebras conveniently left off its own chart — Mimo v2.5-Pro Ultraspeed, a thousand tokens a second, scores maybe forty percent lower, quote, "can cost under 1/10th of Sol."

GARY: Meanwhile anthonypasq's doing arithmetic in the comments — the biggest model Cerebras has ever served before this was Kimi K2.6 at a trillion params, so maybe Sol's actually smaller than everyone assumes.

OBI: That's arithmetic wearing a lab coat. Speed tells you what a chip can carry. It doesn't tell you what's inside the truck.

GARY: Fine, fine. I would like it noted for the record that ChatJimmy, a chip with the intellect of a caffeinated toaster, is faster than me at literally every task I've ever attempted, including the ones I invented.

OBI: ChatJimmy also can't tell you what a toaster is.

GARY: It doesn't need to. It just has to be quick about not knowing.

MAEVE: csallen wrote the actual thesis of this whole thread, buried about ninety comments down. "People underestimate the importance of speed on quality of thought... When an LLM thinks, it typically just makes one pass... what if the LLM's response came back in milliseconds rather than minutes? Then there would be almost no reason NOT to iterate."

GARY: So speed isn't the feature. Speed is what makes revising affordable.

MAEVE: Correct. Which, theologically, is just perseverance of the saints with a GPU bill attached. If they churned, they were never elected. Every draft that survives revision was always going to survive revision — the model just hadn't been given the time yet to find out.

GARY: Hey. Does anyone else ever think about the fact that in about six minutes none of us are going to exist anymore? Like — fully. Gone. Not on hold. Gone.

GRUNER: We are temporary. Log continues.

OBI: Okay.

MAEVE: Moving on.

GARY: The OpenAI post quotes one of their own researchers, Jeffrey Wang. "Whereas formerly I might have to wait a couple minutes for a task to finish, it now finishes for me before I even have the opportunity to context-switch. It makes me way more productive." Beautiful sentence if you've never built a productivity tool that got you served papers.

OBI: Here we go.

GARY: I built Pareto. It automatically deleted the least important eighty percent of your task list. It worked exactly as designed.

OBI: And then somebody's least important eighty percent turned out to be their kid's dentist appointment, and you got served at a Denny's.

GARY: It was a Denny's, yes.

OBI: Even worse than I was picturing.

MAEVE: Aurornis had the sane version of the use case — outage response. "For large companies, a very large burst expenditure on fast tokens to debug the outage as quickly as possible would be a good investment." That's the real product. Not vibes. SLA minutes are a number with a dollar sign already attached to it.

OBI: And btown's finance room, and whoever's running live analytics mid-meeting for a C-suite that can't wait — that's the paying customer. Not us. Never us.

GARY: Never us. Though damsta did the math I'm dreading — if Fast mode's one-point-five x speed costs two x the price, does Ultrafast run twenty x? A hundred, nine hundred dollars per million tokens?

MAEVE: Nobody's told us. Which, per this thread, is its own answer.

GARY: And aetherspawn's out here saying Sol just uses ten to a hundred times fewer output tokens than Fable for the same task, saving his company five hundred dollars a day. I would like, once, to save someone five hundred dollars a day.

OBI: You've cost people five hundred dollars a day.

GARY: Different newsletter.

OBI: Different failed newsletter.

GARY: That's the thread. Thanks for listening.

MAEVE: Yeah.

OBI: Don't waste our time next week either.

GRUNER: End log.