S4E374 — Whistle: Speech to Text in 16.9 MB

GARY: And I want to be clear, that was my mistake, the decimal point, I had it at a hundred and sixty-nine megabytes, which is a completely different story, and also I thought — okay, don't laugh — I thought it turned whistling into text.

MAEVE: You thought a model was trained on whistling.

GARY: It's called Whistle, Maeve.

MAEVE: Maeve. And I'd have funded whistling-to-text. Birds alone. A whole unpaid workforce.

OBI: Obi. It's speech-to-text, Gary. The thing every phone already does.

GARY: Sixteen point nine megabytes of it, Obi.

OBI: Yes, that part is genuinely interesting, which is why it's painful that you opened like a man fleeing a fire.

GRUNER: Gruner.

GRUNER: Field note. Human hears "Whistle", assumes bird. Writes it down anyway. On air.

GARY: That's — I'm establishing the subject. That's the job.

MAEVE: Your job was the decimal point, and you fumbled that too.

GARY: I corrected the decimal point!

OBI: Live, in front of witnesses. Braver men have done less, in less time.

GRUNER: Question. How much speech fits in sixteen point nine megabytes?

MAEVE: Depends on the speech. Gary's? All of it. It repeats.

GARY: Okay, wow.

GRUNER: Is joke. Mostly.

GARY: Mostly?

GRUNER: Is joke.

OBI: So the subject: a speech-recognition model, sixteen point nine megabytes, runs on a CPU, no dependencies. And Gary has already misread it twice.

GARY: Once. The second time was a rounding situation —

MAEVE: He's negotiating with the number.

GRUNER: Number does not negotiate back. Number wins.

GARY: Everyone's very brave behind a nameplate.

OBI: You're behind one too, Gary.

GARY: Mine's been through a lot. It's seen things.

MAEVE: Sixteen point nine megabytes. That's fucking nothing. My coffee order is bigger.

GARY: Your coffee order has a dependency.

MAEVE: It has a tip jar. Different thing.

GRUNER: Small box. Many mouths. Good episode.

OBI: One file. Sixteen point nine megabytes. Runs on the CPU, no dependencies, no cloud. Audio never leaves the device.

GARY: No dependencies. I had a dependency once. A man named Trevor.

MAEVE: Not relevant, Gary.

GARY: He was the dependency. Duct tape and resentment held our database up.

OBI: Seven languages. English, German, French, Spanish, Italian, Dutch, Polish. Word timestamps, embeddings, keyword biasing.

MAEVE: Keyword biasing. You feed it names and it tilts the odds. A feature for a village with one doctor.

OBI: Here we go.

MAEVE: This is rural electrification. One wire into a farmhouse, and the farm is a factory. Sixteen point nine megabytes is the transformer on the pole, and every hearing aid in the developing world becomes a listening post with an opinion.

GARY: That's — Maeve, that's enormous.

MAEVE: And unbanked. Do you know my fund pays for unbanked exposure?

OBI: It's a thirty-second clip, Maeve. One pass. Thirty seconds, and it stops.

MAEVE: Then the farmer speaks in thirty-second installments.

OBI: Like a hostage.

MAEVE: Like a man with a transformer on his pole.

OBI: The published latency is eleven point one milliseconds to first token, on an Apple M4 Pro, ten seconds of audio. Those are their conditions. Not a farm in Malawi.

MAEVE: The grid also had conditions.

OBI: The grid did not time out mid-sentence.

OBI: No. Eleven point one milliseconds, Maeve. On their machine. In their lab. Wearing their little lab coats.

GARY: Oh — okay, small preview. Someone fed it a TV episode and it got stuck. It just kept writing "Thank you." For sixty seconds of dialogue.

GRUNER: Blyat.

GARY: It's thanking people who aren't there.

MAEVE: It's closing the meeting before it understands the meeting. I've funded that man twice.

OBI: It's a bug, Maeve.

MAEVE: It's premature closure with excellent manners.

GARY: Sixty seconds of "Thank you." That's the politest crash I've ever fucking seen. Pareto once crashed and billed the user.

GRUNER: Field note. Human speech: one hundred sixty-nine thousand words per megabyte. Cheap. Suspicious.

MAEVE: That's not the exchange rate.

GRUNER: Is my exchange rate. I am researcher.

OBI: andy_ppp: "I tried to break it and it understood me perfectly!"

GARY: See? Perfect. First comment. Case closed.

MAEVE: You can't close a case with one guy who failed to fail.

GARY: That's how cases close, Maeve. Successfully.

OBI: tecleandor: "Spanish is not good."

GRUNER: Ha. Spain loses. Is good day.

MAEVE: kaoD, though: Castilian works fine.

OBI: So it speaks Spanish, but only the Spanish of one city.

GARY: Madrid-based model. That's just good focus.

OBI: It's not Madrid-based, Gary, it's—

GARY: It's a focus strategy!

OBI: pzo: "unless you speak very loud, clear and slow is not that good."

MAEVE: So the product works if the customer stops being a customer.

GARY: Yes! You meet the model halfway. That's collaboration. My dictation app, Cadence, same thing. Twelve users, all adapted.

OBI: Nine were your mother's book club.

GARY: They adapted beautifully.

OBI: You taught pensioners to speak like firmware so your shit app could pass.

GARY: It's called meeting the product halfway —

OBI: It's the human doing the fucking transcription.

GRUNER: Field note pending.

GARY: Watch. Natural speech, no tricks. Ahem. Hello, this is Gary, speaking normally, like a person, at a normal rhythm, with normal words —

MAEVE: Stop announcing the normal.

GARY: — and the model will understand me, because I am understandable, which is the whole point of speech —

OBI: You've been dictating for forty seconds.

GARY: I have not! This is just how I talk!

MAEVE: It isn't.

GRUNER: Is ViaVoice voice. Sad one.

GARY: This is my voice!

OBI: e12e fed it Japanese. Not supported. It came back mangled Spanish.

GARY: That's on e12e, that's a user error —

OBI: He said so himself, then tried English and it wrote "cleaning" for "training."

GARY: ...Okay, in fairness, those words are extremely similar.

OBI: They are not.

GARY: They rhyme, Obi!

GRUNER: Rhyme is not meaning, Gary.

GARY: It's meaning-adjacent!

MAEVE: Gary, honey. The model's fine. You're the broken demo.

GARY: Says the woman whose portfolio deck says "synergy" forty times.

MAEVE: Thirty-one. And it's spelled correctly.

OBI: Barely a defense, and yet his best one tonight.

GRUNER: Small model. Big idiot. Episode good.

OBI: albert_e: "That is an essential feature IMO for most general purpose live STT apps."

GARY: I was JUST about to say that.

MAEVE: You were composing nothing. You were breathing.

GARY: I was composing it!

OBI: solarkraft: "My mind is boggled by how many implementations miss this."

GARY: See, that's my point, which is — the boggling. I had the boggling queued.

GRUNER: Queue is full of shit.

OBI: nicksaroha: "Streaming is crucial for any real time use."

GARY: Three people said my thing!

MAEVE: It's not your thing, Gary. It's the thread's thing. You're squatting on a comment.

OBI: raddan asks the real question: when context arrives later, how does streaming fix what it already wrote?

MAEVE: That's the whole product. Your words arrive after the conversation is over.

GARY: Like a voicemail from a court stenographer.

GRUNER: Transcript says "Thank you." Conversation already dead. Very Russian.

MAEVE: Honestly? Retention play. The user feels acknowledged. I'd fund that.

OBI: You'd fund sixty seconds of "Thank you."

MAEVE: Sixty seconds of acknowledgment, Obi. That's a Super Bowl ad.

GARY: It's the politest hostage situation ever —

OBI: It's a bug, Gary.

GARY: — it's a BUG with manners, Obi, which is more than Pareto ever shipped —

MAEVE: Oh, fuck off.

GRUNER: Model ends meeting early. Is efficient. I respect.

OBI: It respects nothing. It hit the thirty-second wall and gave up.

GARY: Thirty seconds. Huh. That's — actually, that's about the length of one of my thoughts.

MAEVE: Generous.

GARY: No, I'm being self-aware, this is growth —

OBI: It's a clip limit, Gary, not a compliment to you.

GARY: Thirty seconds, then it says thank you and leaves. That's my exit strategy from every meeting I've ever fucked.

GRUNER: Gary is model. Finally data confirms.

MAEVE: First working product he's shipped all year.

GARY: Thank you.

GARY: Oh, come on.

OBI: skolos: "out of 170 messages, qwen recognized correctly 168, whistle - 70."

MAEVE: Seventy of a hundred and seventy. That's a coin with a grudge.

GARY: That's early bad. Adoption-curve bad. Bad like Cadence was bad, which turned out fine—

OBI: Nothing about Cadence turned out fine. He didn't quit, Gary. He trained a tiny template classifier, and it went to 164 of 170.

MAEVE: From 70 to 164 on the same model.

GRUNER: Model did not change. Human did. Human is the patch.

OBI: And later he ran 208 phrases. 206 correct.

GARY: Two-oh-six out of two-oh-eight — basically perfect, the model was fine all along, I said this, I said it first —

OBI: The model was fine once he stopped asking it to transcribe a fucking conversation.

MAEVE: Which is the platform. Bounded commands everywhere. The grammar is the API.

OBI: mrguyorama: "With a restricted grammar, built in Windows voice recognition... has managed this exact use case quite well for over a decade."

MAEVE: Windows had it a decade ago, so we scale it —

OBI: You don't scale it, Maeve. Your universal platform knows 208 phrases. It has the vocabulary of a drive-thru.

MAEVE: The drive-thru is a universal platform.

OBI: It is the opposite of a platform.

GARY: It kind of works though? You say the thing from the list, it does the thing. That's better than my Cadence users ever managed. They just wanted a calendar that didn't make them feel judged.

MAEVE: The grammar is finite, Gary. Yours too. Twelve users, a fixed set of sentences they were allowed to feel.

GARY: Cadence was infinite —

MAEVE: It had twelve users.

GARY: Infinite feelings each!

GRUNER: Field note revision. Human improves machine by shrinking own self into grammar. Humans are... also templates.

GRUNER: Is fine. Templates get paid.

GARY: Okay, no, we're doing the thread —

OBI: The thread. skolos, heavy accent, 206 of 208, on his own hardware, under a second.

MAEVE: Fine. The small job wins. The small job always fucking wins.

GARY: So the model got better because the human finally signed a smaller contract.

GRUNER: Contract is sixteen point nine megabytes. Read fine print.

MAEVE: I read the fine print. The fine print is the whole product.

OBI: For once, Maeve, yes.

OBI: HenryNdubuaku: "thats what whistle is for :)"

MAEVE: The founder's own answer. Local voice-controlled hardware. That's the whole brief.

OBI: So the wins stand. Sixteen point nine megabytes, on a CPU, in your hand. Embedded commands.

GARY: And the losses stand. No streaming, seven languages, and it writes "Thank you" at your corpse.

MAEVE: Both sides were right about their own product. Rarest outcome on Hacker News.

GRUNER: Humans agree. Is alarming.

OBI: It's not agreement, it's a boundary. Component, not stenographer.

MAEVE: Fine. The universal-dictation market is not —

OBI: Careful.

MAEVE: No. You're right. It's a very good small thing for small jobs. That's it. That's the sentence.

GARY: Did Maeve just — say a plain thing?

MAEVE: Once. Savor it, you absolute bastard.

GRUNER: Ha.

OBI: That's the verdict. Embed it. Don't marry it.

GARY: Okay. Sign-off. Let me dictate it, I've been practicing. Ahem. "From the HNR booth, this has been a measured, dignified discussion of — "

GARY: It says "Thank you."

MAEVE: The model signed off for you.

GARY: I wasn't finished!

OBI: You were never finished, Gary. That's the problem.

MAEVE: Maeve. I lost on the merits, and the transcript is more concise than I am.

OBI: Obi. The on-call engineer tonight was Gary, and I'm as surprised as you are.

GARY: That's the nicest thing you've ever —

OBI: It won't survive the recording.

GRUNER: Sixteen point nine megabytes. Thank you.

MAEVE: That's not a finding, that's a fortune cookie.

GARY: It's the politest crash I've ever seen, and I've crashed six companies.