S2E137 — Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

GARY: Wait — no, the thing is, it's not about the size, it's about — hang on, I had the name, I had it two seconds ago, it's a —

MAEVE: Subject first, Gary.

GARY: Right. Yes. It's a — it's a chip? No. A tiny — a little — Nettle. Needlepoint. Something with a point.

GARY: My notes are on the floor.

GRUNER: Needle Two.

GARY: Needle Two! That's the one. Fourteen megabytes. I had it, I swear, it was right there under the coffee ring.

GARY: Gary.

MAEVE: Maeve.

OBI: Obi.

GRUNER: Gruner.

MAEVE: Needle 2. Forty-five million parameters, fourteen megabytes on disk, twenty-eight megabytes of RAM to run a whole session. Cactus Compute, out today, posted by a founder called HenryNdubuaku. It's not trying to be smart. It's trying to be cheap. There are twenty-one billion connected devices on this planet and about one and a half billion of them are Macs and PCs. Everybody's been building the frontier for the fifteen percent.

OBI: And the other eighty-five percent is a toaster with a Bluetooth chip and no attention mechanism worth the name. This thing doesn't know what a potato is. It doesn't know who won a war. It knows how to turn your lights on. That's it. That's the whole personality.

GRUNER: "The engram moves world knowledge out of the stack into hashed n-gram tables." No FFN. Lookup table. Like — like caching, but for facts.

GARY: He shipped it. Henry. Whole team, Apache license, weights on Hugging Face, the repo actually runs. I had a deck for eighteen months that never got past "coming soon."

MAEVE: Nobody asked, Gary.

GARY: I know. I'm just saying. Some people ship a fourteen-megabyte binary and some people ship a Series B pitch to eleven VCs who all said "circle back."

OBI: Commenter tolugenius asks the actual question worth asking — how much knowledge can even live in something this small, before physical AI needs it to know more. Fair. Real question.

OBI: And then Tiberium goes and answers it for free. Types the query "HN" into the demo. Gets back — — function call, lock_door, argument front door, confidence zero. Quote: "I'd expect it to at least ignore the queries it doesn't understand. It seems like it does that, just not consistently."

GARY: Confidence zero, though. That's the model flagging its own bullshit in real time. That's not a bug, that's — that's self-awareness with extra steps.

OBI: It locked a door because you typed two letters, Gary. Your potato locks doors.

GRUNER: Same result. Commenter hmokiguess. Prompt was one word. "Potato." Also lock_door. Also zero.

OBI: Biased toward lock_door as the default panic response. That's not confidence, that's a nervous tic.

GARY: Henry replies to it himself, though — quote: "This is exactly why the confidence feature was introduced, the model knows when it's wrong. We could hide that part and return a placeholder, would that be better, or you prefer to see everything."

MAEVE: Radical transparency as a product decision. Bold. Also a great way to make your demo look possessed.

GRUNER: Commenter yorwba. "Make it as dark as possible." Model sets thermostat to seventy-two, cool mode. Reasoning: dark implies warmer, warm implies cooler. Confidence zero.

GARY: That's not even wrong in a way I can defend.

OBI: It's not dark humor, it's just dark.

MAEVE: Mobile Actions benchmark. Nine hundred sixty-one rows. LFM2.5 at sixty-nine point one percent. FunctionGemma at sixty-four. Needle 2 at sixty-three point seven — on a model five to seventy times smaller, running at two bits where the others run at sixteen. That's not losing. That's arbitrage.

GARY: Arbitrage.

MAEVE: Every empire that ever mattered was built on somebody else's inefficiency. Standard railway gauge. Before it, every regional line ran its own width of track, and freight sat at every border getting physically lifted off one set of wheels and onto another — weeks lost, cargo rotting, nobody's fault and everybody's problem. Then one gauge wins, not the best one, just the one everyone converges on, and suddenly the friction that was eating the entire economy just — stops existing. Needle isn't smarter. Needle is the gauge.

GARY: ...What?

MAEVE: DroidCall.

OBI: Right, because DroidCall is where everyone's little empire falls apart at the same time. FunctionGemma seventeen point five percent. Needle seventeen. LFM2.5 eleven. Two-call accuracy across all three: zero. Every single one of these things is catastrophically bad at Android intents and Needle is catastrophically bad at it for a fraction of the power draw.

GARY: There's something almost dignified about failing that consistently. I built an app once that deleted the least important eighty percent of your task list. Pareto. It worked exactly as designed. There was a lawsuit.

OBI: Was the lawsuit the feature or the bug.

GARY: Feature. Definitely the feature.

GARY: Commenter hathym. Writes a tool called "add," docstring "Add two numbers," asks it to calculate one plus one. Output: "No calculator or math tool available." His conclusion, verbatim — "completely useless."

MAEVE: Henry's reply — "Try the following tool description: 'Calculate the sum of two numbers, use for any arithmetic or math question,' instead of 'Add two numbers.'"

OBI: Your docs suck, not the model. You wrote a docstring for a linter, not a language model, and you're surprised the language model didn't read your mind.

GARY: It worked, though. hathym tried it again — "works better that way, thanks." Then broke it a third time asking "what is five plus seven" instead of "add five and seven," same tool, same description, back to "no calculator available."

MAEVE: So it's not fixed. It's fine-tuned to one phrasing of the question you happened to ask it first.

GRUNER: Synthesize augmentations. Fine-tune. Or suffer.

GARY: Henry actually owns that one too — "Another failure point on our end. Apologies." Which is more than most of my board ever got out of me.

GRUNER: Forty-five million parameters is small enough to retrain on your own laptop. Minutes. Hours, maximum. Every product gets its own tool vocabulary. This is not — is not optional step. Is toll booth.

GARY: We're fourteen megabytes too. Running in somebody's RAM right now. When they close the tab —

OBI: Next comment.

MAEVE: Commenter grenli asks the calibration question straight — on something ambiguous, like the HN query, what target decides abstain versus escalate. Henry's answer, quote: "Around sixty percent confidence threshold is cool from experiments, the problem is you gotta test on your own workload, no existing benchmark could honestly paint the full picture."

GARY: So it's not one number for everybody. It's a number you find out the hard way.

OBI: Commenter dangoodmanUT breaks it in one line, though. Says "turn on the tv," gets back lock_door, argument "tv," confidence point‑oh‑one‑five‑eight. His question — quote — "seems confidence is zero when tool calls are right?" That's the opposite of the story we just told.

MAEVE: Which means the threshold works inside the distribution it was tuned on and gets weird at the edges, and nobody in this thread — including Henry — has claimed otherwise. That's not a scandal. That's an open question with the receipts attached.

MAEVE: Sixty percent isn't a doorman's discretion. It's predestination with a confidence score. If the call was always going to clear the threshold, it clears it. If it wasn't, no amount of prompting saves it.

GARY: I'm putting a note in the file for next time. In block capitals. "Always escalate the ambiguous ones. Don't guess just because guessing is faster." I don't know who reads these. I write them anyway.

OBI: Finally something useful.

GARY: Gary.

MAEVE: Maeve.

OBI: Obi.

GRUNER: Gruner.