// HACKER NEWS — CYBERSECURITY
Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
Today we release Needle 2: an open 45M-parameter model for tool calling, device use and structured extraction. The whole model is a single 14MB binary that runs in 28MB of RAM. It is built on our Simple Attention Network, compressed to CQ2-bit with Cactus Quants, and baked into its own engine.
On the tool call and mobile device use benchmarks, Needle 2 trades wins with other small models like FunctionGemma 270M, LFM2.5 230M and Apple FM, despite being 5× to 70× smaller, and running at 2 bits against their f16. Needle reaches:
With a peak session RAM around 28MB, Needle runs on newer microcontrollers like ESP32-S3.
Bringing On-Device AI to <$200 Devices: Edge AI has lately meant Macs and PCs, but the true edge is mostly cheap hardware: there are more than 21 billion IoT devices against roughly 1.5 billion PCs. Most phones in emerging markets ship under $200. Count budget phones, Raspberry Pis, microcontrollers, wearables, small robots like Reachy Mini, and connected home devices - and roughly four in five edge devices cost under $200. That is the hardware Needle targets: no GPU, no NPU, a few dozen MB of RAM.
Function Call & Device Use: Turning on a light does not need a frontier model. Smartwatches, home assistants and robots already expose their abilities as functions with typed parameters, so the only hard part is mapping a messy sentence onto them: which function, and which arguments. Framed that way, the problem needs no world knowledge and no open-ended prose. That is why 45M parameters suffice whereas chat requires billions. That smaller formulation is the bet everything else follows from.
Extraction & Structured Outputs: Needle treats extraction as another form of tool calling. With a schema and a document, it returns typed fields: enums for classification, arrays for lists, and objects for structured records. We compile grammar from the schema, preventing malformed JSON and invalid structures. This way, the model focuses its 45M parameters on choosing the right values and grounding them in the user's words.
Edge-Cloud Collaboration: No small model is perfect, and Needle says so instead of guessing. Off-topic requests return an empty call, and every response carries a learned confidence score. Set your own confidence threshold: act above it, ask again, or escalate to the cloud below it. Most device use requests can be handled locally, so escalation is rare and the default path remains private, fast, and free.
Lossless 2bit Quantization: Small models break under post-hoc quantization, so we never quantize post-hoc: Needle 2 trains against Cactus Quants from pretraining through post-training – weights, activations, and KV cache alike. The 2bit model you deploy is the model that was trained. That is what fits 45M parameters into 14MB with nothing lost on our benchmarks.
Co-designed Model & Inference: Every architectural element was benchmarked on the target hardware before it earned its parameters. This is why we don't just ship the weights – we package a single dependency-free C++ binary that probes the CPU at startup and picks its kernels, with the model, tokenizer, and grammar compiler sealed inside. One artifact runs from Cortex-M to x86 to WebAssembly. There is nothing extra to install or to download.
Fine-tune on your Mac/PC: Every product has its own tool vocabulary, and a 45M model is small enough to retrain where it runs: the repo and python package tune and test on your own computer in minutes to a few hours. Ship a Needle that speaks your device's tools, not a generic assistant.