Needle 2: 14MB Agentic LLM for Edge Devices

Cactus has launched Needle 2, an ultra-compact 14MB agentic Large Language Model specifically engineered for resource-constrained edge devices. This model, which operates within 28MB of RAM, targets a vast market of over 21 billion connected IoT devices, including budget smartphones, wearables, smart home devices, and small robots, many of which lack dedicated NPUs or powerful GPUs. Needle 2 excels in tasks like tool calling, device interaction, and structured data extraction, achieving impressive decode speeds of up to 1,500 tokens/sec on devices like the Meta Quest 3S and Raspberry Pi 5. Its efficiency is attributed to a novel architecture based on Simple Attention Networks, consuming significantly less power per token compared to other small LLMs. The model can be fine-tuned quickly on a Mac or PC using an automated data-generation pipeline, allowing for custom task optimization. It also incorporates a learned confidence score, enabling developers to decide whether to act on the model's output or escalate to a larger cloud-based model for higher-stakes scenarios.