Needle2 Brings Powerful LLMs to Edge Devices

Cactus released Needle2, a 14MB agentic large language model (LLM) designed specifically for resource-constrained devices like smartphones, wearables, and microcontrollers. This breakthrough enables advanced AI capabilities on billions of edge devices worldwide.

Key Features & Performance:

• Tiny Footprint: The entire model is a single 14MB binary that runs in just 28MB of RAM
• Fast Inference: Decodes at 500 tokens/sec on a Raspberry Pi 5, and even faster on VR devices like Meta Quest 3S (up to 1,500 tokens/sec)
• Competitive Performance: Trades blows with larger models like LFM2.5 230M and Apple Foundation Model, while being 5-70x smaller
• Structured Output: Can generate structured data based on provided schemas, making it ideal for applications requiring specific output formats
• Tool Calling: Supports tool integration for real-world interactions (e.g., controlling smart home devices)

Technical Innovation:

Needle2 leverages Simple Attention Networks from Cactus’s research (https://arxiv.org/abs/2607.18363) to achieve remarkable efficiency. Unlike conventional transformers, Needle2 requires just 70 MFLOPs per token—a significant reduction compared to even the most optimized alternatives.

Applications:

• Always-on AI assistants for smartphones and wearables
• Smart home automation with local processing
• Edge robotics requiring compact, real-time intelligence
• Industrial IoT devices operating in remote or constrained environments

Needle2 represents a major step toward democratizing access to advanced AI by extending its reach beyond high-end computing platforms. The model can be further customized with just a few examples using Cactus’s Python package (https://github.com/cactus-compute/needle), enabling developers to create tailored AI experiences for their devices.