Zen Nano: 0.6B Edge AI Model
Zen Nano is a 0.6B on-device AI model built for mobile and embedded deployment, achieving 44K tokens/sec on M3 Max with a 40K context window.
Today we are releasing Zen Nano, a 0.6B parameter model purpose-built for on-device inference. It runs on mobile SoCs, microcontrollers, and embedded systems where larger models simply cannot go.
Zen Nano is part of the Zen MoDE (Mixture of Distilled Experts) family, distilled from larger Zen models with the explicit goal of preserving capability at the smallest possible footprint.
Performance on Edge Hardware
On Apple M3 Max with MLX, Zen Nano reaches 44,000 tokens per second -- fast enough for real-time autocomplete, sub-second classification, and responsive on-device chat without any server round-trip.
| Hardware | Tokens/sec | Context Used |
|---|---|---|
| Apple M3 Max (MLX) | 44,000 | 40K |
| M2 MacBook Air (MLX) | 28,000 | 40K |
| Snapdragon 8 Gen 3 (GGUF Q4) | 12,000 | 8K |
| Raspberry Pi 5 (GGUF Q2) | 800 | 4K |
The 40K token context window is unusually large for a model this size. Most sub-1B models cap out at 2K-4K. Zen Nano holds enough context to process a full short document, a complete conversation history, or a dense code file in a single pass.
Formats
Zen Nano ships in formats designed for every edge deployment target:
| Format | Size | Use Case |
|---|---|---|
| GGUF Q2_K | 0.4 GB | Microcontrollers, minimal RAM |
| GGUF Q4_K_M | 0.6 GB | Mobile devices, Raspberry Pi |
| GGUF Q8_0 | 0.9 GB | High-quality CPU inference |
| GGUF F16 | 1.1 GB | Development and testing |
| MLX | 0.7 GB | Apple Silicon native |
The MLX format is the recommended path for Apple Silicon. It uses Apple's ML framework directly, with no GGUF overhead and hardware-accelerated matrix operations on the GPU and Neural Engine.
What It Is Good For
Zen Nano is not trying to replace a 70B model. It is built for tasks that need to run locally, fast, with no network dependency:
- Real-time autocomplete: Finish sentences, fill forms, generate snippets
- On-device classification: Intent detection, sentiment, category tagging
- Offline assistants: Voice-activated queries with no cloud fallback
- Embedded tooling: CLI tools, editor plugins, shell integrations
- Privacy-sensitive workloads: Medical, legal, and personal data that cannot leave the device
For agentic and complex reasoning tasks, Zen Nano pairs well with a larger Zen model on the same device or in the cloud -- Nano handles fast, cheap inference for simple operations, and routes hard queries up the stack.
Part of the Zen MoDE Family
Zen Nano is the smallest model in the Zen MoDE lineup. Zen MoDE (Mixture of Distilled Experts) is our approach to building the full capability spectrum from edge to cloud using a common distillation methodology.
The full family spans from Zen Nano at 0.6B to Zen Max at 671B. Each model is distilled from a larger upstream, retaining the reasoning patterns and instruction-following quality of larger models in progressively smaller footprints.
Get Zen Nano
- HuggingFace: huggingface.co/zenlm -- all formats available
- Zen LM: zenlm.org -- benchmarks and integration guides
- Hanzo Desktop: One-click install, runs entirely locally
Zach Kelling is the founder of Hanzo AI, Techstars '17.
Read more
Zen Omni: Unified Multimodal AI
Zen Omni is a 30B MoE unified multimodal model with Thinker-Talker architecture, handling text, vision, and audio in a single model with real-time speech-to-speech at under 300ms latency.
Zen Designer: 235B Vision-Language Model
Zen Designer is a 235B MoE vision-language model with 22B active parameters, supporting image analysis, video understanding, OCR in 32 languages, and native design reasoning.
Zen Max: 671B Reasoning Model
Zen Max is a 671B MoE reasoning model with 384 experts, 256K context, and abliterated base weights -- achieving AIME 2025 99.1%, SWE-Bench 71.3%, and BrowseComp 60.2%.