Edge0-120B brings large-model capability to local agentic and coding workloads with a latent-space sparse Mixture-of-Experts (MoE) architecture. Its 48 layers each contain 512 experts; every token activates 10 routed experts plus one shared expert, for 8.64B active parameters. The model supports a 256K context window and is trained natively in 4-bit. On a single RTX 5090 D 24 GB, the NVFP4 build reaches 135.5 tokens/s in agentic decoding and 4,357.1 tokens/s in 16K prefill. It scores 75.3 on Terminal-Bench 2.1.
A 120B agent model built for local inference.
Edge0 · Sparse MoE · 120B
Model overview
Edge0-120B combines latent-space MoE, long context and local inference. The NVFP4 weights take 32.96 GiB on disk. With expert residency enabled on an RTX 5090 D 24 GB, the build uses 23.02 GiB of VRAM and 23.40 GiB of host RAM.
135.5 tok/s Agentic decode
4357.1 tok/s 16K prefill
Test setup: RTX 5090 D 24 GB · single GPU · NVFP4
Figure 1. Model overview. Throughput was measured on a single RTX 5090 D 24 GB; 16K prefill uses a 16K-token input.
Benchmark performance
Six models across six benchmarks, shown on a common 0–100 scale. Higher is better. A Claude Opus 4.6 (Max) result was not reported for DeepSWE 1.1.
Agents and coding
DeepSWE 1.1
Terminal-Bench 2.1
Reasoning and knowledge
HLE
GPQA-Diamond
IFBench
AA-LCR
Figure 2. Results across six benchmarks. Missing results are omitted from the bars rather than plotted as zero. Data comes from the model materials supplied by the engineering team.
Inference speed
Throughput is measured in tokens per second and grouped by hardware and build. The figures in the model overview use the NVFP4 build on an RTX 5090 D 24 GB.
Full throughput results
The shaded row marks the NVFP4 build featured on this page.
| Hardware | Build | 4K prefill | 16K prefill | Chat decode | Agentic decode |
|---|---|---|---|---|---|
| RTX 5090 D 24 GB | Q3_K | 3236.7 | 4576.5 | 130.2 | 144.6 |
| RTX 5090 D 24 GB | NVFP4 | 2668.5 | 4357.1 | 124.9 | 135.5 |
| RTX 4090 24 GB | Q3_K | 2637.5 | 3551.1 | 95.0 | 104.9 |
| RTX Spark N1X 60 GiB | Q3_K | 987.0 | 1081.7 | 47.2 | 68.4 |
| RTX Spark N1X 60 GiB | NVFP4 | 996.4 | 1066.9 | 45.1 | 68.9 |
| Apple M2 Ultra (Metal) | Q3_K | 867.6 | 908.7 | 60.4 | 63.0 |
| 16 GiB VRAM + 20 GiB RAM | Q3_K | 430 | 778 | 62.4 | 73.3 |
| 16 GiB VRAM + 20 GiB RAM | NVFP4 | 361.1 | 619.7 | 51.3 | 59.9 |
Test conditions
The 16 GiB VRAM + 20 GiB RAM two rows also use an RTX 5090 D 24 GB, with the budgets capped through cgroup. They do not represent a different GPU. With expert residency enabled, the Q3_K build (27.98 GiB of weights) uses 22.93 GiB of VRAM and 18.40 GiB of host RAM. The NVFP4 build (32.96 GiB of weights) uses 23.02 GiB of VRAM and 23.40 GiB of host RAM. The experts held in VRAM and host RAM do not overlap, so these figures do not double-count the same experts.
Apple M2 Ultra measurements used Metal and do not imply that a macOS build is currently available.
Application demo
See Edge0 in local chat and coding, video production, 3D scene creation and PCB design. This video was recorded with Edge0-145B.
Model architecture
Latent-space sparse MoE based on Qwen3.8-Flash-Next · 48 layers · 512 experts · 10 routed experts per token
Figure 3. Edge0-120B model architecture and expert routing. The original white, green and gray diagram is preserved in full. Open it for a full-size view.
| Component | Edge0-120B |
|---|---|
| Total parameters | 120B total · 8.64B active parameters per token |
| Layers | 48 · hybrid attention with full attention and a sparse indexer every fourth layer |
| Experts | 512 per layer · 10 routed experts + 1 shared expert active per token · SwiGLU width 640 |
| Expert pool | 60.4B parameters · latent width ℓ = 1280 |
| PLE | 51.2B-entry n-gram embedding table · FP8 · memory-mapped |
| MTP head | 1.26B-parameter draft model with 512 experts for speculative decoding |
| Context | 256K · INT8 KV cache with a 32K resident window |
| Builds | Q3_K 27.98 GiB · NVFP4 32.96 GiB |
Left: the complete execution path, from token embeddings through 48 hybrid-attention layers, final normalization and lm_head. Right: an expanded layer. The attention block reads and writes in the full 2,560-dimensional space. Then D projects each routing group into the ℓ-dimensional latent space, where the router selects 10 of 512 experts. Each selected expert applies an ℓ → 640 → ℓ transform while the shared expert runs alongside it. Finally, U projects the merged result back to 2,560 dimensions.
Everything outside the expert pool remains 2,560-dimensional. D and U are shared within each layer, with one pair per layer. Only the 512 experts operate in latent space. Each expert therefore has a fixed 1,164,800-byte record. The engine keeps thousands of experts resident on the GPU and streams the rest from host memory. The resulting overhead scales with the number of resident experts, rather than the model’s total parameter count.
Getting started
Each model folder—Q3_K or NVFP4, Edge0-145B or Edge0-120B—runs with Edge0’s edge0-infer from the command line or in the Edge0 Chat desktop app. Keep the folder structure intact: edge0-infer reads model_name from model.json inside the folder.
Platforms and documentation
The build instructions cover NVIDIA GPUs on Linux or Windows (x64 or ARM64) with CUDA 13. Apple M2 Ultra benchmarks used Metal; they do not indicate an available macOS release.
Build and run from the command line
1. Build. Clone the repository, then build both components:
git clone https://github.com/Edge0-AI/Edge0.git edge0- edge0-strata engine: Follow “Build (Linux, NVIDIA)” or “Build (Windows, NVIDIA)” in edge0-strata/EDGE0.md.
- edge0-infer: Follow “Build” for Linux or “Windows” in edge0-infer/docs/getting-started.md.
2. Run a model. Pass one model folder, such as Edge0-120B-NVFP4, rather than a parent folder containing multiple models. On Windows, the executable is build\windows\bin\edge0-infer.exe.
build/cpu/bin/edge0-infer serve --model /path/to/Edge0-120B-NVFP4Once the weights load, the process prints Listening: http://127.0.0.1:8000 model=<name>, where <name> is edge0_120b. It then serves OpenAI-compatible and Anthropic-compatible endpoints:
curl localhost:8000/v1/chat/completions -H 'content-type: application/json' -d '{"model":"edge0_120b","messages":[{"role":"user","content":"hi"}],"max_tokens":64}'For an interactive session, run edge0-infer chat --model /path/to/Edge0-120B-NVFP4. See edge0-infer/docs/edge0-120b.md for more options.
Desktop app · Windows
Open Edge0 Chat on Windows and select a model folder, such as Edge0-120B-NVFP4, to run the model locally. To build the app from source, see the Edge0 Chat guide.
Acknowledgments and license
Thanks to the Qwen3.8-Flash-Next and Strata teams for sharing their work with the community. Edge0 extends the Strata engine with latent-space MoE support and is released under the Apache 2.0 license.