edge0

A 145B agent model on one consumer GPU.

Edge0 · Sparse MoE · 145B

November 13, 2026

Edge0-145B brings large-model capability to local agentic and coding workloads with a latent-space sparse Mixture-of-Experts (MoE) architecture. Its 48 layers each contain 512 experts; every token activates 10 routed experts plus one shared expert, for 10.12B active parameters. The model supports a 256K context window and is trained natively in 4-bit. On a single RTX 5090 D 24 GB, the NVFP4 build reaches 98.3 tokens/s in agentic decoding and 3,165.7 tokens/s in 16K prefill. It scores 86.5 on Terminal-Bench 2.1.

Model overview

Edge0-145B combines latent-space MoE, long context and local inference. The NVFP4 weights take 47.79 GiB on disk. With expert residency enabled on an RTX 5090 D 24 GB, the build uses 23.25 GiB of VRAM and 38.27 GiB of host RAM.

Layers48
Latent width1792
Experts per layer512 × 10 routed
Context length256K
Parameters · native 4-bit training145B

98.3 tok/s Agentic decode
3165.7 tok/s 16K prefill

Test setup: RTX 5090 D 24 GB · single GPU · NVFP4

BenchmarksResult
Terminal-Bench 2.186.5
DeepSWE 1.164.6
GPQA-Diamond90.4
AA-LCR79.3
16K prefill3165.7 tok/s
Agentic decode98.3 tok/s
Q3_K weights on disk40.81 GiB
NVFP4 weights on disk47.79 GiB

Figure 1. Model overview. Throughput was measured on a single RTX 5090 D 24 GB; 16K prefill uses a 16K-token input.

Benchmark performance

Six models across six benchmarks, shown on a common 0–100 scale. Higher is better. A Claude Opus 4.6 (Max) result was not reported for DeepSWE 1.1.

Edge0-145BEdge0-120BQwen3.8-Flash-NextQwen3.8-27BDeepSeek-V4-Flash-0731Claude-Opus-4.6 (Max)

Agents and coding

DeepSWE 1.1

05010064.6Edge0145B51.3Edge0120B58.7Qwen3.8Flash-Next42.2Qwen3.827B54.4DeepSeekV4-FlashNot reportedClaudeOpus 4.6

Terminal-Bench 2.1

05010086.5Edge0145B75.3Edge0120B83.3Qwen3.8Flash-Next73.0Qwen3.827B82.7DeepSeekV4-Flash78.2ClaudeOpus 4.6

Reasoning and knowledge

HLE

05010035.4Edge0145B31.0Edge0120B35.9Qwen3.8Flash-Next30.8Qwen3.827B33.8DeepSeekV4-Flash40.0ClaudeOpus 4.6

GPQA-Diamond

05010090.4Edge0145B87.4Edge0120B91.7Qwen3.8Flash-Next89.2Qwen3.827B90.8DeepSeekV4-Flash91.3ClaudeOpus 4.6

IFBench

05010080.6Edge0145B79.3Edge0120B81.3Qwen3.8Flash-Next79.5Qwen3.827B79.2DeepSeekV4-Flash62.5ClaudeOpus 4.6

AA-LCR

05010079.3Edge0145B81.0Edge0120B79.1Qwen3.8Flash-Next82.0Qwen3.827B79.7DeepSeekV4-Flash78.0ClaudeOpus 4.6

Figure 2. Results across six benchmarks. Missing results are omitted from the bars rather than plotted as zero. Data comes from the model materials supplied by the engineering team.

Inference speed

Throughput is measured in tokens per second and grouped by hardware and build. The figures in the model overview use the NVFP4 build on an RTX 5090 D 24 GB.

Full throughput results

The shaded row marks the NVFP4 build featured on this page.

HardwareBuild4K prefill16K prefillChat decodeAgentic decode
RTX 5090 D 24 GBQ3_K2169.93554.999.7121.2
RTX 5090 D 24 GBNVFP41839.63165.779.698.3
RTX 4090 24 GBQ3_K2042.03179.469.991.7
RTX Spark N1X 60 GiBQ3_K674.0770.337.736.7
RTX Spark N1X 60 GiBNVFP4821.2922.440.146.0
Apple M2 Ultra (Metal)Q3_K755.6800.360.957.0
16 GiB VRAM + 20 GiB RAMQ3_K24933536.844.9

Test conditions

The 16 GiB VRAM + 20 GiB RAM row also use an RTX 5090 D 24 GB, with the budgets capped through cgroup. They do not represent a different GPU. With expert residency enabled, the Q3_K build (40.81 GiB of weights) uses 23.14 GiB of VRAM and 31.28 GiB of host RAM. The NVFP4 build (47.79 GiB of weights) uses 23.25 GiB of VRAM and 38.27 GiB of host RAM. The experts held in VRAM and host RAM do not overlap, so these figures do not double-count the same experts.

No NVFP4 result is available under the 16 GiB VRAM + 20 GiB RAM budget. Apple M2 Ultra measurements used Metal and do not imply that a macOS build is currently available.

Application demo

See Edge0 in local chat and coding, video production, 3D scene creation and PCB design. The video was recorded with Edge0-145B.

Recorded with Edge0-145BOpen video ↗
Local chat and coding0:00 / 2:01

Model architecture

Edge0-145B

Latent-space sparse MoE based on Qwen3.8-Flash-Next · 48 layers · 512 experts · 10 routed experts per token

Edge0-145B architecture diagram showing the full model, an expanded layer, expert routing, model specifications and legend

Figure 3. Edge0-145B model architecture and expert routing. The original white, green and gray diagram is preserved in full. Open it for a full-size view.

ComponentEdge0-145B
Total parameters145B total · 10.12B active parameters per token
Layers48 · hybrid attention with full attention and a sparse indexer every fourth layer
Experts512 per layer · 10 routed experts + 1 shared expert active per token · SwiGLU width 640
Expert pool84.6B parameters · latent width ℓ = 1792
PLE51.2B-entry n-gram embedding table · FP8 · memory-mapped
MTP head1.26B-parameter draft model with 512 experts for speculative decoding
Context256K · INT8 KV cache with a 32K resident window
BuildsQ3_K 40.81 GiB · NVFP4 47.79 GiB

Left: the complete execution path, from token embeddings through 48 hybrid-attention layers, final normalization and lm_head. Right: an expanded layer. The attention block reads and writes in the full 2,560-dimensional space. Then D projects each routing group into the ℓ-dimensional latent space, where the router selects 10 of 512 experts. Each selected expert applies an ℓ → 640 → ℓ transform while the shared expert runs alongside it. Finally, U projects the merged result back to 2,560 dimensions.

Everything outside the expert pool remains 2,560-dimensional. D and U are shared within each layer, with one pair per layer. Only the 512 experts operate in latent space. Each expert therefore has a fixed 1,630,720-byte record. The engine keeps thousands of experts resident on the GPU and streams the rest from host memory. The resulting overhead scales with the number of resident experts, rather than the model’s total parameter count.

Getting started

Each model folder—Q3_K or NVFP4, Edge0-145B or Edge0-120B—runs with Edge0’s edge0-infer from the command line or in the Edge0 Chat desktop app. Keep the folder structure intact: edge0-infer reads model_name from model.json inside the folder.

Platforms and documentation

The build instructions cover NVIDIA GPUs on Linux or Windows (x64 or ARM64) with CUDA 13. Apple M2 Ultra benchmarks used Metal; they do not indicate an available macOS release.

Build and run from the command line

1. Build. Clone the repository, then build both components:

git clone https://github.com/Edge0-AI/Edge0.git edge0

2. Run a model. Pass one model folder, such as Edge0-145B-NVFP4, rather than a parent folder containing multiple models. On Windows, the executable is build\windows\bin\edge0-infer.exe.

build/cpu/bin/edge0-infer serve --model /path/to/Edge0-145B-NVFP4

Once the weights load, the process prints Listening: http://127.0.0.1:8000 model=<name>, where <name> is edge0_145b. It then serves OpenAI-compatible and Anthropic-compatible endpoints:

curl localhost:8000/v1/chat/completions -H 'content-type: application/json' -d '{"model":"edge0_145b","messages":[{"role":"user","content":"hi"}],"max_tokens":64}'

For an interactive session, run edge0-infer chat --model /path/to/Edge0-145B-NVFP4. See edge0-infer/docs/edge0-145b.md for more options.

Desktop app · Windows

Open Edge0 Chat on Windows and select a model folder, such as Edge0-145B-NVFP4, to run the model locally. To build the app from source, see the Edge0 Chat guide.

Acknowledgments and license

Thanks to the Qwen3.8-Flash-Next and Strata teams for sharing their work with the community. Edge0 extends the Strata engine with latent-space MoE support and is released under the Apache 2.0 license.