What LLMs can I run on a RTX 5090 (32GB)?

Computed with the open FitLLM engine — accurate per-layer KV-cache modeling, not a naive estimate. Updated 2026-07-10.

Every row is computed by the open FitLLM engine from the model's official config.json — sorted so models that fit come first, biggest first. ~4-bit ≈ Q4_K_M (GGUF) / 4-bit (MLX) · ~8-bit ≈ Q8_0 / 8-bit · KV cache F16 at 8K context. Lower quants free memory at some output-quality cost.

Every tracked model on the RTX 5090, ranked

Model	Params	~4-bit	~8-bit
Qwen 3.6 35B-A3B	35B 3B act	✅ up to 117K	❌ won't fit
Qwen-AgentWorld-35B-A3B	34.7B 3B act	✅ up to 120K	❌ won't fit
Gemma 4 31b	30.7B	✅ up to 67K	❌ won't fit
GLM-4.7-Flash	30B 3B act	✅ up to 105K	❌ won't fit
Qwen 3.6 27B	27.2B	✅ up to 110K	❌ won't fit
Gemma 4 26b A4B	25.5B 4B act	✅ up to 229K	⚠️ won't fit
gpt-oss-20b	21B 3.6B act	✅ up to 131K	✅ up to 91K
Gemma 4 12b	11.95B	✅ up to 262K	✅ up to 262K
Llama-3.1-8B-Instruct	8B	✅ up to 131K	✅ up to 115K
Llama-3.2-3B-Instruct	3.2B	✅ up to 131K	✅ up to 131K
GLM-5.2	753B 40B act	❌ won't fit	❌ won't fit
gpt-oss-120b	117B 5.1B act	❌ won't fit	❌ won't fit

"Up to NK" = max context that fits at that quant. Click a model for its full memory breakdown on this hardware.

▶ Open the calculator with Qwen 3.6 35B-A3B on the RTX 5090

Why this list is different

Most "what can I run" guides use a naive formula that ignores sliding-window attention, hybrid/linear layers and MLA compressed KV — so they over- or under-estimate modern models by multiples. Every number above models the real per-layer architecture. See the methodology with reproducible receipts.

All numbers are computed by the open-source fitllm-engine (MIT) from official model config.json values — reproduce or audit them yourself. Estimates; real usage varies with runtime (llama.cpp / MLX / Ollama), driver and display. Found a mismatch? Report it. · FitLLM home