Can I run Hy3 on an RTX 5090 (32GB)?
Computed with the open FitLLM engine — accurate per-layer KV-cache modeling, not a naive estimate. Updated 2026-10-19.
Conditions: Q4_K_M · KV cache F16 · 32K context · Windows + display reserve (2 GB).
Memory breakdown (Q4_K_M, F16 KV, 32K context)
| Model weights | 170 GB |
| KV cache | 10.0 GB |
| Runtime overhead + reserve | 24.9 GB |
| Total used | 205 / 32 GB |
| Short by | 173 GB |
Context at Q4_K_M: does not fit at any context.
Every quantization on the RTX 5090
| Weight quant | Weights | Recommended max ctx | Hard limit | Used @32K |
|---|---|---|---|---|
| Q4_K_M | 170 GB | won't fit | ❌ 205 / 32.0 GB | |
| Q5_K_M | 198 GB | won't fit | ❌ 237 / 32.0 GB | |
| Q6_K | 228 GB | won't fit | ❌ 270 / 32.0 GB | |
| Q8_0 | 296 GB | won't fit | ❌ 346 / 32.0 GB | |
| FP16 | 557 GB | won't fit | ❌ 638 / 32.0 GB | |
Recommended max = the longest context that still keeps the 5% headroom · hard limit = the longest that fits at all (KV cache F16).
Lower weight quants free memory at some output-quality cost — Q4 is the common sweet spot; below that quality drops faster.
KV cache is F16 here (llama.cpp default). Drop it to Q8/Q4 (-ctk/-ctv) for more context.
Embed this verdict
Live badge for your README or model card — recomputed by the engine at exactly these conditions, never stale:
[](https://fitllm.run/can-i-run/hy3-on-rtx-5090)
Or from your terminal (exit 0/1 — works as a pre-download guard):
npx fitllm "Hy3" --gpu "RTX 5090" --quant Q4_K_M --ctx 32768 --kv 16Why most VRAM calculators get this wrong
Hy3 is a Mixture-of-Experts: ~21B of 298.8B parameters are active per token, but all 298.8B must sit in memory. Naive calculators that size memory off active params (or KV off all layers) get this wrong.
What fits on the RTX 5090 instead
same GPU Models that fit on the RTX 5090: GLM-4.7-Flash, gpt-oss-20b, Qwen 3.6 27B, Qwen 3.6 35B-A3B, Qwen-AgentWorld-35B-A3B, Gemma 4 e2b, Gemma 4 e4b, Gemma 4 12b, Gemma 4 26b A4B, Gemma 4 31b, Llama-3.2-3B-Instruct, Llama-3.1-8B-Instruct, MiniCPM5-1B, Qwen3-0.6B, Qwen3-1.7B, Llama-3.2-1B-Instruct, Gemma-3-1B-it, Qwen 3.8 27B, Laguna XS 2.1, Spark-X2.5-4B, Granite-4.2-30B, MiniCPM5-2B, Nex-N2.5-mini.
same model GPUs that run Hy3: 4× DGX Spark 128GB (512GB).
Reproduce it
Hy3 = 298.8B (21B active, MoE), 80 layers. The RTX 5090 has 32GB / 1792GB/s. Same math, open source: fitllm-engine. GGUF bpw from llama.cpp.
All numbers are computed by the open-source fitllm-engine (MIT) from official model config.json values — reproduce or audit them yourself. Estimates; real usage varies with runtime (llama.cpp / MLX / Ollama), driver and display. Found a mismatch? Report it. · FitLLM home