FitLLM

The best GPU or Mac to run Hy3 locally

❌ No listed GPU runs Hy3 at ~4-bit with 8K context

Computed with the open FitLLM engine — accurate per-layer KV-cache modeling, not a naive estimate. Updated 2026-07-10.

Ranked by memory size (a proxy for cost and availability), not price. Every figure is computed by the engine — these are a floor, not a guarantee; leave headroom for your runtime and OS.

Smallest GPU / Mac that fits, by setup

~4-bit ≈ Q4_K_M (GGUF, llama.cpp) on GPU / 4-bit (MLX) on Mac · ~8-bit ≈ Q8_0 / 8-bit · KV cache F16 · "full" = the model's max context (262K).

SetupSmallest GPUSmallest Mac
~4-bit · 8K ctx— 🔴— 🔴
~4-bit · full (262K)— 🔴— 🔴
~8-bit · 33K ctx— 🔴— 🔴
~8-bit · full (262K)— 🔴— 🔴

Every GPU and Mac, ranked by memory

HardwareMemoryMax context (~4-bit)Used @8K
RTX 3060 12GB12 GB❌ won't fit196 / 12 GB
RTX 4080 SUPER16 GB❌ won't fit196 / 16 GB
RTX 508016 GB❌ won't fit196 / 16 GB
RTX 309024 GB❌ won't fit196 / 24 GB
RTX 409024 GB❌ won't fit196 / 24 GB
RTX 509032 GB❌ won't fit196 / 32 GB
M5 Pro 48GB48 GB❌ won't fit167 / 48 GB
M5 Max 64GB64 GB❌ won't fit167 / 64 GB
M5 Max 128GB128 GB❌ won't fit167 / 128 GB

Why bigger isn't always needed — and smaller sometimes won't fit

Hy3 is a Mixture-of-Experts: ~21B of 298.8B parameters are active per token, but all 298.8B must sit in memory. Naive calculators that size memory off active params (or KV off all layers) get this wrong. So Hy3 has no single fixed memory requirement — it shifts with quantization and context. See the full breakdown.

▶ Open the interactive calculator

All numbers are computed by the open-source fitllm-engine (MIT) from official model config.json values — reproduce or audit them yourself. Estimates; real usage varies with runtime (llama.cpp / MLX / Ollama), driver and display. Found a mismatch? Report it. · FitLLM home