Can I run Hy3 on a M5 Ultra 96GB Mac?
Computed with the open FitLLM engine — accurate per-layer KV-cache modeling, not a naive estimate. Updated 2026-10-19.
Conditions: 8-bit · KV cache F16 · 32K context · macOS reserve (8.0 GB).
Memory breakdown (8-bit, F16 KV, 32K context)
| Model weights | 278 GB |
| KV cache | 10.0 GB |
| Runtime + macOS | 43.9 GB |
| Total used | 332 / 96 GB |
| Short by | 236 GB |
Context at 8-bit: does not fit at any context. Unified memory is shared with macOS and your apps — the recommended max leaves 20% of it free.
Every quantization on M5 Ultra 96GB
| Quant | Weights | Recommended max ctx | Hard limit | Used @32K |
|---|---|---|---|---|
| 4bit | 139 GB | won't fit | ❌ 176 / 96 GB | |
| 8bit | 278 GB | won't fit | ❌ 332 / 96 GB | |
| 16bit | 557 GB | won't fit | ❌ 644 / 96 GB | |
Recommended max = the longest context that still keeps the 20% headroom macOS needs · hard limit = the longest that fits at all (KV cache F16).
Lower quants free memory at some output-quality cost — 4-bit is the common sweet spot for local use.
▶ Open the interactive calculator (this exact setup)Embed this verdict
Live badge for your README or model card — recomputed by the engine at exactly these conditions, never stale:
[](https://fitllm.run/can-i-run/hy3-on-m5-ultra-96gb)
Or from your terminal (exit 0/1 — works as a pre-download guard):
npx fitllm "Hy3" --mac 96 --quant 8 --ctx 32768 --kv 16Why most calculators get this wrong
Hy3 is a Mixture-of-Experts: ~21B of 298.8B parameters are active per token, but all 298.8B must sit in memory. Naive calculators that size memory off active params (or KV off all layers) get this wrong.
Other options
same Mac Models that fit in 96GB: GLM-4.7-Flash, gpt-oss-20b, Qwen 3.6 27B, Qwen 3.6 35B-A3B, Qwen-AgentWorld-35B-A3B, Gemma 4 e2b, Gemma 4 e4b, Gemma 4 12b, Gemma 4 26b A4B, Gemma 4 31b, Llama-3.2-3B-Instruct, Llama-3.1-8B-Instruct, MiniCPM5-1B, Qwen3-0.6B, Qwen3-1.7B, Llama-3.2-1B-Instruct, Gemma-3-1B-it, Qwen 3.8 27B, Laguna XS 2.1, Spark-X2.5-4B, Granite-4.2-30B, MiniCPM5-2B, Nex-N2.5-mini.
Reproduce it
Open math: fitllm-engine (MIT), from official config.json.
All numbers are computed by the open-source fitllm-engine (MIT) from official model config.json values — reproduce or audit them yourself. Estimates; real usage varies with runtime (llama.cpp / MLX / Ollama), driver and display. Found a mismatch? Report it. · FitLLM home