FitLLM

The best GPU or Mac to run GLM-5.2 locally

❌ No listed GPU runs GLM-5.2 at ~4-bit with 8K context

Computed with the open FitLLM engine — accurate per-layer KV-cache modeling, not a naive estimate. Updated 2026-07-10.

Ranked by memory size (a proxy for cost and availability), not price. Every figure is computed by the engine — these are a floor, not a guarantee; leave headroom for your runtime and OS.

Smallest GPU / Mac that fits, by setup

~4-bit ≈ Q4_K_M (GGUF, llama.cpp) on GPU / 4-bit (MLX) on Mac · ~8-bit ≈ Q8_0 / 8-bit · KV cache F16 · "full" = the model's max context (1M).

SetupSmallest GPUSmallest Mac
~4-bit · 8K ctx— 🔴— 🔴
~4-bit · full (1M)— 🔴— 🔴
~8-bit · 33K ctx— 🔴— 🔴
~8-bit · full (1M)— 🔴— 🔴

Every GPU and Mac, ranked by memory

HardwareMemoryMax context (~4-bit)Used @8K
RTX 3060 12GB12 GB❌ won't fit484 / 12 GB
RTX 4080 SUPER16 GB❌ won't fit484 / 16 GB
RTX 508016 GB❌ won't fit484 / 16 GB
RTX 309024 GB❌ won't fit484 / 24 GB
RTX 409024 GB❌ won't fit484 / 24 GB
RTX 509032 GB❌ won't fit484 / 32 GB
M5 Pro 48GB48 GB❌ won't fit402 / 48 GB
M5 Max 64GB64 GB❌ won't fit402 / 64 GB
M5 Max 128GB128 GB❌ won't fit402 / 128 GB

Why bigger isn't always needed — and smaller sometimes won't fit

GLM-5.2 uses MLA (Multi-head Latent Attention): K/V are compressed into a single low-rank latent (512 + 64 RoPE dims) shared across all heads — cached once, not per-head K and V. Naive "2 × heads × head_dim × layers" formulas over-count its KV cache by an order of magnitude. So GLM-5.2 has no single fixed memory requirement — it shifts with quantization and context. See the full breakdown.

▶ Open the interactive calculator

All numbers are computed by the open-source fitllm-engine (MIT) from official model config.json values — reproduce or audit them yourself. Estimates; real usage varies with runtime (llama.cpp / MLX / Ollama), driver and display. Found a mismatch? Report it. · FitLLM home