How to Deploy GLM 5.3 Locally

• Min VRAM at 4-bit470 GB · 8K
• Parameters744B
• Active params40B
• Max context1M

Architecture

MoE

Experts offloaded

14 GB VRAM + 457 GB RAM

License

GLM-5.3 License

Launch date

Aug 28, 2026

Vendor

Z.ai / Zhipu AI

Hugging Face repo

zai-org/GLM-5.3

GGUF repo

unsloth/GLM-5.3-GGUF

NVFP4 repo (Blackwell)

nvidia/GLM-5.3-NVFP4

MLX repo (Apple)

mlx-community/GLM-5.3-4bit

Layers

78

Hidden size

6,144

KV heads

64

Head dim

192

Full calculation rules are on the Methodology page.

Only common GPUs and systems are listed, one set for models under 50B and another for larger ones, sorted by price from low to high.

Device	Cards	Total price	Total VRAM	Speed t/s	Total power
DGX Spark 128GB128 GB per unit	4 units	$32,324	512 GB	7.6	960 W
Instinct MI21064 GB per card	8 cards	$39,952	512 GB	38.7	2,400 W
RTX PRO 5000 72GB72 GB per card	8 cards	$85,776	576 GB	32.8	2,400 W
A100 80GB PCIe80 GB per card	8 cards	$151,992	640 GB	44.2	2,400 W
RTX PRO 600096 GB per card	8 cards	$178,800	768 GB	41.6	4,800 W

Mac Studios link up over Thunderbolt 5 and run with MLX or exo; vLLM doesn't support them. Full calculation rules are on the Methodology page.

Detailed calculation rules are on the Methodology page.

As an eBay Partner, VRAMGlass may be compensated if you buy through the eBay links on this page. The price you pay stays the same.

1. 1

   Thinking can be tuned but not turned off

   Set thinking strength with reasoning_effort, choosing among low, high and max; the default is max, and unlike 5.2 it can't be turned off completely. For chat, set clear_thinking to true, or thinking from earlier turns will keep taking up context. Sampling parameters: temperature 1.0 and top_p 0.95 for general tasks; for long-running agent tasks, change top_p to 1.0.

2. 2

   The extra license clause

   The new clause applies only to “model as a service”: API providers with very large revenue must pass Zhipu's safety review before commercial use. Building the model's capabilities into a feature of your own product, or simply forwarding requests, doesn't count as model as a service; individuals and ordinary companies deploying it themselves are unaffected.

3. 3

   Framework versions

   The body of the vLLM recipe says 0.28.0 and up, but the page lists 0.29.0 as the minimum, so installing 0.29 or later is the safest bet. The official main repository has switched to FP8 weights, with BF16 in a separate GLM-5.3-BF16 repository. For llama.cpp with Unsloth's GGUFs, you need a build from July 24, 2026 onward (with the sparse-attention indexer). The glm-5.3 entry in the Ollama library only has a :cloud tag; it's a cloud version that doesn't run locally.

4. 4

   Quantization

   Unsloth recommends UD-IQ2_M among the 2-bit quants. By its tests, the dynamic 1-bit version picks the same top token as the original about 76% of the time, and 2-bit about 81%; the closest to the original are the 4-bit and 5-bit dynamic quants (UD-Q4_K_XL, UD-Q5_K_XL), with 4-bit at about 467 GB.

5. 5

   A chat template gotcha

   GLM's chat template uses a .{id}. syntax that many inference engines don't recognize. Unsloth changed it to [id] in its own version; if you hit template errors with weights from other sources, you can make the same change.

llama-server -hf unsloth/GLM-5.3-GGUF:UD-Q4_K_XL -c 8192

llama-server -hf unsloth/GLM-5.3-GGUF:UD-Q4_K_XL -c 8192 -ngl 99 \
  --cpu-moe

vllm serve zai-org/GLM-5.3 \
  --kv-cache-dtype fp8 \
  --tensor-parallel-size 8 \
  --speculative-config.method mtp \
  --speculative-config.num_speculative_tokens 5 \
  --tool-call-parser glm47 \
  --reasoning-parser glm47 \
  --enable-auto-tool-choice \
  --served-model-name glm-5.3

mlx_lm.server --model mlx-community/GLM-5.3-4bit

File names follow that week's model-library snapshot; the definitions are in the methodology. Methodology

Model	Total params	Active params	4-bit size
GLM 4.7 FlashMoE	30B	3B	18.3 GB
GLM 4.5 AirMoE	106B	12B	73 GB
GLM 5.3 FlashMoE	320B	18B	199.7 GB
GLM 5.2MoE	744B	40B	465.8 GB

About GLM