How to Deploy GLM 5.3 Flash Locally

• Min VRAM at 4-bit201 GB · 8K
• Parameters320B
• Active params18B
• Max context1M

Architecture

MoE

Experts offloaded

9 GB VRAM + 194 GB RAM

License

MIT

Launch date

Aug 26, 2026

Vendor

Z.ai / Zhipu AI

Hugging Face repo

zai-org/GLM-5.3-Flash

GGUF repo

unsloth/GLM-5.3-Flash-GGUF

NVFP4 repo (Blackwell)

nvidia/GLM-5.3-Flash-NVFP4

MLX repo (Apple)

mlx-community/GLM-5.3-Flash-4bit

Layers

45

Hidden size

4,096

KV heads

64

Head dim

256

Full calculation rules are on the Methodology page.

Device	Price	Speed t/s	Max context
Mac Studio M3 Ultra 512GBWhole system · 512 GB	$22,90013 listings	44.1	1M

Only common GPUs and systems are listed, one set for models under 50B and another for larger ones, sorted by price from low to high.

Device	Cards	Total price	Total VRAM	Speed t/s	Total power
Tesla V100 32GB32 GB per card	8 cards	$7,160	256 GB	47.1	2,000 W
Arc Pro B7032 GB per card	8 cards	$14,800	256 GB	34.3	1,840 W
DGX Spark 128GB128 GB per unit	2 units	$16,162	256 GB	16.9	480 W
Instinct MI21064 GB per card	4 cards	$19,976	256 GB	72.5	1,200 W
RTX A600048 GB per card	8 cards	$39,192	384 GB	41.5	2,400 W
RTX PRO 5000 72GB72 GB per card	4 cards	$42,888	288 GB	63.4	1,200 W
GeForce RTX 509032 GB per card	8 cards	$66,376	256 GB	76.9	4,600 W
A100 80GB PCIe80 GB per card	4 cards	$75,996	320 GB	80.7	1,200 W
RTX PRO 600096 GB per card	4 cards	$89,400	384 GB	76.9	2,400 W
GeForce RTX 4090 48GB48 GB per card	8 cards	No price	384 GB	51.4	3,600 W

Full calculation rules are on the Methodology page.

Detailed calculation rules are on the Methodology page.

As an eBay Partner, VRAMGlass may be compensated if you buy through the eBay links on this page. The price you pay stays the same.

1. 1

   Which quantization to pick

   Unsloth measured how often each quant picks the same top token as the original model: about 78% for 2-bit (about 109 GB), about 86% for 3-bit (about 148 GB) and about 92% for 4-bit (about 200 GB). 3-bit is about 8 percentage points above 2-bit, so if you have the memory, we recommend at least 3-bit. But that 3-bit quant doesn't fit on a 128GB device; for 128GB devices, Unsloth's demo uses a smaller 3-bit quant, UD-IQ3_XXS (about 82%).

2. 2

   Mainline llama.cpp runs it now

   Mainline llama.cpp merged support on September 30, 2026 (PR #27773), covering both text and images. Unsloth updated its GGUF shards on October 5 to match the mainline changes, so stock llama.cpp now loads them and you no longer need their fork. MTP speculative decoding also landed in mainline on October 7 (PR #29928), which speeds up generation at long context. The glm-5.3-flash entry in the Ollama library only has a :cloud tag, which calls a cloud service rather than running locally.

3. 3

   vLLM limitations

   The vLLM recipe requires Hopper or newer NVIDIA cards and FlashInfer 0.6.17 or later, and on NVIDIA you need the vllm/vllm-openai:glm53-flash image for now. On Hopper, this model doesn't support storing the context cache in FP8. The official main repository holds FP8 weights; BF16 lives separately in GLM-5.3-Flash-BF16.

4. 4

   Thinking levels

   Usage is the same as for GLM 5.3: there are only three levels, low, high and max, with max as the default, and you should turn on clear_thinking for chat. Unsloth's parameters are temperature 1.0 and top_p 0.95 for general tasks, and temperature 0.95 and top_p 1.0 for coding tasks like DeepSWE.

5. 5

   Why long context takes almost no VRAM

   34 of its 45 layers use linear attention (the same kind as Kimi K3). These layers store only a fixed-size state that doesn't grow with context; only the cache in the other 11 layers grows. So at 4-bit, going from 32K to 1M adds only about 11 GB.

llama-server -hf unsloth/GLM-5.3-Flash-GGUF:UD-Q4_K_XL -c 8192

llama-server -hf unsloth/GLM-5.3-Flash-GGUF:UD-Q4_K_XL -c 8192 -ngl 99 \
  --cpu-moe

vllm serve zai-org/GLM-5.3-Flash \
  --tensor-parallel-size 4 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":5}' \
  --tool-call-parser glm47 \
  --reasoning-parser glm47 \
  --enable-auto-tool-choice \
  --served-model-name zai-org/GLM-5.3-Flash

mlx_vlm.server --model mlx-community/GLM-5.3-Flash-4bit

File names follow that week's model-library snapshot; the definitions are in the methodology. Methodology

Model	Total params	Active params	4-bit size
GLM 4.7 FlashMoE	30B	3B	18.3 GB
GLM 4.5 AirMoE	106B	12B	73 GB
GLM 5.2MoE	744B	40B	465.8 GB
GLM 5.3MoE	744B	40B	467.3 GB

About GLM