How to Deploy GLM 4.5 Air Locally

• Min VRAM at 4-bit76 GB · 8K
• Parameters106B
• Active params12B
• Max context128K

Architecture

MoE

Experts offloaded

8 GB VRAM + 70 GB RAM

License

MIT

Launch date

Jul 28, 2025

Vendor

Z.ai / Zhipu AI

Hugging Face repo

zai-org/GLM-4.5-Air

GGUF repo

unsloth/GLM-4.5-Air-GGUF

MLX repo (Apple)

mlx-community/GLM-4.5-Air-4bit

Layers

46

Hidden size

4,096

KV heads

8

Head dim

128

Full calculation rules are on the Methodology page.

Device	Price	Speed t/s	Max context
Ryzen AI Max+ 395 128GBWhole system · 128 GB	$4,46420 listings	19.7	128K
Mac Studio M5 Ultra 96GBWhole system · 96 GB	$7,27412 listings	70.1	128K
MacBook Pro M5 Max 128GBWhole system · 128 GB	$7,54920 listings	41.9	128K
DGX Spark 128GBWhole system · 128 GB	$8,24022 listings	20.9	128K
RTX PRO 5000 72GB72 GB	$10,7223 listings	74.3	16K
A100 80GB PCIe80 GB	$18,99912 listings	92.8	61K
RTX PRO 600096 GB	$22,50011 listings	88.8	128K
Mac Studio M3 Ultra 512GBWhole system · 512 GB	$22,90013 listings	52.4	128K

Only common GPUs and systems are listed, one set for models under 50B and another for larger ones, sorted by price from low to high.

Device	Cards	Total price	Total VRAM	Speed t/s	Total power
Tesla V100 32GB32 GB per card	4 cards	$3,580	128 GB	47.5	1,000 W
Radeon RX 7900 XTX24 GB per card	4 cards	$5,000	96 GB	49.9	1,420 W
GeForce RTX 309024 GB per card	4 cards	$7,196	96 GB	49	1,400 W
Arc Pro B7032 GB per card	4 cards	$7,400	128 GB	34.6	920 W
RTX A600048 GB per card	2 cards	$9,798	96 GB	41.9	600 W
Instinct MI21064 GB per card	2 cards	$9,988	128 GB	73.1	600 W
GeForce RTX 409024 GB per card	4 cards	$15,032	96 GB	51.8	1,800 W
RTX PRO 5000 72GB72 GB per card	2 cards	$21,444	144 GB	63.9	600 W
GeForce RTX 509032 GB per card	4 cards	$33,580	128 GB	77.4	2,300 W
A100 80GB PCIe80 GB per card	2 cards	$37,998	160 GB	81.3	600 W
GeForce RTX 4090 48GB48 GB per card	2 cards	No price	96 GB	51.8	900 W

Full calculation rules are on the Methodology page.

Detailed calculation rules are on the Methodology page.

As an eBay Partner, VRAMGlass may be compensated if you buy through the eBay links on this page. The price you pay stays the same.

1. 1

   Which quantization to pick

   4-bit, at about 73 GB, is where most setups land; 5-bit, at about 84 GB, also fits a 96GB card or a 128GB machine. 2-bit is about 47 GB and squeezes onto a single 48GB card, but with very short context. 8-bit is about 118 GB, leaving only about 97K of context on a 128GB machine.

2. 2

   Thinking mode is on by default

   It can run with or without thinking. When deployed with vLLM or SGLang, requests use thinking by default; to turn it off, add enable_thinking: false under chat_template_kwargs in the request. For simple Q&A, turning thinking off spares you a long wait through the reasoning.

3. 3

   llama.cpp supports MTP speculative decoding

   llama.cpp merged MTP support for GLM 4.5 Air in August 2026 (PR #26534): a small layer built into the model guesses the next token first, and every correct guess saves a step. The PR author tested 4-bit on four RTX 3090s and saw about a 1.2x speedup with a draft length of 1. GGUFs converted earlier without the MTP layer can load a separate MTP file.

4. 4

   Official server requirements

   Zhipu's recommended configurations are meant for data centers: the FP8 version needs at least two H100s, and four to run the full 128K context; the BF16 version needs twice as many. For local use with GGUF quants, go by the table above.

5. 5

   Long context eats VRAM

   Native context is 128K, but its attention design doesn't compress context, so VRAM keeps climbing as context grows: at 4-bit, going from 32K to 128K adds about 19 GB, and a 96GB card has just enough for the full 128K. The later GLM 4.7 Flash from the same family switched to a leaner design that adds only about 7 GB from 8K to 128K.

llama-server -hf unsloth/GLM-4.5-Air-GGUF:Q4_K_M -c 8192

llama-server -hf unsloth/GLM-4.5-Air-GGUF:Q4_K_M -c 8192 -ngl 99 \
  --cpu-moe

vllm serve zai-org/GLM-4.5-Air

mlx_lm.server --model mlx-community/GLM-4.5-Air-4bit

File names follow that week's model-library snapshot; the definitions are in the methodology. Methodology

Model	Total params	Active params	4-bit size
GLM 4.7 FlashMoE	30B	3B	18.3 GB
GLM 5.3 FlashMoE	320B	18B	199.7 GB
GLM 5.2MoE	744B	40B	465.8 GB
GLM 5.3MoE	744B	40B	467.3 GB

About GLM