How to Deploy GLM 4.7 Flash Locally

• Min VRAM at 4-bit20 GB · 8K
• Parameters30B
• Active params3B
• Max context198K

Architecture

MoE

Experts offloaded

4 GB VRAM + 18 GB RAM

License

MIT

Launch date

Jan 19, 2026

Vendor

Z.ai / Zhipu AI

Hugging Face repo

zai-org/GLM-4.7-Flash

GGUF repo

unsloth/GLM-4.7-Flash-GGUF

NVFP4 repo (Blackwell)

unsloth/GLM-4.7-Flash-NVFP4

MLX repo (Apple)

mlx-community/GLM-4.7-Flash-4bit

Ollama tag

glm-4.7-flash

Layers

47

Hidden size

2,048

KV heads

20

Head dim

256

Full calculation rules are on the Methodology page.

Device	Price	Speed t/s	Max context
Tesla V100 32GB32 GB	$89518 listings	130.4	198K
Radeon RX 7900 XTX24 GB	$1,25029 listings	133.7	115K
GeForce RTX 309024 GB	$1,799107 listings	132.4	115K
GeForce RTX 409024 GB	$3,75868 listings	136.1	115K
Ryzen AI Max+ 395 128GBWhole system · 128 GB	$4,46420 listings	66.1	198K
RTX A600048 GB	$4,89916 listings	122.3	198K
Mac Studio M5 Ultra 96GBWhole system · 96 GB	$7,27412 listings	145.5	198K
MacBook Pro M5 Max 128GBWhole system · 128 GB	$7,54920 listings	110.5	198K
DGX Spark 128GBWhole system · 128 GB	$8,24022 listings	69.1	198K
GeForce RTX 509032 GB	$8,395123 listings	161.5	198K
RTX PRO 600096 GB	$22,50011 listings	161.5	198K

Only common GPUs and systems are listed, one set for models under 50B and another for larger ones, sorted by price from low to high.

Detailed calculation rules are on the Methodology page.

As an eBay Partner, VRAMGlass may be compensated if you buy through the eBay links on this page. The price you pay stays the same.

1. 1

   Which quantization to pick

   On a 24GB card, use 4-bit (about 18 GB); 5-bit, at about 21 GB, also fits, but with a good chunk less context. On a 16GB card, use 2-bit or 3-bit (about 12 GB and 14 GB). Unsloth recommends at least 4-bit.

2. 2

   Official sampling parameters

   For general tasks, temperature 1.0 and top_p 0.95; for coding and terminal tasks, temperature 0.7 and top_p 1.0. Turn the repetition penalty off (set it to 1.0). In llama.cpp, also set min_p to 0.01, since its default of 0.05 is on the high side.

3. 3

   Re-download GGUFs from before late January

   On January 21, 2026, llama.cpp fixed a bug where the routing function was written wrong (it should have been sigmoid, not softmax). Before the fix, the model repeated itself and output quality was poor, and Unsloth re-uploaded its GGUFs afterward. If you downloaded files before then, download them again.

4. 4

   Ollama and MTP

   Ollama's official library has glm-4.7-flash with three tags: q4_K_M, q8_0 and bf16. Unsloth's docs still advise against running their GGUFs in Ollama because the chat template may be incompatible, so if you use Ollama, use the official library version. llama.cpp has supported its MTP speculative decoding since August 2026 (PR #24868); the PR author tested at 8-bit and measured an average speedup of about 1.18x with a draft length of 1.

5. 5

   Long context is light

   Its attention design (MLA) compresses the context cache, so long context takes little VRAM: at 4-bit, going from 8K to 128K adds only about 7 GB, and the full 198K comes to about 30 GB in total.

ollama run glm-4.7-flash

llama-server -hf unsloth/GLM-4.7-Flash-GGUF:Q4_K_M -c 8192

llama-server -hf unsloth/GLM-4.7-Flash-GGUF:Q4_K_M -c 8192 -ngl 99 \
  --cpu-moe

vllm serve zai-org/GLM-4.7-Flash

vllm serve unsloth/GLM-4.7-Flash-NVFP4

mlx_lm.server --model mlx-community/GLM-4.7-Flash-4bit

File names follow that week's model-library snapshot; the definitions are in the methodology. Methodology

Model	Total params	Active params	4-bit size
GLM 4.5 AirMoE	106B	12B	73 GB
GLM 5.3 FlashMoE	320B	18B	199.7 GB
GLM 5.2MoE	744B	40B	465.8 GB
GLM 5.3MoE	744B	40B	467.3 GB

About GLM