How to Deploy DeepSeek V3.2 Locally

• Min VRAM at 4-bit408 GB · 8K
• Parameters671B
• Active params37B
• Max context160K

Architecture

MoE

Experts offloaded

13 GB VRAM + 396 GB RAM

License

MIT

Launch date

Dec 1, 2025

Vendor

DeepSeek

Hugging Face repo

deepseek-ai/DeepSeek-V3.2

GGUF repo

unsloth/DeepSeek-V3.2-GGUF

NVFP4 repo (Blackwell)

nvidia/DeepSeek-V3.2-NVFP4

MLX repo (Apple)

mlx-community/DeepSeek-V3.2-4bit

Layers

61

Hidden size

7,168

KV heads

128

Head dim

192

Full calculation rules are on the Methodology page.

Only common GPUs and systems are listed, one set for models under 50B and another for larger ones, sorted by price from low to high.

Device	Cards	Total price	Total VRAM	Speed t/s	Total power
DGX Spark 128GB128 GB per unit	4 units	$32,960	512 GB	8.6	960 W
Instinct MI21064 GB per card	8 cards	$39,952	512 GB	42.7	2,400 W
RTX PRO 5000 72GB72 GB per card	8 cards	$85,776	576 GB	36.4	2,400 W
A100 80GB PCIe80 GB per card	8 cards	$151,992	640 GB	48.7	2,400 W
RTX PRO 600096 GB per card	8 cards	$180,000	768 GB	45.9	4,800 W

Mac Studios link up over Thunderbolt 5 and run with MLX or exo; vLLM doesn't support them. Full calculation rules are on the Methodology page.

Detailed calculation rules are on the Methodology page.

As an eBay Partner, VRAMGlass may be compensated if you buy through the eBay links on this page. The price you pay stays the same.

1. 1

   Which framework to use

   For local use, llama.cpp is the first choice. V3.2 support (including DSA sparse attention) was merged in late May 2026, support for the model's built-in MTP speculative decoding followed in August, and unsloth has a full set of GGUFs from 1-bit to BF16. vLLM's official recipe has only been validated on H100 and H200.

2. 2

   No ready-made chat template

   Instead of the usual Jinja chat template, DeepSeek provides a set of Python encoding scripts, and the tool-calling format also differs from earlier models. llama.cpp has had a dedicated V3.2 parser since April 2026; if you write your own client, build prompts following the scripts in the official encoding directory. The new developer role is used only for the search agent.

3. 3

   Sampling parameters and the Speciale version

   For local deployment, DeepSeek recommends temperature 1.0 and top_p 0.95. There is also V3.2-Speciale with the same architecture, built only for deep reasoning; it doesn't support tool calling, so don't use it for agents.

4. 4

   Quantization and context

   The original weights are already FP8, so 8-bit (about 713 GB) is essentially the original model. Its attention design is very economical with context: at 4-bit, going from 8K to 160K adds only about 11 GB, so if the weights fit, you can basically run the full context.

llama-server -hf unsloth/DeepSeek-V3.2-GGUF:Q4_K_M -c 8192

llama-server -hf unsloth/DeepSeek-V3.2-GGUF:Q4_K_M -c 8192 -ngl 99 \
  --cpu-moe

vllm serve deepseek-ai/DeepSeek-V3.2 \
  --trust-remote-code \
  --kernel-config.enable_flashinfer_autotune=False \
  --tensor-parallel-size 8 \
  --tokenizer-mode deepseek_v32 \
  --tool-call-parser deepseek_v32 \
  --enable-auto-tool-choice \
  --reasoning-parser deepseek_v3

mlx_lm.server --model mlx-community/DeepSeek-V3.2-4bit

File names follow that week's model-library snapshot; the definitions are in the methodology. Methodology

Model	Total params	Active params	4-bit size
DeepSeek R1 Qwen3 8BDense	8.2B	8.2B	5 GB
DeepSeek R1 Distill Qwen 14BDense	14.8B	14.8B	9 GB
DeepSeek R1 Distill Qwen 32BDense	32.8B	32.8B	19.9 GB
DeepSeek V4 FlashMoE	284B	13B	155.1 GB
DeepSeek V4.1 FlashMoE	763B	8 / 16B	444.7 GB
DeepSeek V4 ProMoE	1,600B	49B	849.7 GB

About DeepSeek