How to Deploy DeepSeek V4 Flash Locally

• Min VRAM at 4-bit157 GB · 8K
• Parameters284B
• Active params13B
• Max context1M

Architecture

MoE

Experts offloaded

6 GB VRAM + 152 GB RAM

License

MIT

Launch date

Jul 31, 2026

Vendor

DeepSeek

Hugging Face repo

deepseek-ai/DeepSeek-V4-Flash-0731

GGUF repo

unsloth/DeepSeek-V4-Flash-0731-GGUF

NVFP4 repo (Blackwell)

nvidia/DeepSeek-V4-Flash-0731-NVFP4

Revision

0731

Layers

43

Hidden size

4,096

KV heads

1

Head dim

512

Full calculation rules are on the Methodology page.

Device	Price	Speed t/s	Max context
Mac Studio M3 Ultra 512GBWhole system · 512 GB	$22,90013 listings	62.3	1M

Only common GPUs and systems are listed, one set for models under 50B and another for larger ones, sorted by price from low to high.

Device	Cards	Total price	Total VRAM	Speed t/s	Total power
Tesla V100 32GB32 GB per card	8 cards	$7,160	256 GB	66.1	2,000 W
Radeon RX 7900 XTX24 GB per card	8 cards	$10,000	192 GB	69.1	2,840 W
GeForce RTX 309024 GB per card	8 cards	$14,392	192 GB	67.9	2,800 W
Arc Pro B7032 GB per card	8 cards	$14,800	256 GB	49.7	1,840 W
DGX Spark 128GB128 GB per unit	2 units	$16,480	256 GB	25.6	480 W
RTX A600048 GB per card	4 cards	$19,596	192 GB	59.1	1,200 W
Instinct MI21064 GB per card	4 cards	$19,976	256 GB	95.9	1,200 W
GeForce RTX 409024 GB per card	8 cards	$30,064	192 GB	71.4	3,600 W
RTX PRO 5000 72GB72 GB per card	4 cards	$42,888	288 GB	85.6	1,200 W
RTX PRO 600096 GB per card	2 cards	$45,000	192 GB	100.7	1,200 W
GeForce RTX 509032 GB per card	8 cards	$67,160	256 GB	100.7	4,600 W
A100 80GB PCIe80 GB per card	4 cards	$75,996	320 GB	104.7	1,200 W
GeForce RTX 4090 48GB48 GB per card	4 cards	No price	192 GB	71.4	1,800 W

Full calculation rules are on the Methodology page.

Detailed calculation rules are on the Methodology page.

As an eBay Partner, VRAMGlass may be compensated if you buy through the eBay links on this page. The price you pay stays the same.

1. 1

   Quantization: use 8-bit if you can

   According to unsloth's analysis, the experts, which make up 96% of the official weights, already come in a 4-bit format (MXFP4), and the rest is FP8 and BF16. So 4-bit quantization is nearly lossless and 8-bit essentially lossless, and 8-bit is only about 7 GB larger than 4-bit (161.9 vs. 155.1 GB). Only 2-bit and 3-bit are truly lossy compression; use them only when memory is short.

2. 2

   llama.cpp supports it, including DSpark

   llama.cpp merged V4 support in late June 2026 and added DSpark speculative decoding in July (a small draft model guesses the next tokens, then the large model checks them). The draft model is a separate Q8_0 GGUF that takes about 10 GB more, and unsloth says it makes generation up to about twice as fast. Running the 0731 version in vLLM requires 0.25.0 or later.

3. 3

   Three thinking levels

   The 0731 version has three thinking levels, low, high and max, and unsloth says the default is high. To turn thinking off in llama.cpp, use --chat-template-kwargs '{"enable_thinking":false}'. For high and max, DeepSeek recommends leaving room for up to 384K of output. That sounds like a lot, but its context cache is tiny: at 4-bit, going from 32K to 256K adds only about 1.2 GB, so leaving plenty of context takes almost no VRAM.

4. 4

   Official sampling parameters

   Temperature 1.0; top_p 0.95 for agent use and 1.0 otherwise. unsloth's llama.cpp example also adds --min-p 0.01.

llama-server -hf unsloth/DeepSeek-V4-Flash-0731-GGUF:UD-Q4_K_XL -c 8192

llama-server -hf unsloth/DeepSeek-V4-Flash-0731-GGUF:UD-Q4_K_XL \
  -c 8192 -ngl 99 --cpu-moe

vllm serve deepseek-ai/DeepSeek-V4-Flash-0731 \
  --trust-remote-code \
  --kv-cache-dtype fp8 \
  --block-size 256 \
  --enable-expert-parallel \
  --tensor-parallel-size 8 \
  --tokenizer-mode deepseek_v4 \
  --tool-call-parser deepseek_v4 \
  --enable-auto-tool-choice \
  --reasoning-parser deepseek_v4 \
  --reasoning-config '{"reasoning_parser":"deepseek_v4","reasoning_start_str":"","reasoning_end_str":""}' \
  --speculative-config '{"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"probabilistic"}'

File names follow that week's model-library snapshot; the definitions are in the methodology. Methodology

Model	Total params	Active params	4-bit size
DeepSeek R1 Qwen3 8BDense	8.2B	8.2B	5 GB
DeepSeek R1 Distill Qwen 14BDense	14.8B	14.8B	9 GB
DeepSeek R1 Distill Qwen 32BDense	32.8B	32.8B	19.9 GB
DeepSeek V3.2MoE	671B	37B	405.4 GB
DeepSeek V4.1 FlashMoE	763B	8 / 16B	444.7 GB
DeepSeek V4 ProMoE	1,600B	49B	849.7 GB

About DeepSeek