How to Deploy DeepSeek V4 Pro Locally

DeepSeek V4 Pro is DeepSeek's largest open model to date, with 1.6T parameters; at 4-bit it is about 851 GB, too big even for eight 96GB cards. An individual has two routes: a server with lots of RAM plus one GPU, with the experts kept in system RAM, at about 22 tokens/s, or two linked 512GB Mac Studios, which have the capacity for 4-bit. Where it beats V4 Flash is breadth of knowledge: on the official knowledge QA test for the base models (SimpleQA-Verified), it scores 55.2 to 30.1.

DeepSeek V4 Pro details

Min VRAM at 4-bit851 GB · 8K
Parameters1,600B
Active params49B
Max context1M

Architecture: MoE
Experts offloaded: 16 GB VRAM + 837 GB RAM
License: MIT
Launch date: Aug 13, 2026
Vendor: DeepSeek
Hugging Face repo: deepseek-ai/DeepSeek-V4-Pro-0813
GGUF repo: unsloth/DeepSeek-V4-Pro-0813-GGUF
NVFP4 repo (Blackwell): nvidia/DeepSeek-V4-Pro-0813-NVFP4
Revision: 0813
Layers: 61
Hidden size: 7,168
KV heads: 1
Head dim: 512

How much VRAM does DeepSeek V4 Pro need for local deployment?

Quantization	WeightsGB	32K contextGB	128K contextGB	256K contextGB	1M context (max)GB
1-bit	403.2est.	404.5	405.3	406.4	412.6
2-bit	638.4est.	639.7	640.5	641.6	647.8
3-bit	823.2est.	824.5	825.3	826.4	832.6
4-bit	849.7GGUF	851	851.8	852.9	859.1
5-bit	—	—	—	—	—
8-bit	873.4GGUF	874.7	875.5	876.6	882.8
BF16/FP16	3,360est.	3,361.3	3,362.1	3,363.2	3,369.4

Full calculation rules are on the Methodology page.

Can a single GPU run DeepSeek V4 Pro?

No; nothing you can put on a desk comes close. At 1.6T parameters, 4-bit with 32K of context is about 851 GB, so even the 512GB Mac Studio could hold only about 60% of it. And there's nothing smaller to fall back on: unsloth only offers 4-bit and 8-bit files, and they're nearly the same size because the experts already ship in 4-bit form. The one single-machine route is a server filled with 768GB of RAM plus one RTX PRO 6000, keeping most experts in system memory. That runs 4-bit at about 22 tokens/s, and 8-bit is barely slower on the same box. Without a server, you'd need two 512GB Mac Studios linked together.

How many GPUs do you need to run DeepSeek V4 Pro?

No GPU setup holds 4-bit. Eight 96GB RTX PRO 6000 cards add up to 768GB, still short of the roughly 851 GB that 4-bit needs. Going by parameter count, 2-bit would squeeze onto those eight cards at about 53 tokens/s, but no 2-bit file exists yet. On capacity, two 512GB Mac Studio M3 Ultras linked over Thunderbolt 5 can hold 4-bit at an estimated 22 tokens/s, and 8-bit also takes two at nearly the same speed; check first whether the Mac distributed-inference tools support this new architecture. Running 4-bit on GPUs takes an eight-GPU data center server such as H200 or B200, and that class of hardware plus AMD's MI355X is all vLLM's official recipe covers.

Device	Cards	Total price	Total VRAM	Speed t/s	Total power
Mac Studio M3 Ultra 512GB512 GB per unit	2 units	$45,800	1,024 GB	27.7	960 W
RTX PRO 600096 GB per card	8 cards	$180,000	768 GB	52.6	4,800 W

Device	Cards	Total price	Total VRAM	Speed t/s	Total power
Mac Studio M3 Ultra 512GB512 GB per unit	2 units	$45,800	1,024 GB	21.6	960 W

Device	Cards	Total price	Total VRAM	Speed t/s	Total power
Mac Studio M3 Ultra 512GB512 GB per unit	2 units	$45,800	1,024 GB	21	960 W

Mac Studios link up over Thunderbolt 5 and run with MLX or exo; vLLM doesn't support them. Full calculation rules are on the Methodology page.

What PC or server build do you need to run DeepSeek V4 Pro locally?

The workable setup is a 12-channel DDR5 server with one RTX PRO 6000, at about 22 tokens/s at 4-bit. Putting all the 4-bit experts in system RAM takes about 837 GB, more than the 768 GB you get from filling 12 slots with 64GB sticks, so the setup on this page keeps 6 layers of experts on the GPU and puts the rest in RAM. Switching to 8-bit also gives about 22 tokens/s, nearly as fast, because it is only about 3% larger than 4-bit.

4 × Tesla V100 32GB + Dual Xeon E5 8-GPU server

Budget

Generation speed6t/s

Memory bandwidth137GB/s

GPU: 4 × Tesla V100 32GB8 / 61 expert layers in 128 GB VRAM; $3,580
RAM: 768 GB DDR4-213324 × 32 GB sticks; $3,912
CPU: 2 × Intel Xeon E5-2680 v428 cores / 56 threads · 8-channel in total; $38
Barebone: Supermicro SYS-4028GR-TR8 GPU slots · 24 DIMM slots; $1,300
PSU: Built-in 4 × 1,600 W; Included

Reference build price$8,830

4 × GeForce RTX 5090 + Dual EPYC DDR5

Flagship

Generation speed27.5t/s

Memory bandwidth922GB/s

GPU: 4 × GeForce RTX 50908 / 61 expert layers in 128 GB VRAM; $33,580
RAM: 768 GB DDR5-480024 × 32 GB sticks; $29,328
CPU: 2 × AMD EPYC 935464 cores / 128 threads · 24-channel in total; $3,386
Board: Gigabyte MZ73-LM024 DIMM slots; $1,300
PSU: 2 × Seasonic PRIME PX-22004,400 W; $1,320

Reference build price$68,914

8 × RTX PRO 6000 + Dual EPYC DDR5 8-GPU server

Giant models

Generation speed41t/s

Memory bandwidth922GB/s

GPU: 8 × RTX PRO 600058 / 61 expert layers in 768 GB VRAM; $180,000
RAM: 384 GB DDR5-480024 × 16 GB sticks; $14,328
CPU: 2 × AMD EPYC 935464 cores / 128 threads · 24-channel in total; $3,386
Barebone: ASUS ESC8000A-E128 GPU slots · 24 DIMM slots; $6,900
PSU: Built-in 4 × 3,000 W; Included

Reference build price$204,614

Detailed calculation rules are on the Methodology page.

As an eBay Partner, VRAMGlass may be compensated if you buy through the eBay links on this page. The price you pay stays the same.

What to know before running DeepSeek V4 Pro locally

Only two quantization levels
In the official weights, the experts already come in a 4-bit format (FP4) and the rest is FP8 (about an 893 GB download, including the DSpark draft module). So 4-bit (about 850 GB) and 8-bit (about 873 GB) differ by only a little over 20 GB. If you can spare that extra 20-plus GB of RAM, just use 8-bit.
Framework support
The V4 support merged into mainline llama.cpp in late June 2026 covers Pro as well, and unsloth's GGUFs are made for it. Everything the vLLM recipe has validated is data center hardware (H200, B200, GB200, B300, GB300 and MI355X); MI300X and MI325X are marked as unsupported.
Don't default to max thinking
Thinking levels and sampling parameters are the same as for V4 Flash (temperature 1.0, top_p 0.95 for agent use). DeepSeek's agent results were measured at max, and for both high and max it recommends leaving room for up to 384K of output. But at about 22 tokens/s with 4-bit expert offloading, actually filling 384K would take about 5 hours, so low or high is more practical for everyday use.
1M of context takes almost no memory
Its attention design compresses context heavily: at 4-bit, going from 32K to 1M adds only about 8.1 GB, negligible next to 850 GB of weights, so running the full context doesn't call for extra memory.

Launch commands

llama.cpp

NVIDIA
AMD
Mac
CPU

All on GPU

Terminal

llama-server -hf unsloth/DeepSeek-V4-Pro-0813-GGUF:UD-Q4_K_XL -c 8192

MoE offloading

Short on VRAM? Add --cpu-moe to keep the routed experts in system RAM (about 837 GB needed).

Terminal

llama-server -hf unsloth/DeepSeek-V4-Pro-0813-GGUF:UD-Q4_K_XL -c 8192 \
  -ngl 99 --cpu-moe

vLLM

Linux
NVIDIA

Follow the deployment guide to prepare the required runtime and hardware.

Terminal

vllm serve deepseek-ai/DeepSeek-V4-Pro-0813 \
  --trust-remote-code \
  --kv-cache-dtype fp8 \
  --block-size 256 \
  --enable-expert-parallel \
  --tensor-parallel-size 8 \
  --max-model-len 800000 \
  --gpu-memory-utilization 0.95 \
  --max-num-seqs 512 \
  --no-enable-flashinfer-autotune \
  --compilation-config '{"mode": 0, "cudagraph_mode": "FULL_DECODE_ONLY"}' \
  --tokenizer-mode deepseek_v4 \
  --tool-call-parser deepseek_v4 \
  --enable-auto-tool-choice \
  --reasoning-parser deepseek_v4 \
  --speculative-config '{"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"probabilistic"}'

Ollama

Windows
macOS
Linux

Ollama only has a cloud version, deepseek-v4-pro:cloud, which runs on Ollama's servers rather than on your machine.

File names follow that week's model-library snapshot; the definitions are in the methodology. Methodology

Other models in the DeepSeek family

Model	Total params	Active params	4-bit size
DeepSeek R1 Qwen3 8BDense	8.2B	8.2B	5 GB
DeepSeek R1 Distill Qwen 14BDense	14.8B	14.8B	9 GB
DeepSeek R1 Distill Qwen 32BDense	32.8B	32.8B	19.9 GB
DeepSeek V4 FlashMoE	284B	13B	155.1 GB
DeepSeek V3.2MoE	671B	37B	405.4 GB
DeepSeek V4.1 FlashMoE	763B	8 / 16B	444.7 GB

About DeepSeek

How to Deploy DeepSeek V4 Pro Locally

DeepSeek V4 Pro details

How much VRAM does DeepSeek V4 Pro need for local deployment?

Can a single GPU run DeepSeek V4 Pro?

How many GPUs do you need to run DeepSeek V4 Pro?

What PC or server build do you need to run DeepSeek V4 Pro locally?

Tesla V100 32GB + Dual Xeon E5 8-GPU server

GeForce RTX 5090 + Dual EPYC DDR5

4 × RTX PRO 6000 + Dual EPYC DDR5

4 × Tesla V100 32GB + Dual Xeon E5 8-GPU server

4 × GeForce RTX 5090 + Dual EPYC DDR5

8 × RTX PRO 6000 + Dual EPYC DDR5 8-GPU server

8 × Tesla V100 32GB + Dual Xeon E5 8-GPU server

GeForce RTX 5090 + Dual EPYC DDR5

8 × RTX PRO 6000 + Dual EPYC DDR5 8-GPU server

What to know before running DeepSeek V4 Pro locally

Only two quantization levels

Framework support

Don't default to max thinking

1M of context takes almost no memory

Launch commands

llama.cpp

vLLM

Ollama

Other models in the DeepSeek family