How to Deploy DeepSeek V4.1 Flash Locally

• Min VRAM at 4-bit (8K context)446 GB
• Parameters763B
• Active params8 / 16B
• Max context1M

Architecture

MoE

License

MIT

Launch date

Oct 17, 2026

Vendor

DeepSeek

Hugging Face repo

deepseek-ai/DeepSeek-V4.1-Flash-1017

GGUF repo

mrfakename/DeepSeek-V4.1-Flash-1017-GGUF

NVFP4 repo (Blackwell)

nvidia/DeepSeek-V4.1-Flash-NVFP4

Revision

1017

Layers

40

Hidden size

5,120

KV heads

1

Head dim

512

1017 is a weights-only update DeepSeek released on October 17: it scores higher on reasoning, coding, agent and vision benchmarks and gives shorter answers, while the architecture and file sizes match the September launch exactly. The biggest jump in DeepSeek's own results is on Terminal-Bench 4.0, which tests getting work done in a terminal, up from 31.2 to 42.3, and the same tasks now finish faster with fewer tokens. VRAM needs and supported hardware are unchanged; if you already run the September version, just change the model name in your launch command to DeepSeek-V4.1-Flash-1017. NVIDIA hasn't published an NVFP4 build of 1017 yet, so its NVFP4 repo still gives you the September weights.

Model size is shown as 763B to match the official Hugging Face repository. The model card lists 552B backbone parameters and roughly 196B Engram parameters separately. The file sizes below include the GGUF Engram tables and come from complete shard totals, not parameter-count formulas. File sizes are verified; quantization quality and runtime memory use are not.

Verified source · 2026-10-17

Full calculation rules are on the Methodology page.

Only common GPUs and systems are listed, from cheapest to most expensive.

Device	Cards	Total price	Total VRAM	Speed t/s	Total power
Instinct MI21064 GB per card	8 cardsEngram in RAM	$40,344	512 GB	64	2,400 W
RTX PRO 5000 72GB72 GB per card	8 cardsEngram in RAM	$85,776	576 GB	55.5	2,400 W
RTX PRO 600096 GB per card	4 cardsEngram in RAM	$89,400	384 GB	68.1	2,400 W
A100 80GB PCIe80 GB per card	8 cardsEngram in RAM	$168,000	640 GB	71.7	2,400 W

Rows marked “Engram in RAM” need about 203 GB of extra system RAM. Full calculation rules are on the Methodology page.

Detailed calculation rules are on the Methodology page.

As an eBay Partner, VRAMGlass may be compensated if you buy through the eBay links on this page. The price you pay stays the same.

1. 1

   Use vLLM 0.31.0 or later

   The recipe now asks for vLLM 0.31.0 or newer, and both the pip package and the official Docker image work. The nightly image and the deepseekv41-flash-0909 tag you needed at launch are no longer required. The flag that moves Engram into system RAM is --engram-config '{"cpu_offload":true}'. The first load is very slow, so the recipe sets the readiness timeout to 3600 seconds.

2. 2

   Run the community GGUFs with ik_llama.cpp

   mrfakename has built 2-, 3-, 4- and 8-bit files from the 1017 weights, and their sizes check out. Mainline llama.cpp's V4.1 pull request (PR #28696) still hadn't been merged as of October 18, but ik_llama.cpp merged V4.1 support on September 22, and these files carry the Engram settings its loader reads. The files have no vision part, so they handle text only. To keep the two Engram tables in system RAM, add -ot "engram_embd.weight=CPU".

3. 3

   Thinking is on by default

   With no parameters set, thinking is on at strength 50. If max_tokens is too small, the whole output length goes to thinking and the answer comes back empty. Strength takes an integer from 1 to 100, but the level names mean different things in different places: in vLLM, low, high, xhigh and max map to 25, 50, 75 and 100, while in the DeepSeek API, low, high and max map to 50, 75 and 100. The vLLM recipe cites the technical report: a strength of 60 to 80 recovers most of the accuracy with less than half the tokens.

4. 4

   Sampling parameters and text-only deployment

   The 1017 model card recommends temperature 1.0, top_p 1.0, and max_tokens of at least 256K. Its context is very light on VRAM: the cache for 1M of context is under 1 GB, so there is little downside to running the full context. For text-only work, add --language-model-only to skip the vision components and leave the VRAM for context.

docker run --gpus all --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e VLLM_ENGINE_READY_TIMEOUT_S=3600 -e VLLM_USE_RUST_FRONTEND=1 \
  vllm/vllm-openai:latest deepseek-ai/DeepSeek-V4.1-Flash-1017 \
  --tokenizer-mode deepseek_v41 --tensor-parallel-size 2 \
  --attention-config '{"backend":"FLASHINFER_MLA_SPARSE_DSV41","indexer_kv_dtype":"mxfp4","indexer_sparse_logits":true}' \
  --kv-cache-dtype fp8 --max-num-seqs 128 \
  --engram-config '{"cpu_offload":true}' \
  --tool-call-parser deepseek_v41 --enable-auto-tool-choice \
  --reasoning-parser deepseek_v41 --mm-encoder-tp-mode data

File names follow that week's model-library snapshot; the definitions are in the methodology. Methodology

Model	Total params	Active params	4-bit size
DeepSeek R1 Qwen3 8BDense	8.2B	8.2B	5 GB
DeepSeek R1 Distill Qwen 14BDense	14.8B	14.8B	9 GB
DeepSeek R1 Distill Qwen 32BDense	32.8B	32.8B	19.9 GB
DeepSeek V4 FlashMoE	284B	13B	155.1 GB
DeepSeek V3.2MoE	671B	37B	405.4 GB
DeepSeek V4 ProMoE	1,600B	49B	849.7 GB

About DeepSeek