DeepSeek V4.1 Flash: The VRAM Killer. Can You Run It Locally?

What separates V4.1 Flash from V4 Flash?

V4.1 Flash is a new architecture, not a touch-up of V4 Flash: layers drop from 43 to 40, total parameters rise from 284B to 769B, decode activation rises from 13B to 16B, and vision is native. The key differences across the three versions:

The capability gap is real: in the official evaluations V4.1 Flash scores 90.6 on Terminal Bench 2.1 (V4 Flash 82.7, V4 Pro 87.9), resolves 74.2% on DeepSWE v1.1 (V4 Flash 54.4%), and rates 3471 on Codeforces. DeepSeek's own conclusion is that V4.1 Flash beats V4 Pro on performance, cost and speed alike, which is why deepseek-flash on the API already points at it and deepseek-v4-pro will route to V4.1 Flash from midday on September 14 until V4.1 Pro ships.

Sources: model card, technical report, DeepSeek API docs.

What makes up the 511 GB of weights?

The Engram lookup tables take 189 GiB and the experts 260 GiB; together that is 94% of the file, and neither can be cut. The vLLM team gives this breakdown of the native checkpoint (source):

Two points matter for local deployment:

• The experts are already 4-bit. The "Q4 quantization" the community usually reaches for compresses almost nothing in V4.1 Flash's experts; only the Engram tables and attention layers can lose more precision. So when a V4.1 Flash GGUF does appear, the Q4 tier will not shrink to a quarter of the native size the way it does for other models. Expect something around 400 GB.
• Engram can in principle live on disk, but the runtime has to support it. It is sparse access through 2/3/4-gram hash tables: two modules with 8 hash heads each, 48 rows and about 12 KB read per token, behaving like an embedding lookup rather than a matrix multiply. Which rows get read depends only on the input tokens, so every address for a whole prompt can be computed before the forward pass starts and prefetched in one batch. Section 2.4.2 of the technical report states that official inference keeps the tables in host memory and prefetches them in the background over RDMA, never touching VRAM; moving them to NVMe only turns microseconds of latency into hundreds of microseconds, and decoding at 10 tokens/s needs just 120 KB/s of read bandwidth. So when you plan local RAM you can take those 189 GiB out of the budget and leave only NVMe space for them. The condition is that the runtime implements an on-demand disk-lookup operator, and as of September 10 llama.cpp, vLLM and MLX all lack it, with the official reference code loading the whole table into memory as well.

The table below is the VRAM requirement our model catalog computes with the general formula. The catalog follows the official figures: the parameter count is the 552B backbone excluding Engram, and the active parameters are listed as "8 / 16". The Q4 column assumes a Q4_K_M GGUF that does not exist yet, and the real file will be larger:

Notice that the 8K and 128K columns are nearly identical: V4.1 Flash's global KV cache is only 890 bytes per token, so a 128K context costs 0.11 GiB and a 1M context 0.87 GiB. This is the first model in our catalog whose KV cache can be ignored; the next section explains why.

Why call it a VRAM killer: where did the KV cache go?

What the "VRAM killer" kills is the KV cache line item: V4.1 Flash squeezes global KV down to 890 bytes per token, under 1 GiB at a 1M context, so context length no longer decides how much VRAM you need. Start with the gap against other models. KV bytes per token are computed from each model's attention structure, and V4 Flash uses the official "V4.1 is a quarter of it" figure:

By DeepSeek's own chart, that number is 437 times smaller than in the V1 era. Sections 2.3 to 2.4 of the technical report describe four compression steps, every one of them architectural rather than bought with quantization loss:

1. KV is a 512-dim latent vector stored in FP4. Following MLA's idea, each token stores one compressed 512-dimensional latent vector per layer instead of the K and V of 64 separate heads; V4.1 goes further and stores it in E2M1 format with one E4M3 scale factor per 16 channels, halving V4's FP8 again. That required quantization-aware training in the post-training stage.
2. Only 4 of the 40 layers actually produce KV. Compressed Sparse Attention 2 (CSA2) puts every layer in one of three modes — Full, Reindex or Reuse — and in the config file kv_source_layer_ids lists only layers 2, 8, 14 and 20 as Full mode; the rest reuse their KV and sparse indices, which is the cross-layer reuse.
3. The encoder merges every two tokens into one entry. The compression ratio of the encoder's 18 CSA2 layers is 2, so the number of global KV entries is half the number of tokens; the decoder's 20 layers get their KV projected from the output of the encoder's last layer (CED) and store none of their own.
4. The local attention window is fixed at 128. The sliding-window copy of the KV is stored in FP8, but it keeps only the most recent 128 tokens — a few MB across all 40 layers, independent of context length.

Compute does not grow linearly with context either: global attention runs only over the 512 positions the indexer selects, and the decoder further restricts candidates to the pool produced by the previous Full layer, so a 1M context is not just something you can fit but something you can actually compute.

For local deployment this means three things:

• Context length disappears from the VRAM budget. Running Qwen3.8 27B on a 24GB card, a 32K context already demands 8 GiB for KV; V4.1 Flash's 7 GB of VRAM in offload mode is a constant, the same at 8K as at 1M. That is why the "what GPU can run this" table on the model page puts the suggested context straight at 128K.
• Multiple sessions and saving state to disk are almost free. A 200,000-token agent session holds only 178 MB of global KV, so the llama.cpp habit of saving a slot to disk and restoring it can be done per session without a second thought; on the server side, many concurrent long sessions are no longer bottlenecked by KV either, which is exactly what lets DeepSeek push its API prices down.
• But it cannot kill the weights. Not one byte comes off the 511 GB of weights; the VRAM requirement merely goes from "weights + an ever-growing KV" to "weights + a KV of almost zero." So the question of whether you can run it locally moves from VRAM to RAM and disk, and the three routes below are all about where to put the weights.

What does the CED architecture mean for local inference?

Causal Encoder-Decoder (CED) halves the work in prefill but saves nothing in decode; for local users it improves the wait on long prompts, not the tokens per second. Section 2.2 of the technical report describes the mechanism:

1. The 40 layers are split into a 20-layer "causal encoder" and a 20-layer "decoder." The decoder layers no longer compute their global KV from their own hidden states; those are projected directly from the output of layer 20.
2. So prefill (processing the prompt) only has to run the first 20 layers, activating 8B parameters per token, while decode (generating the reply) still runs all 40 layers and activates 16B.
3. The decoder's sliding-window attention (SWA, 128-token window) is still computed layer by layer, so at the end of prefill the last 128 tokens of the prompt have to go through the decoder again to rebuild its SWA state. That step is called SWA Bounded Replay, and its cost is fixed at 128 tokens regardless of prompt length.

For local deployment this means:

• Prefill on long prompts costs roughly half as much. With expert offload, prefill has to move the activated expert weights of every layer from RAM into the GPU. V4.1 Flash moves 20 layers and 8B parameters where V4 Flash moves 43 layers and 13B. Feeding an agent tens of KB of code or documents, that wait will be noticeably shorter than on V4 Flash.
• Decode gets no benefit and is in fact slower. Decode activates 16B against V4 Flash's 13B, 23% more, and in an offload setup pinned by memory bandwidth we estimate about 9 tokens/s versus about 12 tokens/s for V4 Flash.
• Session state takes almost no space. At 890 bytes per token of global KV, a 200,000-token agent session is only 178 MB, which makes the llama.cpp habit of saving a slot to disk and restoring it very cheap. SWA state does not need to be persisted: on restore, replay the last 128 tokens the way the official implementation does.
• A single-user machine can be more exact than the official service. The server-side SWA replay is an approximate reconstruction, and section 6 of the technical report admits there may be unknown degradation at cache-restore boundaries. A single-user local runtime does not need that approximation: keeping the SWA KV in memory the whole time is the exact state, and 40 layers × 128 tokens of SWA cache is only a few MB.
• It is harder to implement than V4. Prefill and decode run different numbers of layers, and on top of that come the layerwise sparse indexer, the Engram hash lookup, FP4 KV dequantization and the mHC residual stream. llama.cpp has far more new graph structure to write than it did for V4 Flash. On the day V4 Flash 0731 shipped, unsloth had a GGUF repo up; do not expect that pace for V4.1 Flash.

What does each of the three routes require?

The datacenter route works today; the Mac Studio and consumer-card routes are both waiting on software. Estimated speeds by machine under our formulas:

Starred cells are expert-offload speeds, and they assume llama.cpp support. Our model catalog computes from the official 552B backbone parameter count, excluding Engram, so the 304 GB of RAM the offload needs and the 280 GB native figure of layout B below are on the same basis. The DGX Spark has only 128GB of unified memory, so the offload figures in the table do not apply to it.

Route one: vLLM on a multi-GPU datacenter node, working today

From version 0.30.0 vLLM ships a dedicated Docker image, verified on four hardware setups: an 8-GPU H200 node (1128 GB VRAM), a single GB200 NVL4 tray (768 GB), GB300, and AMD MI350X. vLLM's stated minimum is 614 GB of VRAM (511 GB × 1.2 headroom), so a 4-GPU H200 at 564 GB is not enough; the A100 generation has neither FP8 nor FP4 support and is not on the verified list. DeepSeek's own inference code also uses 8-way tensor parallelism in its example. The first load has to read all 511 GB of weights, which is why the official engine-ready timeout is set to one hour.

Route two: Mac Studio M3 Ultra 512GB, waiting for MLX to support Engram on disk

512GB of unified memory is the only home-machine capacity that comes close. Putting all 476 GiB of native weights into it does not work: macOS lets the GPU claim only about 75% of unified memory by default, and even pushing iogpu.wired_limit_mb to the limit leaves just a few GB of headroom. But if Engram stays on the SSD, memory holds only the experts, attention and embeddings, about 290 GiB at native FP4, which fits in 512GB with room to spare and needs no further cut to expert precision. So what this route waits on is not a low-bit build but MLX implementing the V4.1 architecture plus on-disk lookups, and as of September 10 mlx-community has no V4.1 conversion at all. Once one appears, 819 GB/s of bandwidth against 16B of active parameters works out to about 40 tokens/s of decode, which would make it the fastest home option.

Route three: a consumer or workstation card + lots of RAM, waiting for llama.cpp

With Engram on disk, the experts, attention and embeddings that remain can be placed two ways, both of them mechanisms llama.cpp already has, waiting only on V4.1 architecture support:

Layout A decodes at GPU bandwidth, on the order of 80 tokens/s for an RTX PRO 6000 under our formula, discounted once the model is split across cards; layout B is pinned by memory bandwidth, about 9 tokens/s on dual-channel DDR5, and 20 to 30 tokens/s on 8-channel DDR5, which has roughly four times the bandwidth of dual-channel. There is a middle ground between them: which 6 experts fire is decided per token by the router, so you cannot keep just the active experts in VRAM, but you can keep every expert of the first N layers on the card and offload the rest, which is exactly what llama.cpp's --n-cpu-moe means. At native FP4 each layer of experts is about 6.8 GB, so a 24GB card can hold two extra layers and a 96GB PRO 6000 can hold 12.

The bad news is that as of September 10 there is no V4.1 architecture PR in the llama.cpp repository and no disk-lookup operator either, so both layouts are closed for now. If you already have a large-RAM workstation, get the offload pipeline working with the V4 Flash 0731 GGUF first, see does adding RAM help when VRAM runs out.

What command can you run today?

Only the vLLM command works; no other runtime has model files for it yet. Launching on an 8-GPU H200 node with the official vLLM recipe (source):

docker run --gpus all --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e VLLM_ENGINE_READY_TIMEOUT_S=3600 \
  -e VLLM_USE_RUST_FRONTEND=1 \
  vllm/vllm-openai:deepseekv41-flash-0909 deepseek-ai/DeepSeek-V4.1-Flash \
  --tokenizer-mode deepseek_v41 \
  --tensor-parallel-size 8 \
  --tool-call-parser deepseek_v41 \
  --enable-auto-tool-choice \
  --reasoning-parser deepseek_v41 \
  --mm-encoder-tp-mode data

On a single GB200 NVL4 tray, change --tensor-parallel-size to 4; on AMD cards, switch to the vllm/vllm-openai-rocm:deepseekv41-flash-0909 image. For text-only work add --language-model-only, which skips the vision encoder and hands that VRAM to the KV cache (it is mutually exclusive with --mm-encoder-tp-mode data).

Two easy traps:

• Thinking mode is on by default at reasoning effort 50. Pass a small max_tokens and the whole budget goes into the reasoning trace, returning empty content and finish_reason=length, which looks like a broken model. Either set thinking: false in chat_template_kwargs or raise max_tokens.
• Reasoning effort only accepts low / high / xhigh / max or an integer from 1 to 100; medium and minimal are rejected.

Ollama, LM Studio, llama.cpp, MLX: as of September 10 none of them has usable files. When they do, our model pages will carry the launch commands.

Should you wait or buy?

Do not buy hardware for V4.1 Flash. Three reasons:

• Its advantage is concentrated in long-context agent work, where a single session runs to hundreds of thousands of tokens. A local offload setup at 9 tokens/s cannot carry that; the API with deepseek-flash is both cheaper and faster, and the arithmetic is in what running LLMs locally really costs.
• The full DeepSeek you can run at home is still V4 Flash 0731: a 155 GB Q4 GGUF that runs with offload on one 24GB card plus 192GB of RAM. Configurations are in which GPU to run DeepSeek locally.
• "Flash" no longer means small. V4 Flash is 284B and V4.1 Flash is 769B; the name refers to an API tier, not a size. Wait for V4.1 Pro before judging whether the whole V4.1 line is worth a hardware upgrade.

If you already own a workstation with 256GB of RAM or more, or a Mac Studio M3 Ultra 512GB, V4.1 support in llama.cpp and the conversion progress at mlx-community are worth watching: the moment a GGUF or MLX build appears with support for Engram on disk, those are the first home machines that will run it.