How to Deploy DeepSeek R1 Distill Qwen 14B Locally

• Min VRAM at 4-bit12 GB · 8K
• Parameters14.8B
• Max context128K

Architecture

Dense

License

MIT

Launch date

Jan 20, 2025

Vendor

DeepSeek

Hugging Face repo

deepseek-ai/DeepSeek-R1-Distill-Qwen-14B

GGUF repo

bartowski/DeepSeek-R1-Distill-Qwen-14B-GGUF

MLX repo (Apple)

mlx-community/DeepSeek-R1-Distill-Qwen-14B-4bit

Ollama tag

deepseek-r1:14b

Layers

48

Hidden size

5,120

KV heads

8

Head dim

128

Full calculation rules are on the Methodology page.

Device	Price	Speed t/s	Max context
Tesla V100 16GB16 GB	$37024 listings	64.2	35K
GeForce RTX 5060 Ti 16GB16 GB	$86958 listings	33.4	35K
Tesla V100 32GB32 GB	$89518 listings	64.2	120K
Radeon RX 7900 XTX24 GB	$1,25029 listings	68	77K
GeForce RTX 309024 GB	$1,799107 listings	66.5	77K
GeForce RTX 409024 GB	$3,75868 listings	71.1	77K
Ryzen AI Max+ 395 128GBWhole system · 128 GB	$4,46420 listings	19.5	128K
RTX A600048 GB	$4,89916 listings	55.5	128K
Mac Studio M5 Ultra 96GBWhole system · 96 GB	$7,27412 listings	84.8	128K
MacBook Pro M5 Max 128GBWhole system · 128 GB	$7,54920 listings	45.1	128K
DGX Spark 128GBWhole system · 128 GB	$8,24022 listings	20.8	128K
GeForce RTX 509032 GB	$8,395123 listings	117.3	120K
RTX PRO 600096 GB	$22,50011 listings	117.3	128K

Only common GPUs and systems are listed, one set for models under 50B and another for larger ones, sorted by price from low to high.

1. 1

   Follow the official prompting guidance

   DeepSeek makes three recommendations. Set temperature between 0.5 and 0.7 (0.6 recommended); higher values tend to cause endless repetition or incoherent output. Don't use a system prompt; put every instruction in the user message. For math problems, add “Please reason step by step, and put your final answer within \boxed{}” to the prompt.

2. 2

   It sometimes skips thinking

   R1-series models occasionally output an empty thinking block and answer directly, which hurts quality. DeepSeek recommends forcing every response to start with “<think>” followed by a newline. Versions from 0528 onward (such as R1 Qwen3 8B) no longer need this step.

3. 3

   Context, not weights, is the bottleneck

   It uses standard attention, so the KV cache that holds the context grows linearly as context gets longer. At 4-bit, going from 8K to 128K adds about 24.1 GB, more than twice the size of the weights themselves. So on a 16GB card, what limits it is context length; before tackling long reasoning problems, work out how much room to leave.

4. 4

   Which framework to use

   It is simply the Qwen2.5 architecture, so llama.cpp, Ollama, vLLM and SGLang all support it directly. The Ollama tag is deepseek-r1:14b (4-bit, about a 9 GB download). DeepSeek says it made small changes to the config and tokenizer, so deploy with the files from DeepSeek's repository.

ollama run deepseek-r1:14b

llama-server -hf bartowski/DeepSeek-R1-Distill-Qwen-14B-GGUF:Q4_K_M \
  -c 8192

vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-14B

mlx_lm.server --model mlx-community/DeepSeek-R1-Distill-Qwen-14B-4bit

File names follow that week's model-library snapshot; the definitions are in the methodology. Methodology

Model	Total params	Active params	4-bit size
DeepSeek R1 Qwen3 8BDense	8.2B	8.2B	5 GB
DeepSeek R1 Distill Qwen 32BDense	32.8B	32.8B	19.9 GB
DeepSeek V4 FlashMoE	284B	13B	155.1 GB
DeepSeek V3.2MoE	671B	37B	405.4 GB
DeepSeek V4.1 FlashMoE	763B	8 / 16B	444.7 GB
DeepSeek V4 ProMoE	1,600B	49B	849.7 GB

About DeepSeek