VRAMGlass contains affiliate links. As an eBay Partner, we may be compensated if you make a purchase.As an eBay Partner, we may be compensated.Learn more

GPUs tracked57Price history49 daysPrice points (24h)43,400 pointsBiggest 24h dropGeForce RTX 3060 12GB-11.1%Biggest 24h riseCMP 170HX 40GB+19.8%Updated
$ USD

How to Deploy DeepSeek R1 Distill Qwen 32B Locally

DeepSeek R1 Distill Qwen 32B is the largest Qwen-based model in the first batch of R1 distills. A 24GB card can hold its 4-bit version, but only about 17K of context is left over. It was fine-tuned from Qwen2.5 32B, and at launch DeepSeek said it beat OpenAI o1-mini on several benchmarks. Today, though, the similarly sized Qwen3.8 27B scores much higher and also needs less VRAM for long context.

DeepSeek R1 Distill Qwen 32B details

  • Min VRAM at 4-bit24 GB · 8K
  • Parameters32.8B
  • Max context128K
Architecture
Dense
License
MIT
Launch date
Jan 20, 2025
Vendor
DeepSeek
Hugging Face repo
deepseek-ai/DeepSeek-R1-Distill-Qwen-32B
GGUF repo
bartowski/DeepSeek-R1-Distill-Qwen-32B-GGUF
MLX repo (Apple)
mlx-community/DeepSeek-R1-Distill-Qwen-32B-4bit
Ollama tag
deepseek-r1:32b
Layers
64
Hidden size
5,120
KV heads
8
Head dim
128

How much VRAM does DeepSeek R1 Distill Qwen 32B need for local deployment?

QuantizationWeightsGB8K contextGB32K contextGB128K context (max)GB
1-bit8.3est.11.517.943.7
2-bit13.1GGUF16.322.848.5
3-bit17.9GGUF21.127.653.3
4-bit19.9GGUF23.129.655.3
5-bit23.3GGUF26.53358.7
8-bit34.8GGUF3844.570.2
BF16/FP1665.5GGUF68.775.2100.9

Full calculation rules are on the Methodology page.

Can a single GPU run DeepSeek R1 Distill Qwen 32B?

A 24GB card runs 4-bit, but context is cramped. The 4-bit weights take about 20 GB, so an RTX 3090 or RTX 4090 is left with roughly 17K of context, and a reasoning model working through a hard problem can run out of room before it finishes. Halving the memory that context uses in llama.cpp stretches those cards to around 32K. To really open it up, the 32GB RTX 5090 reaches about 49K and is nearly twice as fast; the full window takes 48GB or more. 16GB cards are stuck at 2-bit, which isn't worth it. 128GB unified-memory machines fit 8-bit, but this is a dense model that reads every weight for every token, so expect single-digit speeds.

eBay
DevicePriceSpeed t/sMax context
Tesla V100 32GB32 GB$89518 listings31.549K
Radeon RX 7900 XTX24 GB$1,25029 listings33.517K
GeForce RTX 309024 GB$1,799107 listings32.717K
GeForce RTX 409024 GB$3,75868 listings35.117K
Ryzen AI Max+ 395 128GBWhole system · 128 GB$4,46420 listings9.2128K
RTX A600048 GB$4,89916 listings27113K
Mac Studio M5 Ultra 96GBWhole system · 96 GB$7,27412 listings42.3128K
MacBook Pro M5 Max 128GBWhole system · 128 GB$7,54920 listings21.8128K
DGX Spark 128GBWhole system · 128 GB$8,24022 listings9.9128K
GeForce RTX 509032 GB$8,395123 listings60.149K
RTX PRO 600096 GB$22,50011 listings60.1128K

Only common GPUs and systems are listed, one set for models under 50B and another for larger ones, sorted by price from low to high.

What to know before running DeepSeek R1 Distill Qwen 32B locally

  1. Squeezing more context onto a 24GB card

    At 4-bit, every extra 1K of context takes about 0.27 GB (going from 8K to 32K adds 6.5 GB). In llama.cpp, -ctk q8_0 -ctv q8_0 shrinks the context cache to about half; by this site's VRAM formula, that stretches context on a 24GB card from about 17K to about 32K. Don't move up to 5-bit: 5-bit with 8K already needs 26.5 GB, which a 24GB card can't hold.

  2. Use it the same way as the 14B distill

    The repository defaults are temperature 0.6 and top_p 0.95, and DeepSeek asks you to keep temperature within 0.5 to 0.7. Don't use a system prompt. It occasionally skips thinking, so forcing the output to start with “<think>” is recommended. See the 14B distill for details.

  3. The official vLLM example uses two GPUs at 32K

    The vLLM command on the model card uses two GPUs and 32K of context (--tensor-parallel-size 2 --max-model-len 32768), but it is written for the original BF16 weights, about 66 GB, which don't fit even on two 24GB cards. On consumer cards, stick with the 4-bit GGUF; the Ollama tag is deepseek-r1:32b, about a 20 GB download.

Launch commands

Ollama

  • Windows
  • macOS
  • Linux
Terminal
ollama run deepseek-r1:32b

llama.cpp

  • NVIDIA
  • AMD
  • Mac
  • CPU
Terminal
llama-server -hf bartowski/DeepSeek-R1-Distill-Qwen-32B-GGUF:Q4_K_M \
  -c 8192

vLLM

  • Linux
  • NVIDIA
  • AMD
Terminal
vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-32B

MLX

  • Apple Silicon

On a Mac, pip install mlx-lm first; this command downloads the MLX weights from Hugging Face and starts an OpenAI-compatible server.

Terminal
mlx_lm.server --model mlx-community/DeepSeek-R1-Distill-Qwen-32B-4bit

File names follow that week's model-library snapshot; the definitions are in the methodology. Methodology

Other models in the DeepSeek family

ModelTotal paramsActive params4-bit size
DeepSeek R1 Qwen3 8BDense8.2B8.2B5 GB
DeepSeek R1 Distill Qwen 14BDense14.8B14.8B9 GB
DeepSeek V4 FlashMoE284B13B155.1 GB
DeepSeek V3.2MoE671B37B405.4 GB
DeepSeek V4.1 FlashMoE763B8 / 16B444.7 GB
DeepSeek V4 ProMoE1,600B49B849.7 GB

About DeepSeek