VRAMGlass contains affiliate links. As an eBay Partner, we may be compensated if you make a purchase.As an eBay Partner, we may be compensated.Learn more

GPUs tracked57Price history49 daysPrice points (24h)43,400 pointsBiggest 24h dropGeForce RTX 3060 12GB-11.1%Biggest 24h riseCMP 170HX 40GB+19.8%Updated
$ USD

How to Deploy Qwen3 32B Locally

Qwen3 32B is the largest dense model in the April 2025 Qwen3 lineup, with both thinking and non-thinking modes. A 24GB card can hold its 4-bit version (about 23 GB at 8K), but context is heavy on VRAM: on an RTX 3090 it only reaches about 18K, which fills up as soon as thinking runs a little long. For a new setup, we recommend switching to Qwen3.8 27B: it also fits on one 24GB card, but its context goes up to about 122K.

Qwen3 32B details

  • Min VRAM at 4-bit24 GB · 8K
  • Parameters32.8B
  • Max context40K
Architecture
Dense
License
Apache-2.0
Launch date
Apr 27, 2025
Vendor
Alibaba
Hugging Face repo
Qwen/Qwen3-32B
GGUF repo
Qwen/Qwen3-32B-GGUF
NVFP4 repo (Blackwell)
nvidia/Qwen3-32B-NVFP4
MLX repo (Apple)
Qwen/Qwen3-32B-MLX-4bit
Ollama tag
qwen3:32b
Layers
64
Hidden size
5,120
KV heads
8
Head dim
128

How much VRAM does Qwen3 32B need for local deployment?

QuantizationWeightsGB8K contextGB32K contextGB40K context (max)GB
1-bit8.3GGUF11.51820.1
2-bit12.8GGUF1622.524.6
3-bit16.4GGUF19.626.128.2
4-bit19.8GGUF2329.531.6
5-bit23.2GGUF26.432.935
8-bit34.8GGUF3844.546.6
BF16/FP1665.5GGUF68.775.277.3

Full calculation rules are on the Methodology page.

Can a single GPU run Qwen3 32B?

Yes, a single 24GB card runs 4-bit, but context is the catch. 4-bit with 8K of context takes about 23 GB, which leaves an RTX 3090, 4090 or RX 7900 XTX room for only about 18K. Every layer keeps its own copy of the full context, so memory use climbs quickly as the window grows, and one long stretch of reasoning in thinking mode can eat what's left. To use the full 40K window you need the 32GB RTX 5090, which runs 4-bit at about 60 tokens/s. 16GB cards are stuck at 2-bit with a cramped window, so we'd skip them. For a new setup, Qwen3.8 27B on the same 24GB card leaves far more room for context.

eBay
DevicePriceSpeed t/sMax context
Tesla V100 32GB32 GB$89518 listings31.640K
Radeon RX 7900 XTX24 GB$1,25029 listings33.618K
GeForce RTX 309024 GB$1,799107 listings32.818K
GeForce RTX 409024 GB$3,75868 listings35.218K
Ryzen AI Max+ 395 128GBWhole system · 128 GB$4,46420 listings9.340K
RTX A600048 GB$4,89916 listings27.240K
Mac Studio M5 Ultra 96GBWhole system · 96 GB$7,27412 listings42.540K
MacBook Pro M5 Max 128GBWhole system · 128 GB$7,54920 listings21.940K
DGX Spark 128GBWhole system · 128 GB$8,24022 listings9.940K
GeForce RTX 509032 GB$8,395123 listings60.340K
RTX PRO 600096 GB$22,50011 listings60.340K

Only common GPUs and systems are listed, one set for models under 50B and another for larger ones, sorted by price from low to high.

What to know before running Qwen3 32B locally

  1. Why context is so heavy on VRAM

    All 64 of its layers store a full context cache, so every extra 1K of context adds about 0.27 GB: at 4-bit, going from 8K to 32K takes VRAM from about 23.0 GB to 29.5 GB, and the full 40K needs 31.6 GB. The 27B models from Qwen3.5 onward switched to having only 1 in every 4 layers see the full context, so going from 8K to 32K adds only about 1.6 GB. That's why the newer models can open long context on a 24GB card.

  2. Trade-offs on a 24GB card

    Thinking takes up context too, so for everyday Q&A, add /no_think to your prompt to turn thinking off and save a good amount of room. If you really need more, drop to 3-bit (about 19.6 GB at 8K); by this site's formula, a 24GB card then reaches about 30K.

  3. Leave enough output length for thinking

    Qwen recommends an output length of 32,768 tokens for most problems and 38,912 for competition-level ones; at 4-bit, that takes an RTX 5090 or a 48GB card to fit. Sampling parameters are the same as for Qwen3 8B: temperature 0.6, top_p 0.95 and top_k 20 in thinking mode, and 0.7, 0.8 and 20 in non-thinking mode.

  4. Which framework to use

    Qwen's official repository provides GGUFs, and the Ollama tag is qwen3:32b; vLLM needs 0.8.5 or later and SGLang 0.4.6.post1 or later.

Launch commands

Ollama

  • Windows
  • macOS
  • Linux
Terminal
ollama run qwen3:32b

llama.cpp

  • NVIDIA
  • AMD
  • Mac
  • CPU
Terminal
llama-server -hf Qwen/Qwen3-32B-GGUF:Q4_K_M -c 8192

vLLM

  • Linux
  • NVIDIA
  • AMD
Original weights
Terminal
vllm serve Qwen/Qwen3-32B
NVFP4 · Blackwell

Blackwell GPUs (RTX 50 series, RTX PRO 6000, B200) run NVFP4 weights natively, at about 30% of the original size.

Terminal
vllm serve nvidia/Qwen3-32B-NVFP4

MLX

  • Apple Silicon

On a Mac, pip install mlx-lm first; this command downloads the MLX weights from Hugging Face and starts an OpenAI-compatible server.

Terminal
mlx_lm.server --model Qwen/Qwen3-32B-MLX-4bit

File names follow that week's model-library snapshot; the definitions are in the methodology. Methodology

Other models in the Qwen family

ModelTotal paramsActive params4-bit size
Qwen3 8BDense8.2B8.2B5 GB
Qwen3.5 9BDense9.7B9.7B5.7 GB
Qwen3.6 27BDense27.8B27.8B16.8 GB
Qwen3.8 27BDense27.8B27.8B16.5 GB
Qwen3.6 35B A3BMoE36B3B22.1 GB
Qwen3.8 Flash NextMoE176B6B111.3 GB
Qwen3.8 2.4T A95BMoE2,446B95B1,310.9 GB

About Qwen