How to Deploy Qwen3.6 35B A3B Locally

• Min VRAM at 4-bit24 GB · 8K
• Parameters36B
• Active params3B
• Max context256K

Architecture

MoE

Experts offloaded

4 GB VRAM + 21 GB RAM

License

Apache-2.0

Launch date

Apr 16, 2026

Vendor

Alibaba

Hugging Face repo

Qwen/Qwen3.6-35B-A3B

GGUF repo

unsloth/Qwen3.6-35B-A3B-GGUF

NVFP4 repo (Blackwell)

nvidia/Qwen3.6-35B-A3B-NVFP4

MLX repo (Apple)

mlx-community/Qwen3.6-35B-A3B-4bit

Ollama tag

qwen3.6:35b-a3b

Layers

40

Hidden size

2,048

KV heads

2

Head dim

256

Full calculation rules are on the Methodology page.

Device	Price	Speed t/s	Max context
Tesla V100 32GB32 GB	$89518 listings	133.5	256K
Radeon RX 7900 XTX24 GB	$1,25029 listings	136.7	123K
GeForce RTX 309024 GB	$1,799107 listings	135.4	123K
GeForce RTX 409024 GB	$3,75868 listings	139	123K
Ryzen AI Max+ 395 128GBWhole system · 128 GB	$4,46420 listings	68.9	256K
RTX A600048 GB	$4,89916 listings	125.4	256K
Mac Studio M5 Ultra 96GBWhole system · 96 GB	$7,27412 listings	148.3	256K
MacBook Pro M5 Max 128GBWhole system · 128 GB	$7,54920 listings	113.7	256K
DGX Spark 128GBWhole system · 128 GB	$8,24022 listings	71.9	256K
GeForce RTX 509032 GB	$8,395123 listings	163.9	256K
RTX PRO 600096 GB	$22,50011 listings	163.9	256K

Only common GPUs and systems are listed, one set for models under 50B and another for larger ones, sorted by price from low to high.

Detailed calculation rules are on the Methodology page.

As an eBay Partner, VRAMGlass may be compensated if you buy through the eBay links on this page. The price you pay stays the same.

1. 1

   Sampling parameters differ from the 27B

   For general tasks in thinking mode, Qwen recommends temperature 1.0, top_p 0.95, top_k 20 and presence_penalty 1.5 (the 27B of the same generation uses 0 here); for coding, temperature 0.6 and presence_penalty 0; in non-thinking mode, 0.7, 0.8 and 20, with presence_penalty 1.5.

2. 2

   Which quantization to pick

   It reads only 3B parameters per token, so the quant level affects its speed less than a dense model's, but affects its size just as much. On a 24GB card, use 4-bit (about 22.1 GB); on cards with 48GB or more, or unified-memory machines, go straight to 8-bit (about 36.9 GB), which still runs at about 99 tokens/s on an RTX A6000.

3. 3

   MTP adds another step of speed

   llama.cpp now supports MTP speculative decoding, and the MTP GGUF takes only about 1 GB more. Unsloth says this model runs at about 240 tokens/s with MTP on an “RTX 6000,” without saying which generation of card or which quant; that's Unsloth's number, and this site's speeds don't include MTP. The Ollama tag is qwen3.6:35b-a3b.

4. 4

   Leave enough context for thinking

   Qwen recommends leaving at least 128K of context to preserve thinking quality. On a 24GB card, 4-bit tops out at about 123K, slightly under that recommendation; to leave the full amount, drop to 3-bit (about 20.6 GB at 128K) or switch to a 32GB card.

ollama run qwen3.6:35b-a3b

llama-server -hf unsloth/Qwen3.6-35B-A3B-GGUF:Q4_K_M -c 8192

llama-server -hf unsloth/Qwen3.6-35B-A3B-GGUF:Q4_K_M -c 8192 -ngl 99 \
  --cpu-moe

vllm serve Qwen/Qwen3.6-35B-A3B

vllm serve nvidia/Qwen3.6-35B-A3B-NVFP4

mlx_vlm.server --model mlx-community/Qwen3.6-35B-A3B-4bit

File names follow that week's model-library snapshot; the definitions are in the methodology. Methodology

Model	Total params	Active params	4-bit size
Qwen3 8BDense	8.2B	8.2B	5 GB
Qwen3.5 9BDense	9.7B	9.7B	5.7 GB
Qwen3.6 27BDense	27.8B	27.8B	16.8 GB
Qwen3.8 27BDense	27.8B	27.8B	16.5 GB
Qwen3 32BDense	32.8B	32.8B	19.8 GB
Qwen3.8 Flash NextMoE	176B	6B	111.3 GB
Qwen3.8 2.4T A95BMoE	2,446B	95B	1,310.9 GB

About Qwen