How to Deploy Qwen3.8 2.4T A95B Locally

Qwen3.8 2.4T A95B is the first Max-class model Qwen has open-sourced, with about 2.4T parameters, text-only input and thinking mode only. It is so large that even 2-bit with 8K of context needs about 733 GB, more than any single GPU or complete machine can hold, and slightly too much even for eight 96GB cards. This site can work out only one single-machine setup: a 12-channel DDR5 server with every slot filled for 768GB of RAM plus one 96GB card, running 2-bit at about 21 tokens/s.

Qwen3.8 2.4T A95B details

Min VRAM at 4-bit1,313 GB · 8K
Parameters2,446B
Active params95B
Max context256K

Architecture: MoE
Experts offloaded: 29 GB VRAM + 1,286 GB RAM
License: Qwen3.8-Max License
Launch date: Aug 12, 2026
Vendor: Alibaba
Hugging Face repo: Qwen/Qwen3.8-2.4T-A95B
GGUF repo: unsloth/Qwen3.8-2.4T-A95B-GGUF
NVFP4 repo (Blackwell): nvidia/Qwen3.8-2.4T-A95B-NVFP4
Layers: 92
Hidden size: 8,192
KV heads: 4
Head dim: 256

How much VRAM does Qwen3.8 2.4T A95B need for local deployment?

Quantization	WeightsGB	8K contextGB	32K contextGB	128K contextGB	256K context (max)GB
1-bit	564GGUF	565.8	568.2	577.4	589.8
2-bit	730.7GGUF	732.5	734.9	744.1	756.5
3-bit	955.5GGUF	957.3	959.7	968.9	981.3
4-bit	1,310.9GGUF	1,312.7	1,315.1	1,324.3	1,336.7
5-bit	1,772.1est.	1,774	1,776.3	1,785.5	1,797.9
8-bit	2,600.2GGUF	2,602	2,604.4	2,613.6	2,626
BF16/FP16	4,893.2GGUF	4,895	4,897.4	4,906.6	4,919

Full calculation rules are on the Methodology page.

Can a single GPU run Qwen3.8 2.4T A95B?

No. No single card or standalone machine can hold it. Even the smallest 1-bit build is about 566 GB, more than the 512GB unified-memory Mac Studio M3 Ultra. Eight 96GB cards at 2-bit come up just short too, because each card can give only about 90% of its memory to the model and still has to make room for context. The one way to run it on a single machine is a 12-channel DDR5 server filled with 768GB of RAM plus one RTX PRO 6000, with most of the experts in system RAM, at about 21 tokens/s for 2-bit. In the data center, the smallest deployment is a third-party 4-bit build on a single 8-GPU B300 server.

How many GPUs do you need to run Qwen3.8 2.4T A95B?

Device	Cards	Total price	Total VRAM	Speed t/s	Total power
Mac Studio M3 Ultra 512GB512 GB per unit	2 units	$45,800	1,024 GB	19.1	960 W

Device	Cards	Total price	Total VRAM	Speed t/s	Total power
Mac Studio M3 Ultra 512GB512 GB per unit	4 units	$91,600	2,048 GB	11.3	1,920 W

Mac Studios link up over Thunderbolt 5 and run with MLX or exo; vLLM doesn't support them. Full calculation rules are on the Methodology page.

What PC or server build do you need to run Qwen3.8 2.4T A95B locally?

Only one setup can run it on a single machine: a 12-channel DDR5 server with every slot filled for 768GB of RAM (12 x 64GB sticks) plus one 96GB RTX PRO 6000. The GPU holds the experts from 11 of the 92 layers and everything else goes to RAM, for about 21 tokens/s at 2-bit. It reads 95B parameters per token, nearly 16 times as many as Qwen3.8 Flash Next, so speed is entirely bound by memory bandwidth. 4-bit with 8K of context is about 1,313 GB, too big even for this machine.

GeForce RTX 5090 + Dual EPYC DDR5

Flagship

Generation speed14.3t/s

Memory bandwidth922GB/s

GPU: GeForce RTX 50900 / 92 expert layers in 32 GB VRAM; $8,395
RAM: 1,536 GB DDR5-480024 × 64 GB sticks; $59,976
CPU: 2 × AMD EPYC 935464 cores / 128 threads · 24-channel in total; $3,386
Board: Gigabyte MZ73-LM024 DIMM slots; $1,300
PSU: Seasonic PRIME PX-16001,600 W; $516

Reference build price$73,573

8 × RTX PRO 6000 + Dual EPYC DDR5 8-GPU server

Giant models

Generation speed18.9t/s

Memory bandwidth922GB/s

GPU: 8 × RTX PRO 600056 / 92 expert layers in 768 GB VRAM; $180,000
RAM: 768 GB DDR5-480024 × 32 GB sticks; $29,328
CPU: 2 × AMD EPYC 935464 cores / 128 threads · 24-channel in total; $3,386
Barebone: ASUS ESC8000A-E128 GPU slots · 24 DIMM slots; $6,900
PSU: Built-in 4 × 3,000 W; Included

Reference build price$219,614

Detailed calculation rules are on the Methodology page.

As an eBay Partner, VRAMGlass may be compensated if you buy through the eBay links on this page. The price you pay stays the same.

What to know before running Qwen3.8 2.4T A95B locally

Thinking only, text only
The open weights don't accept image input and can't turn thinking off, so every answer starts with thinking output; image input and non-thinking mode are only available in Qwen3.8-Max on Qwen Cloud. Thinking depth can be set with reasoning_effort to xhigh (the default), medium or low; the official sampling parameters are temperature 1.0, top_p 0.95, top_k 20 and presence_penalty 0.
Quant levels and sizes
This site goes by the GGUF files: about 731 GB at 2-bit, 956 GB at 3-bit, 1,311 GB at 4-bit and 2,600 GB at 8-bit. Unsloth has also made more extreme 1-bit quants; the smallest, Q1_0, is listed at about 397 GB, but it needs a dedicated llama.cpp fork and at least 450 GB of RAM. The 1-bit row in this site's table uses the larger IQ1_M and leaves out Q1_0.
Server deployment
vLLM requires a nightly build, and transformers 5.4.0 or later. Per the vLLM recipe, the official FP8 version needs 16 B300s (two 8-GPU servers); the third-party NVFP4 version from Inferact fits on one 8-GPU B300 server, and the MXFP4 version for AMD runs on one 8-GPU MI355X or MI325X server. The number of GPUs for tensor parallelism has to divide the 64 attention heads evenly, so a 12-GPU setup won't work.
The license isn't Apache 2.0
It uses the Qwen3.8-Max License. Commercial products with more than 100 million monthly active users or monthly revenue above a threshold must prominently display the model name in their interface, and companies with revenue above a threshold that run model API services or AI coding and office assistants must first apply to Qwen for a separate license. Internal use that isn't open to third parties is exempt from this clause.
Long context is light
Only 23 of its 92 layers see the full context, so at 2-bit, going from 8K to 256K adds only about 24 GB, negligible next to more than 700 GB of weights. Native context is 256K, extendable to about 1M.

Launch commands

llama.cpp

NVIDIA
AMD
Mac
CPU

All on GPU

Terminal

llama-server -hf unsloth/Qwen3.8-2.4T-A95B-GGUF:UD-IQ4_XS -c 8192

MoE offloading

Short on VRAM? Add --cpu-moe to keep the routed experts in system RAM (about 1,286 GB needed).

Terminal

llama-server -hf unsloth/Qwen3.8-2.4T-A95B-GGUF:UD-IQ4_XS -c 8192 \
  -ngl 99 --cpu-moe

vLLM

Linux
NVIDIA
AMD

Original weights

Terminal

vllm serve Qwen/Qwen3.8-2.4T-A95B

NVFP4 · Blackwell

Blackwell GPUs (RTX 50 series, RTX PRO 6000, B200) run NVFP4 weights natively, at about 30% of the original size. For multiple GPUs, add --tensor-parallel-size with the card count.

Terminal

vllm serve nvidia/Qwen3.8-2.4T-A95B-NVFP4

Ollama

Windows
macOS
Linux

The Ollama library has no official tag for it yet.

File names follow that week's model-library snapshot; the definitions are in the methodology. Methodology

Other models in the Qwen family

Model	Total params	Active params	4-bit size
Qwen3 8BDense	8.2B	8.2B	5 GB
Qwen3.5 9BDense	9.7B	9.7B	5.7 GB
Qwen3.6 27BDense	27.8B	27.8B	16.8 GB
Qwen3.8 27BDense	27.8B	27.8B	16.5 GB
Qwen3 32BDense	32.8B	32.8B	19.8 GB
Qwen3.6 35B A3BMoE	36B	3B	22.1 GB
Qwen3.8 Flash NextMoE	176B	6B	111.3 GB

About Qwen

How to Deploy Qwen3.8 2.4T A95B Locally

Qwen3.8 2.4T A95B details

How much VRAM does Qwen3.8 2.4T A95B need for local deployment?

Can a single GPU run Qwen3.8 2.4T A95B?

How many GPUs do you need to run Qwen3.8 2.4T A95B?

What PC or server build do you need to run Qwen3.8 2.4T A95B locally?

Tesla V100 32GB + Dual Xeon E5 8-GPU server

GeForce RTX 5090 + Dual EPYC DDR5

4 × RTX PRO 6000 + Dual EPYC DDR5

GeForce RTX 5090 + Dual EPYC DDR5

8 × RTX PRO 6000 + Dual EPYC DDR5 8-GPU server

What to know before running Qwen3.8 2.4T A95B locally

Thinking only, text only

Quant levels and sizes

Server deployment

The license isn't Apache 2.0

Long context is light

Launch commands

llama.cpp

vLLM

Ollama

Other models in the Qwen family