Qwen3.8 Flash Next Local Deployment Guide: Setup & Hardware

Qwen3.8 Flash Next details

Min VRAM at 4-bit113 GB · 8K
Parameters176B
Active params6B
Max context256K

Architecture: MoE
Experts offloaded: 5 GB VRAM + 110 GB RAM
License: Qwen Community License 1.0
Launch date: Aug 26, 2026
Vendor: Alibaba
Hugging Face repo: Qwen/Qwen3.8-Flash-Next
GGUF repo: unsloth/Qwen3.8-Flash-Next-GGUF
NVFP4 repo (Blackwell): nvidia/Qwen3.8-Flash-Next-NVFP4
MLX repo (Apple): mlx-community/Qwen3.8-Flash-Next-4bit
Ollama tag: qwen3.8-flash-next:125b-a6b-q4_K_M
Layers: 48
Hidden size: 2,560
KV heads: 2
Head dim: 256

How much VRAM does Qwen3.8 Flash Next need for local deployment?

Quantization	WeightsGB	8K contextGB	32K contextGB	128K contextGB	256K context (max)GB
1-bit	74.5GGUF	75.8	76.4	78.8	82
2-bit	78.9GGUF	80.2	80.8	83.2	86.4
3-bit	90GGUF	91.3	91.9	94.3	97.5
4-bit	111.3GGUF	112.6	113.2	115.6	118.8
5-bit	158.3GGUF	159.6	160.2	162.6	165.8
8-bit	188.2GGUF	189.5	190.1	192.5	195.7
BF16/FP16	354GGUF	355.3	355.9	358.3	361.5

Full calculation rules are on the Methodology page.

Can a single GPU run Qwen3.8 Flash Next?

No single graphics card holds the 4-bit version; the only standalone machines that fit it are 128GB unified-memory systems. 4-bit with 8K of context is about 113 GB, so even the largest single card, the RTX PRO 6000, has to drop to 2-bit or 3-bit, though it stays remarkably quick, at about 149 tokens/s for 2-bit. Machines like the DGX Spark and Ryzen AI Max+ 395 hold 4-bit at a little over 40 tokens/s. The model touches only about 6B parameters per token, so their modest bandwidth isn't much of a handicap. With one ordinary GPU, keep most of the experts in system RAM and leave only the parts every step needs on the card.

Device	Price	Speed t/s	Max context
Ryzen AI Max+ 395 128GBWhole system · 128 GB	$4,46420 listings	52.9	256K
Mac Studio M5 Ultra 96GBWhole system · 96 GB	$7,27412 listings	130.5	256K
MacBook Pro M5 Max 128GBWhole system · 128 GB	$7,54920 listings	94.1	256K
DGX Spark 128GBWhole system · 128 GB	$8,24022 listings	55.5	256K
A100 80GB PCIe80 GB	$18,99912 listings	152	235K
RTX PRO 600096 GB	$22,50011 listings	148.6	256K
Mac Studio M3 Ultra 512GBWhole system · 512 GB	$22,90013 listings	109.4	256K

Device	Price	Speed t/s	Max context
Ryzen AI Max+ 395 128GBWhole system · 128 GB	$4,46420 listings	40.8	256K
MacBook Pro M5 Max 128GBWhole system · 128 GB	$7,54920 listings	77.1	256K
DGX Spark 128GBWhole system · 128 GB	$8,24022 listings	42.9	256K
Mac Studio M3 Ultra 512GBWhole system · 512 GB	$22,90013 listings	91.7	256K

Device	Price	Speed t/s	Max context
Mac Studio M3 Ultra 512GBWhole system · 512 GB	$22,90013 listings	66.3	256K

Only common GPUs and systems are listed, one set for models under 50B and another for larger ones, sorted by price from low to high.

How many GPUs do you need to run Qwen3.8 Flash Next?

For 4-bit, the simplest setup is two 96GB RTX PRO 6000s, at about 129 tokens/s. On consumer cards, 2-bit is the practical choice: four 24GB RTX 3090s or RX 7900 XTXs hold it, at about 111 tokens/s. 4-bit plus 32K of context overflows four 24GB cards, and the next step up is eight. Four 32GB RTX 5090s are just enough for 4-bit and match the two RTX PRO 6000s on speed. 8-bit takes four RTX PRO 6000s. Qwen publishes deployment recipes for vLLM and SGLang.

Device	Cards	Total price	Total VRAM	Speed t/s	Total power
Tesla V100 32GB32 GB per card	4 cards	$3,580	128 GB	109.1	1,000 W
Radeon RX 7900 XTX24 GB per card	4 cards	$5,000	96 GB	112.5	1,420 W
GeForce RTX 309024 GB per card	4 cards	$7,196	96 GB	111.1	1,400 W
Arc Pro B7032 GB per card	4 cards	$7,400	128 GB	88.4	920 W
RTX A600048 GB per card	2 cards	$9,798	96 GB	100.6	600 W
Instinct MI21064 GB per card	2 cards	$9,988	128 GB	139.8	600 W
GeForce RTX 409024 GB per card	4 cards	$15,032	96 GB	115.1	1,800 W
RTX PRO 5000 72GB72 GB per card	2 cards	$21,444	144 GB	130	600 W
GeForce RTX 509032 GB per card	4 cards	$33,580	128 GB	144	2,300 W
A100 80GB PCIe80 GB per card	2 cards	$37,998	160 GB	147.5	600 W
GeForce RTX 4090 48GB48 GB per card	2 cards	No price	96 GB	115.1	900 W

Device	Cards	Total price	Total VRAM	Speed t/s	Total power
Tesla V100 32GB32 GB per card	4 cards	$3,580	128 GB	92.8	1,000 W
Arc Pro B7032 GB per card	4 cards	$7,400	128 GB	73.1	920 W
Instinct MI21064 GB per card	2 cards	$9,988	128 GB	124.5	600 W
Radeon RX 7900 XTX24 GB per card	8 cards	$10,000	192 GB	96.2	2,840 W
GeForce RTX 309024 GB per card	8 cards	$14,392	192 GB	94.9	2,800 W
RTX A600048 GB per card	4 cards	$19,596	192 GB	84.6	1,200 W
RTX PRO 5000 72GB72 GB per card	2 cards	$21,444	144 GB	114.1	600 W
GeForce RTX 409024 GB per card	8 cards	$30,064	192 GB	98.8	3,600 W
GeForce RTX 509032 GB per card	4 cards	$33,580	128 GB	129	2,300 W
A100 80GB PCIe80 GB per card	2 cards	$37,998	160 GB	132.9	600 W
RTX PRO 600096 GB per card	2 cards	$45,000	192 GB	129	1,200 W
GeForce RTX 4090 48GB48 GB per card	4 cards	No price	192 GB	98.8	1,800 W

Device	Cards	Total price	Total VRAM	Speed t/s	Total power
Tesla V100 32GB32 GB per card	8 cards	$7,160	256 GB	68.6	2,000 W
Arc Pro B7032 GB per card	8 cards	$14,800	256 GB	51.8	1,840 W
DGX Spark 128GB128 GB per unit	2 units	$16,480	256 GB	26.9	480 W
Instinct MI21064 GB per card	4 cards	$19,976	256 GB	98.8	1,200 W
RTX A600048 GB per card	8 cards	$39,192	384 GB	61.5	2,400 W
RTX PRO 5000 72GB72 GB per card	4 cards	$42,888	288 GB	88.4	1,200 W
GeForce RTX 509032 GB per card	8 cards	$67,160	256 GB	103.5	4,600 W
A100 80GB PCIe80 GB per card	4 cards	$75,996	320 GB	107.6	1,200 W
RTX PRO 600096 GB per card	4 cards	$90,000	384 GB	103.5	2,400 W
GeForce RTX 4090 48GB48 GB per card	8 cards	No price	384 GB	74	3,600 W

Full calculation rules are on the Methodology page.

What PC or server build do you need to run Qwen3.8 Flash Next locally?

When VRAM runs short, keep the experts in system RAM: the GPU holds only the shared parts every token uses plus a few experts, and the rest stays in RAM. Speed then depends mainly on memory bandwidth: at 4-bit, a dual-channel DDR4 desktop does about 17 tokens/s, a DDR5 desktop about 29, and a used 8-channel DDR4 server about 46. There's also Strata, an engine written specifically for this model, which runs the 2-bit version on a 12GB card with 64GB of RAM at more than twice the speed of llama.cpp.

GeForce RTX 5070 + Desktop DDR5

Home PC

Generation speed27.2t/s

Memory bandwidth96GB/s

GPU: GeForce RTX 50703 / 48 expert layers in 12 GB VRAM; $899
RAM: 128 GB DDR5-60004 × 32 GB sticks; $1,952
CPU: AMD Ryzen 7 77008 cores / 16 threads · 2-channel; $212
Board: Gigabyte B650 EAGLE AX4 DIMM slots; $114
PSU: Corsair RM750e750 W; $70

Reference build price$3,247

Tesla V100 32GB + Dual Xeon E5 DDR4

Budget

Generation speed36.3t/s

Memory bandwidth137GB/s

GPU: Tesla V100 32GB13 / 48 expert layers in 32 GB VRAM; $895
RAM: 128 GB DDR4-21338 × 16 GB sticks; $632
CPU: 2 × Intel Xeon E5-2680 v428 cores / 56 threads · 8-channel in total; $38
Board: Supermicro X10DRi16 DIMM slots; $193
PSU: Corsair RM750e750 W; $70

Reference build price$1,828

GeForce RTX 3090 + Dual EPYC DDR4

Best value

Generation speed61.1t/s

Memory bandwidth341GB/s

GPU: GeForce RTX 30909 / 48 expert layers in 24 GB VRAM; $1,799
RAM: 256 GB DDR4-266616 × 16 GB sticks; $1,888
CPU: 2 × AMD EPYC 753264 cores / 128 threads · 16-channel in total; $524
Board: Supermicro H12DSi-N616 DIMM slots; $1,100
PSU: Corsair RM1200x SHIFT1,200 W; $135

Reference build price$5,446

GeForce RTX 5090 + Single EPYC DDR5

Flagship

Generation speed90.8t/s

Memory bandwidth461GB/s

GPU: GeForce RTX 509013 / 48 expert layers in 32 GB VRAM; $8,395
RAM: 192 GB DDR5-480012 × 16 GB sticks; $7,164
CPU: AMD EPYC 935432 cores / 64 threads · 12-channel; $1,693
Board: Supermicro H13SSL-N12 DIMM slots; $1,000
PSU: Corsair RM1200x SHIFT1,200 W; $135

Reference build price$18,387

Detailed calculation rules are on the Methodology page.

As an eBay Partner, VRAMGlass may be compensated if you buy through the eBay links on this page. The price you pay stays the same.

What to know before running Qwen3.8 Flash Next locally

Why we list 176B parameters
The official 125B doesn't include the 51B n-gram lookup table or the 4B multi-token prediction layer. This site estimates VRAM from 125B plus the 51B lookup table, 176B in total, because the table takes up space too; but during generation it is only read as needed, so speed is still estimated from the 6B used per token.
Which quantization to pick
If RAM or VRAM is tight, pick 2-bit (about 79 GB) or 3-bit (about 90 GB), and move up to 4-bit (about 111 GB) if you have room; if answer quality matters, don't go below 3-bit. The quants Strata uses were made by ISTA-DASLab, who say their IQ3_S matches the original model on public benchmarks; this site hasn't re-tested that.
Which framework to use
For servers, Qwen recommends SGLang, KTransformers or vLLM. On a personal computer, llama.cpp and Ollama are the easiest; the Ollama tag is qwen3.8-flash-next:125b-a6b-q4_K_M. For speed on a 12GB to 24GB card, use Strata.
Official sampling parameters
Thinking mode: temperature 1.0, top_p 0.95, top_k 20. Non-thinking mode: temperature 0.7, top_p 0.8, top_k 20, presence_penalty 1.5.
Long context takes almost no VRAM
Native context is 256K, extendable to 1M. Most of its layers use attention that doesn't grow with context, so the context cache is small: at 4-bit, going from 8K to 256K adds only about 6 GB of VRAM.

Launch commands

Ollama

Windows
macOS
Linux

Terminal

ollama run qwen3.8-flash-next:125b-a6b-q4_K_M

llama.cpp

NVIDIA
AMD
Mac
CPU

All on GPU

Terminal

llama-server -hf unsloth/Qwen3.8-Flash-Next-GGUF:UD-Q4_K_XL -c 8192

MoE offloading

Short on VRAM? Add --cpu-moe to keep the routed experts in system RAM (about 110 GB needed).

Terminal

llama-server -hf unsloth/Qwen3.8-Flash-Next-GGUF:UD-Q4_K_XL -c 8192 \
  -ngl 99 --cpu-moe

vLLM

Linux
NVIDIA
AMD

Original weights

Terminal

vllm serve Qwen/Qwen3.8-Flash-Next

NVFP4 · Blackwell

Blackwell GPUs (RTX 50 series, RTX PRO 6000, B200) run NVFP4 weights natively, at about 30% of the original size. For multiple GPUs, add --tensor-parallel-size with the card count.

Terminal

vllm serve nvidia/Qwen3.8-Flash-Next-NVFP4

MLX

Apple Silicon

Vision models run on mlx-vlm: on a Mac, pip install mlx-vlm first; this command downloads the MLX weights from Hugging Face and starts a server.

Terminal

mlx_vlm.server --model mlx-community/Qwen3.8-Flash-Next-4bit

File names follow that week's model-library snapshot; the definitions are in the methodology. Methodology

Other models in the Qwen family

Model	Total params	Active params	4-bit size
Qwen3 8BDense	8.2B	8.2B	5 GB
Qwen3.5 9BDense	9.7B	9.7B	5.7 GB
Qwen3.6 27BDense	27.8B	27.8B	16.8 GB
Qwen3.8 27BDense	27.8B	27.8B	16.5 GB
Qwen3 32BDense	32.8B	32.8B	19.8 GB
Qwen3.6 35B A3BMoE	36B	3B	22.1 GB
Qwen3.8 2.4T A95BMoE	2,446B	95B	1,310.9 GB

About Qwen

How to Deploy Qwen3.8 Flash Next Locally

Qwen3.8 Flash Next details

How much VRAM does Qwen3.8 Flash Next need for local deployment?

Can a single GPU run Qwen3.8 Flash Next?

How many GPUs do you need to run Qwen3.8 Flash Next?

What PC or server build do you need to run Qwen3.8 Flash Next locally?

GeForce RTX 5070 + Desktop DDR5

2 × Tesla V100 32GB + Dual Xeon E5 DDR4

GeForce RTX 3090 + Single EPYC DDR4

GeForce RTX 5090 + Single EPYC DDR5

GeForce RTX 5070 + Desktop DDR5

Tesla V100 32GB + Dual Xeon E5 DDR4

GeForce RTX 3090 + Dual EPYC DDR4

GeForce RTX 5090 + Single EPYC DDR5

Tesla V100 32GB + Dual Xeon E5 DDR4

GeForce RTX 3090 + Dual EPYC DDR4

GeForce RTX 5090 + Single EPYC DDR5

What to know before running Qwen3.8 Flash Next locally

Why we list 176B parameters

Which quantization to pick

Which framework to use

Official sampling parameters

Long context takes almost no VRAM

Launch commands

Ollama

llama.cpp

vLLM

MLX

Other models in the Qwen family