VRAMGlass contains affiliate links. As an eBay Partner, we may be compensated if you make a purchase.As an eBay Partner, we may be compensated.Learn more

GPUs tracked57Price history49 daysPrice points (24h)43,400 pointsBiggest 24h dropGeForce RTX 3060 12GB-11.1%Biggest 24h riseCMP 170HX 40GB+19.8%Updated
$ USD

How to Deploy Qwen3.8 27B Locally

Qwen3.8 27B is a 27B dense model that Qwen released in August 2026, and it understands images and video. It's now the first choice for running a Qwen dense model on a single graphics card. One 24GB card runs its 4-bit version at about 40 tokens/s with context up to about 122K; only 1 in every 4 layers sees the full context, so long context doesn't take much VRAM.

Qwen3.8 27B details

  • Min VRAM at 4-bit19 GB · 8K
  • Parameters27.8B
  • Max context256K
Architecture
Dense
License
Apache-2.0
Launch date
Aug 14, 2026
Vendor
Alibaba
Hugging Face repo
Qwen/Qwen3.8-27B
GGUF repo
unsloth/Qwen3.8-27B-GGUF
NVFP4 repo (Blackwell)
nvidia/Qwen3.8-27B-NVFP4
MLX repo (Apple)
mlx-community/Qwen3.8-27B-4bit
Ollama tag
qwen3.8:27b
Layers
64
Hidden size
5,120
KV heads
4
Head dim
256

How much VRAM does Qwen3.8 27B need for local deployment?

QuantizationWeightsGB8K contextGB32K contextGB128K contextGB256K context (max)GB
1-bit6.7GGUF8.39.916.425
2-bit9.8GGUF11.41319.528.1
3-bit13.1GGUF14.716.322.831.4
4-bit16.5GGUF18.119.726.234.8
5-bit19.8GGUF21.42329.538.1
8-bit29GGUF30.632.238.747.3
BF16/FP1654.7GGUF56.357.964.473

Full calculation rules are on the Methodology page.

Can a single GPU run Qwen3.8 27B?

Yes. A 24GB card is the natural starting point, and 16GB can squeeze it in. On a 24GB card, 4-bit runs at about 40 tokens/s with roughly 122K of context, and the 32GB RTX 5090 is nearly twice as fast. 16GB works because Unsloth's 2-bit build is only about 10 GB, small enough for a card like the Tesla V100, though answer quality takes a hit. By default it keeps the reasoning from every earlier turn in the conversation, so context fills fast in long chats and spare VRAM helps. 128GB unified-memory machines hold even 8-bit, but their bandwidth is low and a dense model reads all of its weights for every token, so 4-bit manages only about 12 tokens/s.

eBay
DevicePriceSpeed t/sMax context
Tesla V100 32GB32 GB$89518 listings38.9250K
Radeon RX 7900 XTX24 GB$1,25029 listings41.4122K
GeForce RTX 309024 GB$1,799107 listings40.4122K
GeForce RTX 409024 GB$3,75868 listings43.3122K
Ryzen AI Max+ 395 128GBWhole system · 128 GB$4,46420 listings11.5256K
RTX A600048 GB$4,89916 listings33.5256K
Mac Studio M5 Ultra 96GBWhole system · 96 GB$7,27412 listings52.1256K
MacBook Pro M5 Max 128GBWhole system · 128 GB$7,54920 listings27256K
DGX Spark 128GBWhole system · 128 GB$8,24022 listings12.3256K
GeForce RTX 509032 GB$8,395123 listings73.5250K
RTX PRO 600096 GB$22,50011 listings73.5256K

Only common GPUs and systems are listed, one set for models under 50B and another for larger ones, sorted by price from low to high.

What to know before running Qwen3.8 27B locally

  1. Three thinking levels

    Set thinking depth with reasoning_effort: xhigh (the default), medium or low. Qwen notes that in multi-turn agent tasks, a lower setting isn't necessarily faster: if each turn doesn't think enough, it's more likely to fail and retry. In llama.cpp, pass reasoning_effort via --chat-template-kwargs.

  2. Thinking is kept by default

    By default, 3.8 keeps the thinking from every earlier turn in the context, so context grows quickly in multi-turn conversations. Fortunately its context cache is very light: at 4-bit, going from 8K to 128K adds only about 8.1 GB. To keep only the latest turn's thinking, set preserve_thinking to false.

  3. Official sampling parameters

    Thinking mode: temperature 1.0, top_p 0.95, top_k 20, presence_penalty 0. Non-thinking mode: temperature 0.7, top_p 0.8, top_k 20, presence_penalty 1.5.

  4. Quantization and formats

    For GGUF, use Unsloth's Dynamic 3.0 version; 4-bit is about 16.5 GB, and the Ollama tag is qwen3.8:27b. RTX 50 series cards and the DGX Spark can run Unsloth's NVFP4 version in vLLM, which Unsloth says is about 1.5x faster than BF16; Qwen also offers an official FP8 version.

Launch commands

Ollama

  • Windows
  • macOS
  • Linux
Terminal
ollama run qwen3.8:27b

llama.cpp

  • NVIDIA
  • AMD
  • Mac
  • CPU
Terminal
llama-server -hf unsloth/Qwen3.8-27B-GGUF:Q4_K_M -c 8192

vLLM

  • Linux
  • NVIDIA
  • AMD
Original weights
Terminal
vllm serve Qwen/Qwen3.8-27B
NVFP4 · Blackwell

Blackwell GPUs (RTX 50 series, RTX PRO 6000, B200) run NVFP4 weights natively, at about 30% of the original size.

Terminal
vllm serve nvidia/Qwen3.8-27B-NVFP4

MLX

  • Apple Silicon

Vision models run on mlx-vlm: on a Mac, pip install mlx-vlm first; this command downloads the MLX weights from Hugging Face and starts a server.

Terminal
mlx_vlm.server --model mlx-community/Qwen3.8-27B-4bit

File names follow that week's model-library snapshot; the definitions are in the methodology. Methodology

Other models in the Qwen family

ModelTotal paramsActive params4-bit size
Qwen3 8BDense8.2B8.2B5 GB
Qwen3.5 9BDense9.7B9.7B5.7 GB
Qwen3.6 27BDense27.8B27.8B16.8 GB
Qwen3 32BDense32.8B32.8B19.8 GB
Qwen3.6 35B A3BMoE36B3B22.1 GB
Qwen3.8 Flash NextMoE176B6B111.3 GB
Qwen3.8 2.4T A95BMoE2,446B95B1,310.9 GB

About Qwen