How to Deploy gpt-oss 20B Locally

• Min VRAM at 4-bit13 GB · 8K
• Parameters20.9B
• Active params3.6B
• Max context128K

Architecture

MoE

Experts offloaded

3 GB VRAM + 11 GB RAM

License

Apache-2.0

Launch date

Aug 5, 2025

Vendor

OpenAI

Hugging Face repo

openai/gpt-oss-20b

GGUF repo

unsloth/gpt-oss-20b-GGUF

MLX repo (Apple)

mlx-community/gpt-oss-20b-MXFP4-Q8

Ollama tag

gpt-oss:20b

Layers

24

Hidden size

2,880

KV heads

8

Head dim

64

Full calculation rules are on the Methodology page.

Device	Price	Speed t/s	Max context
Tesla V100 16GB16 GB	$37024 listings	129.2	128K
GeForce RTX 5060 Ti 16GB16 GB	$86958 listings	92.5	128K
Tesla V100 32GB32 GB	$89518 listings	129.2	128K
Radeon RX 7900 XTX24 GB	$1,25029 listings	132.5	128K
GeForce RTX 309024 GB	$1,799107 listings	131.2	128K
GeForce RTX 409024 GB	$3,75868 listings	134.9	128K
Ryzen AI Max+ 395 128GBWhole system · 128 GB	$4,46420 listings	65	128K
RTX A600048 GB	$4,89916 listings	121	128K
Mac Studio M5 Ultra 96GBWhole system · 96 GB	$7,27412 listings	144.4	128K
MacBook Pro M5 Max 128GBWhole system · 128 GB	$7,54920 listings	109.2	128K
DGX Spark 128GBWhole system · 128 GB	$8,24022 listings	67.9	128K
GeForce RTX 509032 GB	$8,395123 listings	160.6	128K
RTX PRO 600096 GB	$22,50011 listings	160.6	128K

Only common GPUs and systems are listed, one set for models under 50B and another for larger ones, sorted by price from low to high.

Detailed calculation rules are on the Methodology page.

As an eBay Partner, VRAMGlass may be compensated if you buy through the eBay links on this page. The price you pay stays the same.

1. 1

   Every quant is almost the same size

   Its expert weights always stay in a 4-bit format (MXFP4, about 4.25 bits per parameter), so the GGUFs from 3-bit to 8-bit all fall between 11.5 and 12.1 GB, and even the FP16 one is only 13.8 GB. If you have the VRAM, just use 8-bit or FP16. The 1-bit and 2-bit rows in the table (5.3 and 8.3 GB) are estimated from parameter count; no files that small exist, and the speeds in the 2-bit rows of the single-GPU table are estimated from those sizes too. The real 2-bit file is about as big as the 3-bit one; Unsloth says the low quants only compress a few layers such as attention, and results are nearly identical to the original.

2. 2

   It requires the harmony format

   OpenAI says it was trained only on the harmony format and won't work properly with other chat formats. Ollama and llama.cpp (with --jinja, which uses the model's built-in template) apply it automatically; mistakes usually come from building prompts by hand or using a front end that doesn't recognize the template. To hook up tool calling in vLLM, add --tool-call-parser openai --enable-auto-tool-choice.

3. 3

   Reasoning effort and sampling parameters

   Reasoning effort has three levels, low, medium and high; just write “Reasoning: high” in the system prompt. In llama.cpp you can also set a default by passing reasoning_effort through --chat-template-kwargs. Low answers quickly; high thinks longer and outputs more tokens. OpenAI recommends temperature 1.0 and top_p 1.0.

4. 4

   Long context adds almost no VRAM

   Half of its 24 layers only look at the most recent 128 tokens, and their cache doesn't grow with context; the native 4K context is extended to 128K with YaRN. So going from 8K to 128K, 4-bit VRAM rises only from 12.9 GB to 15.9 GB.

ollama run gpt-oss:20b

llama-server -hf unsloth/gpt-oss-20b-GGUF:Q4_K_M -c 8192

llama-server -hf unsloth/gpt-oss-20b-GGUF:Q4_K_M -c 8192 -ngl 99 \
  --cpu-moe

vllm serve openai/gpt-oss-20b

mlx_lm.server --model mlx-community/gpt-oss-20b-MXFP4-Q8

File names follow that week's model-library snapshot; the definitions are in the methodology. Methodology

About gpt-oss