VRAMGlass contains affiliate links. As an eBay Partner, we may be compensated if you make a purchase.As an eBay Partner, we may be compensated.Learn more

GPUs tracked57Price history49 daysPrice points (24h)43,400 pointsBiggest 24h dropGeForce RTX 3060 12GB-11.1%Biggest 24h riseCMP 170HX 40GB+19.8%Updated
$ USD

How to Deploy gpt-oss 120B Locally

gpt-oss 120B is OpenAI's large open-weight MoE, with 116.8B parameters but only 5.1B used per token, so it runs fast on machines with lots of memory. At 4-bit with 8K it takes about 64 GB, so a 128GB machine holds it entirely, at about 52 to 54 tokens/s; with just one ordinary graphics card, it still runs if you keep the experts in system RAM. OpenAI says a single 80GB H100 is enough.

gpt-oss 120B details

  • Min VRAM at 4-bit65 GB · 8K
  • Parameters116.8B
  • Active params5.1B
  • Max context128K
Architecture
MoE
Experts offloaded
4 GB VRAM + 62 GB RAM
License
Apache-2.0
Launch date
Aug 5, 2025
Vendor
OpenAI
Hugging Face repo
openai/gpt-oss-120b
GGUF repo
unsloth/gpt-oss-120b-GGUF
MLX repo (Apple)
mlx-community/gpt-oss-120b-MXFP4-Q8
Ollama tag
gpt-oss:120b
Layers
36
Hidden size
2,880
KV heads
8
Head dim
64

How much VRAM does gpt-oss 120B need for local deployment?

QuantizationWeightsGB8K contextGB32K contextGB128K context (max)GB
1-bit29.4est.30.831.735.4
2-bit46.6est.4848.952.5
3-bit62.6GGUF6464.968.5
4-bit62.8GGUF64.265.168.7
5-bit62.9GGUF64.365.268.8
8-bit63.4GGUF64.865.769.3
BF16/FP1665.4GGUF66.867.771.3

Full calculation rules are on the Methodology page.

Can a single GPU run gpt-oss 120B?

No consumer GPU can; it takes a workstation or data center card, or a big unified-memory machine. The RTX PRO 6000 holds 4-bit with the full context window to spare, at about 147 tokens/s, and the 80GB A100 and 72GB RTX PRO 5000 fit it too. Don't count on a smaller quant to squeeze it onto a lesser card: the model ships in a 4-bit format, so files from 2-bit to 8-bit are nearly the same size, and the 48GB RTX A6000 still can't fit it. Among all-in-one machines, the 96GB Mac Studio M5 Ultra does about 129 tokens/s and the 128GB DGX Spark about 54, because with only about 5B parameters used per token, there's little data to read and modest bandwidth goes a long way.

eBay
DevicePriceSpeed t/sMax context
Ryzen AI Max+ 395 128GBWhole system · 128 GB$4,46420 listings51.5128K
Instinct MI21064 GB$4,99411 listings142.8128K
Mac Studio M5 Ultra 96GBWhole system · 96 GB$7,27412 listings128.7128K
MacBook Pro M5 Max 128GBWhole system · 128 GB$7,54920 listings92.2128K
DGX Spark 128GBWhole system · 128 GB$8,24022 listings54128K
RTX PRO 5000 72GB72 GB$10,7223 listings133.2128K
A100 80GB PCIe80 GB$18,99912 listings150.4128K
RTX PRO 600096 GB$22,50011 listings147128K
Mac Studio M3 Ultra 512GBWhole system · 512 GB$22,90013 listings107.5128K

Only common GPUs and systems are listed, one set for models under 50B and another for larger ones, sorted by price from low to high.

How many GPUs do you need to run gpt-oss 120B?

A pair of 48GB RTX A6000 cards is all it takes, running 4-bit at about 97 tokens/s. 24GB cards need four: the model plus 32K of context comes to about 65 GB, more than two of them can hold, and four RTX 3090s do about 107 tokens/s. The 32GB RTX 5090 also starts at four. Dropping to 2-bit won't save a card, since every quant is about the same size. One caveat: vLLM's gpt-oss recipe currently lists data center parts such as the H100, B200 and MI300, with support for GeForce cards and the A6000 still in progress. On those cards, splitting the layers across GPUs with llama.cpp is the safer route.

eBay
DeviceCardsTotal priceTotal VRAMSpeed t/sTotal power
Tesla V100 32GB32 GB per card4 cards$3,580128 GB104.91,000 W
Radeon RX 7900 XTX24 GB per card4 cards$5,00096 GB108.31,420 W
GeForce RTX 309024 GB per card4 cards$7,19696 GB1071,400 W
Arc Pro B7032 GB per card4 cards$7,400128 GB84.4920 W
RTX A600048 GB per card2 cards$9,79896 GB96.5600 W
Instinct MI21064 GB per card2 cards$9,988128 GB136600 W
GeForce RTX 409024 GB per card4 cards$15,03296 GB110.91,800 W
GeForce RTX 509032 GB per card4 cards$33,580128 GB140.32,300 W
GeForce RTX 4090 48GB48 GB per card2 cardsNo price96 GB110.9900 W

Full calculation rules are on the Methodology page.

What PC or server build do you need to run gpt-oss 120B locally?

This model uses only 5.1B parameters per token, so very little expert data has to be read from system RAM, which makes expert offloading its most practical way to run. At 4-bit, a 16GB Tesla V100 with a dual-channel DDR5 desktop does about 36 tokens/s, and with DDR4 about 21; a used 8-channel DDR4 server with an RTX 3090 does about 59. In llama.cpp, --cpu-moe puts all the experts in system RAM, while --n-cpu-moe N moves only the first N layers; if you have VRAM to spare, lower N for more speed.

eBay

GeForce RTX 5070 + Desktop DDR5

Home PC
Generation speed32.6t/s
Memory bandwidth96GB/s
GPU
GeForce RTX 50705 / 36 expert layers in 12 GB VRAM
$899
RAM
64 GB DDR5-60004 × 16 GB sticks
$900
CPU
AMD Ryzen 7 77008 cores / 16 threads · 2-channel
$212
Board
Gigabyte B650 EAGLE AX4 DIMM slots
$114
PSU
Corsair RM750e750 W
$70
Reference build price$2,195

Tesla V100 32GB + Dual Xeon E5 DDR4

Budget
Generation speed51.1t/s
Memory bandwidth137GB/s
GPU
Tesla V100 32GB18 / 36 expert layers in 32 GB VRAM
$895
RAM
128 GB DDR4-21338 × 16 GB sticks
$632
CPU
2 × Intel Xeon E5-2680 v428 cores / 56 threads · 8-channel in total
$38
Board
Supermicro X10DRi16 DIMM slots
$193
PSU
Corsair RM750e750 W
$70
Reference build price$1,828

GeForce RTX 3090 + Single EPYC DDR4

Best value
Generation speed59.3t/s
Memory bandwidth171GB/s
GPU
GeForce RTX 309013 / 36 expert layers in 24 GB VRAM
$1,799
RAM
128 GB DDR4-26668 × 16 GB sticks
$944
CPU
AMD EPYC 753232 cores / 64 threads · 8-channel
$262
Board
Supermicro H12SSL-i8 DIMM slots
$1,100
PSU
Corsair RM850x850 W
$70
Reference build price$4,175

GeForce RTX 5090 + Single EPYC DDR5

Flagship
Generation speed111.6t/s
Memory bandwidth461GB/s
GPU
GeForce RTX 509018 / 36 expert layers in 32 GB VRAM
$8,395
RAM
192 GB DDR5-480012 × 16 GB sticks
$7,164
CPU
AMD EPYC 935432 cores / 64 threads · 12-channel
$1,693
Board
Supermicro H13SSL-N12 DIMM slots
$1,000
PSU
Corsair RM1200x SHIFT1,200 W
$135
Reference build price$18,387

Detailed calculation rules are on the Methodology page.

As an eBay Partner, VRAMGlass may be compensated if you buy through the eBay links on this page. The price you pay stays the same.

What to know before running gpt-oss 120B locally

  1. Just use the original precision

    Its experts already come in a 4-bit format (MXFP4), so every GGUF quant is almost the same size; even the FP16 one is only about 65 GB, the same as the original. There's no point stepping down a level to save two or three GB. Unsloth recommends at least 66 GB of VRAM and RAM combined to get above 6 tokens per second.

  2. Weaker in Chinese, strong at code and reasoning

    Qwen used gpt-oss 120B as a comparison point on the Qwen3.5 9B model card. On the Chinese benchmark C-Eval it scores only 76.2, below the 9B Qwen3.5 (88.2), but on the coding benchmark LiveCodeBench v6 it scores 82.7, the highest in that comparison table. If your use is mainly in Chinese, test it on your own tasks before deploying.

  3. Reasoning effort and tools

    Like the 20B, it has three levels, low, medium and high, set in the system prompt (for example “Reasoning: high”). It was trained with browser and Python tool calling, and adding --tool-call-parser openai --enable-auto-tool-choice in vLLM hooks up function calling. Sampling parameters are the same as for the 20B: temperature 1.0 and top_p 1.0.

  4. If it fits, run the full 128K

    The full 128K takes only about 4.5 GB more than 8K (68.7 vs. 64.2 GB at 4-bit), so any device that holds it can go straight to 128K, with no trade-off between quantization and context. The reason is that half its layers only look at the most recent 128 tokens, and their cache doesn't grow with context.

Launch commands

Ollama

  • Windows
  • macOS
  • Linux
Terminal
ollama run gpt-oss:120b

llama.cpp

  • NVIDIA
  • AMD
  • Mac
  • CPU
All on GPU
Terminal
llama-server -hf unsloth/gpt-oss-120b-GGUF:Q4_K_M -c 8192
MoE offloading

Short on VRAM? Add --cpu-moe to keep the routed experts in system RAM (about 62 GB needed).

Terminal
llama-server -hf unsloth/gpt-oss-120b-GGUF:Q4_K_M -c 8192 -ngl 99 \
  --cpu-moe

vLLM

  • Linux
  • NVIDIA
  • AMD
Terminal
vllm serve openai/gpt-oss-120b

MLX

  • Apple Silicon

On a Mac, pip install mlx-lm first; this command downloads the MLX weights from Hugging Face and starts an OpenAI-compatible server.

Terminal
mlx_lm.server --model mlx-community/gpt-oss-120b-MXFP4-Q8

File names follow that week's model-library snapshot; the definitions are in the methodology. Methodology

Other models in the gpt-oss family

ModelTotal paramsActive params4-bit size
gpt-oss 20BMoE20.9B3.6B11.6 GB
gpt-oss Safeguard 20BMoE20.9B3.6B11.6 GB

About gpt-oss