GeForce RTX 5070 + Desktop DDR5
Home PCGeneration speed66.3t/s
Memory bandwidth96GB/s
Reference build price$1,745
How to Deploy Gemma 4 26B A4B Locally
Gemma 4 26B A4B is the only MoE in the Gemma 4 lineup, and it uses just 3.8B parameters per token. That makes it nearly as fast as a 4B model, while its scores come close to the 31B dense version: on Google's model card, it trails the 31B by only 1 to 3 points on most tests. At 4-bit with 8K it takes about 18.4 GB, so a 24GB card is the best fit; on a 16GB card you need to drop to 2-bit or move a few experts into system RAM.
Gemma 4 26B A4B details
- Min VRAM at 4-bit19 GB · 8K
- Parameters25.2B
- Active params3.8B
- Max context256K
- Architecture
- MoE
- Experts offloaded
- 5 GB VRAM + 16 GB RAM
- License
- Apache-2.0
- Launch date
- Apr 2, 2026
- Vendor
- Google DeepMind
- Hugging Face repo
- google/gemma-4-26B-A4B-it
- GGUF repo
- unsloth/gemma-4-26B-A4B-it-GGUF
- NVFP4 repo (Blackwell)
- nvidia/Gemma-4-26B-A4B-NVFP4
- MLX repo (Apple)
- mlx-community/gemma-4-26b-a4b-it-4bit
- Ollama tag
- gemma4:26b
- Layers
- 30
- Hidden size
- 2,816
- KV heads
- 8
- Head dim
- 256
How much VRAM does Gemma 4 26B A4B need for local deployment?
| Quantization | WeightsGB | 8K contextGB | 32K contextGB | 128K contextGB | 256K context (max)GB |
|---|---|---|---|---|---|
| 1-bit | 6.4est. | 7.8 | 8.3 | 10.3 | 13 |
| 2-bit | 10.5GGUF | 12 | 12.5 | 14.5 | 17.2 |
| 3-bit | 12.9GGUF | 14.4 | 14.9 | 16.9 | 19.6 |
| 4-bit | 16.9GGUF | 18.4 | 18.9 | 20.9 | 23.6 |
| 5-bit | 21.2GGUF | 22.7 | 23.2 | 25.2 | 27.9 |
| 8-bit | 26.9GGUF | 28.4 | 28.9 | 30.9 | 33.6 |
| BF16/FP16 | 50.5GGUF | 52 | 52.5 | 54.5 | 57.2 |
Full calculation rules are on the Methodology page.
Can a single GPU run Gemma 4 26B A4B?
Yes, and 24GB cards are its natural home: 4-bit with the full 256K context comes to about 23.6 GB, a snug fit on an RTX 3090, 4090 or RX 7900 XTX. Each token reads only 3.8B parameters, so the RTX 3090 turns out about 118 tokens/s and replies feel instant. 16GB cards can't hold 4-bit; drop to 3-bit or 2-bit and it's still fast, or keep 4-bit and move a few layers of experts into system RAM. A 32GB RTX 5090 can go straight to 8-bit. The 128GB unified-memory machines handle it well too, with the DGX Spark doing 50-plus tokens/s at 4-bit, because reading so few parameters hides their low memory bandwidth.
| Device | Price | Speed t/s | Max context |
|---|---|---|---|
 Tesla V100 16GB16 GB | $37024 listings | 137.7 | 256K |
| GeForce RTX 5060 Ti 16GB16 GB | $86958 listings | 101.5 | 256K |
| Tesla V100 32GB32 GB | $89518 listings | 137.7 | 256K |
| Radeon RX 7900 XTX24 GB | $1,25029 listings | 140.8 | 256K |
| GeForce RTX 309024 GB | $1,799107 listings | 139.6 | 256K |
| GeForce RTX 409024 GB | $3,75868 listings | 143.1 | 256K |
| Ryzen AI Max+ 395 128GBWhole system · 128 GB | $4,46420 listings | 72.9 | 256K |
| RTX A600048 GB | $4,89916 listings | 129.8 | 256K |
| Mac Studio M5 Ultra 96GBWhole system · 96 GB | $7,27412 listings | 152 | 256K |
| MacBook Pro M5 Max 128GBWhole system · 128 GB | $7,54920 listings | 118.3 | 256K |
| DGX Spark 128GBWhole system · 128 GB | $8,24022 listings | 76.1 | 256K |
| GeForce RTX 509032 GB | $8,395123 listings | 167 | 256K |
| RTX PRO 600096 GB | $22,50011 listings | 167 | 256K |
| Device | Price | Speed t/s | Max context |
|---|---|---|---|
| Tesla V100 32GB32 GB | $89518 listings | 115.5 | 256K |
| Radeon RX 7900 XTX24 GB | $1,25029 listings | 118.8 | 256K |
| GeForce RTX 309024 GB | $1,799107 listings | 117.5 | 256K |
| GeForce RTX 409024 GB | $3,75868 listings | 121.4 | 256K |
| Ryzen AI Max+ 395 128GBWhole system · 128 GB | $4,46420 listings | 53.7 | 256K |
| RTX A600048 GB | $4,89916 listings | 107 | 256K |
| Mac Studio M5 Ultra 96GBWhole system · 96 GB | $7,27412 listings | 131.6 | 256K |
| MacBook Pro M5 Max 128GBWhole system · 128 GB | $7,54920 listings | 95.2 | 256K |
| DGX Spark 128GBWhole system · 128 GB | $8,24022 listings | 56.3 | 256K |
| GeForce RTX 509032 GB | $8,395123 listings | 149.5 | 256K |
| RTX PRO 600096 GB | $22,50011 listings | 149.5 | 256K |
| Device | Price | Speed t/s | Max context |
|---|---|---|---|
| Tesla V100 32GB32 GB | $89518 listings | 92.2 | 256K |
| Ryzen AI Max+ 395 128GBWhole system · 128 GB | $4,46420 listings | 38 | 256K |
| RTX A600048 GB | $4,89916 listings | 84 | 256K |
| Mac Studio M5 Ultra 96GBWhole system · 96 GB | $7,27412 listings | 108.7 | 256K |
| MacBook Pro M5 Max 128GBWhole system · 128 GB | $7,54920 listings | 72.9 | 256K |
| DGX Spark 128GBWhole system · 128 GB | $8,24022 listings | 40.1 | 256K |
| GeForce RTX 509032 GB | $8,395123 listings | 128.4 | 256K |
| RTX PRO 600096 GB | $22,50011 listings | 128.4 | 256K |
Only common GPUs and systems are listed, one set for models under 50B and another for larger ones, sorted by price from low to high.
What PC or server build do you need to run Gemma 4 26B A4B locally?
If you'd rather not drop to 2-bit on a 16GB card, you can move just a few layers of experts into system RAM. At 4-bit, a DDR4 desktop with a 16GB V100 does about 79 tokens/s and a DDR5 one about 94, with 25 of the 30 layers of experts still in VRAM (llama.cpp flag --n-cpu-moe 5). On the GPU pages, the cell for 16GB cards says about 32 to 39 tokens/s; that estimate assumes all experts in system RAM (the --cpu-moe command on this page). Offloading only 5 layers is more than twice as fast. At 8-bit, an 8-channel DDR4 server with an RTX 3090 does about 81 tokens/s.
2 × GeForce RTX 5070 + Desktop DDR5
Home PCGeneration speed60t/s
Memory bandwidth96GB/s
Reference build price$2,644
GeForce RTX 3090 + Single EPYC DDR4
Best valueGeneration speed81.4t/s
Memory bandwidth171GB/s
Reference build price$4,175
Detailed calculation rules are on the Methodology page.
As an eBay Partner, VRAMGlass may be compensated if you buy through the eBay links on this page. The price you pay stays the same.
What to know before running Gemma 4 26B A4B locally
Which quantization to pick
On a 24GB card, use 4-bit (16.9 GB of weights) or 5-bit (about 22.7 GB at 8K, leaving less room for context). 8-bit with 8K is about 28.4 GB and needs a 32GB card. Because it reads only 3.8B parameters per token, 8-bit still runs at about 128 tokens/s on an RTX 5090.
It sees images but can't hear audio
26B A4B and 31B include a vision encoder of about 550 million parameters, so they handle images and video frames, but not audio. The 25.2B on this site follows the model card and excludes that encoder. For audio, switch to E4B or 12B.
Fixes in the first week
llama.cpp supported Gemma 4 on launch day, April 2, 2026, but fixed several issues over the following week, including the tokenizer and chat template. If you converted GGUFs or built llama.cpp in early April, update them. The Ollama tag is gemma4:26b; the thinking toggle and sampling parameters are the same as for Gemma 4 12B.
Launch commands
Ollama
- Windows
- macOS
- Linux
ollama run gemma4:26bllama.cpp
- NVIDIA
- AMD
- Mac
- CPU
All on GPU
llama-server -hf unsloth/gemma-4-26B-A4B-it-GGUF:Q4_K_M -c 8192MoE offloading
Short on VRAM? Add --cpu-moe to keep the routed experts in system RAM (about 16 GB needed).
llama-server -hf unsloth/gemma-4-26B-A4B-it-GGUF:Q4_K_M -c 8192 \ -ngl 99 --cpu-moe
vLLM
- Linux
- NVIDIA
- AMD
Original weights
vllm serve google/gemma-4-26B-A4B-itNVFP4 · Blackwell
Blackwell GPUs (RTX 50 series, RTX PRO 6000, B200) run NVFP4 weights natively, at about 30% of the original size.
vllm serve nvidia/Gemma-4-26B-A4B-NVFP4MLX
- Apple Silicon
Vision models run on mlx-vlm: on a Mac, pip install mlx-vlm first; this command downloads the MLX weights from Hugging Face and starts a server.
mlx_vlm.server --model mlx-community/gemma-4-26b-a4b-it-4bitFile names follow that week's model-library snapshot; the definitions are in the methodology. Methodology
Other models in the Gemma family
| Model | Total params | Active params | 4-bit size |
|---|---|---|---|
| Gemma 4 E4BDense | 8B | 8B | 5 GB |
| Gemma 4 12BDense | 12B | 12B | 7.1 GB |
| Gemma 4 31BDense | 30.7B | 30.7B | 18.3 GB |