What Do You Need to Run Gemma Locally? VRAM by Version
Gemma currently has 4 versions you can deploy locally. The smallest, Gemma 4 E4B, needs 7GB of VRAM at 4-bit; the largest, Gemma 4 31B, needs 21GB. The cheapest device that gets one running is the Tesla V100 16GB, at about $370.
| Version | Released | Context | License | ||
|---|---|---|---|---|---|
| Gemma 4 12BDense | Jun 3, 2026 | 12B | 12B | 256K | Apache-2.0 |
| Gemma 4 31BDense | Apr 2, 2026 | 30.7B | 30.7B | 256K | Apache-2.0 |
| Gemma 4 26B A4BMoE | Apr 2, 2026 | 25.2B | 3.8B | 256K | Apache-2.0 |
| Gemma 4 E4BDense | Apr 2, 2026 | 8B | 8B | 128K | Apache-2.0 |
About Gemma
Gemma is Google DeepMind's family of open-weight models. Google says it is built from the same research and technology as Gemini, and positions it as a lightweight model that runs on personal devices such as laptops and desktops. The first generation, released in February 2024, came in just two sizes, 2B and 7B. By Google's count, Gemma had been downloaded more than 400 million times as of April 2026, and the community had built more than 100,000 variants. Google DeepMind was formed in April 2023 by merging DeepMind and Google Brain, and is headquartered in London.
Each generation since has brought clear changes. Gemma 2 (June 2024) alternates sliding-window attention, which looks only at nearby text, with global attention, which sees the full text. Gemma 3 (March 2025, 1B to 27B) added image understanding and extended the context to 128K. Gemma 3n (June 2025) was designed for on-device hardware such as phones. Gemma 4, in April 2026, came in four models, E2B, E4B, 26B A4B (MoE) and 31B, with a 12B added in June; it supports up to 256K of context and has a built-in reasoning mode. The license changed with Gemma 4 too: earlier generations used Google's own Gemma Terms of Use, which allow commercial use but come with a prohibited use policy, and from Gemma 4 on the license is Apache 2.0.
- VendorGoogle DeepMind
- Versions4
- Parameters8B – 30.7B
- Latest versionGemma 4 12B
- License
- Apache-2.0
- Official site
- ai.google.dev/gemma
- Hugging Face org
How much VRAM do Gemma models need at 2-bit?
2-bit, also called Q2, covers common GGUF files such as Q2_K and UD-Q2_K_XL. They are the smallest and lose the most quality, which makes them a way to get a model running first when VRAM is tight. Each row in the table below is a Gemma version and each column is a context length; each cell is the total VRAM needed to deploy it at Q2 (weights + KV cache + 1.1GB runtime overhead).
| Version | WeightsGB | 32K contextGB | 128K contextGB | 256K contextGB | 1M contextGB |
|---|---|---|---|---|---|
| Gemma 4 12B12B | 4.7GGUF | 6.6 | 8.3 | 10.4 | — |
| Gemma 4 31B30.7B | 11.8GGUF | 16.4 | 24.5 | 35.2 | — |
| Gemma 4 26B A4B25.2B (3.8B active) | 10.5GGUF | 12.5 | 14.5 | 17.2 | — |
| Gemma 4 E4B8B | 3.8GGUF | 5.4 | 7 | — | — |
How much VRAM do Gemma models need at 4-bit?
4-bit, also called Q4, corresponds to the widely used Q4_K_M files, which balance size and quality and are the default choice for local deployment. Each row in the table below is a Gemma version and each column is a context length; each cell is the total VRAM needed to deploy it at Q4.
| Version | WeightsGB | 32K contextGB | 128K contextGB | 256K contextGB | 1M contextGB |
|---|---|---|---|---|---|
| Gemma 4 12B12B | 7.1GGUF | 9 | 10.7 | 12.8 | — |
| Gemma 4 31B30.7B | 18.3GGUF | 22.9 | 31 | 41.7 | — |
| Gemma 4 26B A4B25.2B (3.8B active) | 16.9GGUF | 18.9 | 20.9 | 23.6 | — |
| Gemma 4 E4B8B | 5GGUF | 6.6 | 8.2 | — | — |
How much VRAM do Gemma models need at 8-bit?
8-bit, also called Q8, corresponds to Q8_0 files, which come closest to the original in quality and are also the largest. Each row in the table below is a Gemma version and each column is a context length; each cell is the total VRAM needed to deploy it at Q8.
| Version | WeightsGB | 32K contextGB | 128K contextGB | 256K contextGB | 1M contextGB |
|---|---|---|---|---|---|
| Gemma 4 12B12B | 12.7GGUF | 14.6 | 16.3 | 18.4 | — |
| Gemma 4 31B30.7B | 32.6GGUF | 37.2 | 45.3 | 56 | — |
| Gemma 4 26B A4B25.2B (3.8B active) | 26.9GGUF | 28.9 | 30.9 | 33.6 | — |
| Gemma 4 E4B8B | 8.2GGUF | 9.8 | 11.4 | — | — |
FAQ
Is Gemma open source and free for commercial use?
For Gemma 4, yes. All four versions, E4B, 12B, 26B A4B and 31B, are licensed under Apache 2.0, so you can use them commercially, fine-tune them and redistribute them, as long as you include the license, keep the copyright and attribution notices, and mark any files you changed. Gemma 3 and earlier are different: they use Google's own Gemma Terms of Use, which come with a prohibited use policy. The terms page now notes that Gemma 4 falls under Apache 2.0 instead. So when you read that “the Gemma license has extra restrictions,” first check which generation it's talking about.
Can you run Gemma with Ollama?
Yes. All four versions have official tags: gemma4:e4b, gemma4:12b, gemma4:26b (the 26B A4B MoE version, which you can also write as gemma4:26b-a4b) and gemma4:31b. Running ollama run gemma4 without a tag pulls E4B. Each size also has tags for specific precisions, such as gemma4:31b-it-q8_0 and gemma4:31b-it-bf16. The cloud tags, gemma4:cloud and gemma4:31b-cloud, run on Ollama's servers and don't use your local GPU.
