How to Deploy Gemma 4 E4B Locally

• Min VRAM at 4-bit7 GB · 8K
• Parameters8B
• Max context128K

Architecture

Dense

License

Apache-2.0

Launch date

Apr 2, 2026

Vendor

Google DeepMind

Hugging Face repo

google/gemma-4-E4B-it

GGUF repo

unsloth/gemma-4-E4B-it-GGUF

NVFP4 repo (Blackwell)

unsloth/gemma-4-E4B-it-NVFP4

MLX repo (Apple)

mlx-community/gemma-4-e4b-it-4bit

Ollama tag

gemma4:e4b

Layers

42

Hidden size

2,560

KV heads

2

Head dim

256

Full calculation rules are on the Methodology page.

Device	Price	Speed t/s	Max context
Tesla V100 16GB16 GB	$37024 listings	114.2	128K
GeForce RTX 5060 Ti 16GB16 GB	$86958 listings	61.8	128K
Tesla V100 32GB32 GB	$89518 listings	114.2	128K
Radeon RX 7900 XTX24 GB	$1,25029 listings	120.5	128K
GeForce RTX 309024 GB	$1,799107 listings	118	128K
GeForce RTX 409024 GB	$3,75868 listings	125.5	128K
Ryzen AI Max+ 395 128GBWhole system · 128 GB	$4,46420 listings	36.7	128K
RTX A600048 GB	$4,89916 listings	99.8	128K
Mac Studio M5 Ultra 96GBWhole system · 96 GB	$7,27412 listings	147.3	128K
MacBook Pro M5 Max 128GBWhole system · 128 GB	$7,54920 listings	82.1	128K
DGX Spark 128GBWhole system · 128 GB	$8,24022 listings	39	128K
GeForce RTX 509032 GB	$8,395123 listings	196.3	128K
RTX PRO 600096 GB	$22,50011 listings	196.3	128K

Only common GPUs and systems are listed, one set for models under 50B and another for larger ones, sorted by price from low to high.

1. 1

   The “E” in E4B

   Every layer of E4B has its own embedding table. These tables are large, but during generation they are only looked up, with each token using just a few rows, so only about 4.5B parameters actually do computation; that's why Google calls it an “effective 4B.” This site estimates both VRAM and speed from 8B, so real generation may be somewhat faster than the speeds in the table.

2. 2

   Thinking is off by default

   Thinking is off by default and only turns on when you add <|think|> at the start of the system prompt (in Transformers that maps to enable_thinking=True, and llama.cpp's chat template handles it for you). With thinking off, it answers directly, unlike the 12B and larger models, which still output an empty thinking block. Google's sampling parameters apply to every use case: temperature 1.0, top_p 0.95, top_k 64.

3. 3

   How to feed it images and audio

   Put images before the text and audio after it. Audio is limited to 30 seconds, and video is read at 1 frame per second for up to 60 seconds. You can choose how many tokens represent an image (several levels from 70 to 1120): use the higher levels for OCR and small text, and the lower ones for classification and video. llama.cpp has supported Gemma 4 audio input since April 12, 2026.

4. 4

   Where its limits are

   Its knowledge and common sense are solid: in Google's model card, E4B scores 69.4 on MMLU Pro, slightly above the previous-generation Gemma 3 27B's 67.6 without thinking. But it clearly trails the next size up on hard reasoning problems, scoring only 58.6 on GPQA Diamond versus 78.8 for the 12B. It's better suited to speech transcription, image Q&A and a lightweight assistant on a laptop.

ollama run gemma4:e4b

llama-server -hf unsloth/gemma-4-E4B-it-GGUF:Q4_K_M -c 8192

vllm serve google/gemma-4-E4B-it

vllm serve unsloth/gemma-4-E4B-it-NVFP4

mlx_vlm.server --model mlx-community/gemma-4-e4b-it-4bit

File names follow that week's model-library snapshot; the definitions are in the methodology. Methodology

Model	Total params	Active params	4-bit size
Gemma 4 12BDense	12B	12B	7.1 GB
Gemma 4 26B A4BMoE	25.2B	3.8B	16.9 GB
Gemma 4 31BDense	30.7B	30.7B	18.3 GB

About Gemma