How to Deploy Ministral 3 8B Locally
Ministral 3 8B is the small 8B model in the Mistral 3 series, and it understands images. The Instruct version answers directly without thinking first and is Apache 2.0 licensed. At 4-bit with 8K it takes about 7.4 GB, so an 8GB card can run it; but its context is heavy on VRAM, and the full 256K needs about 43 GB.
Ministral 3 8B details
- Min VRAM at 4-bit8 GB · 8K
- Parameters8.4B
- Max context256K
- Architecture
- Dense
- License
- Apache-2.0
- Launch date
- Dec 2, 2025
- Vendor
- Mistral AI
- Hugging Face repo
- mistralai/Ministral-3-8B-Instruct-2512
- MLX repo (Apple)
- mlx-community/Ministral-3-8B-Instruct-2512-4bit
- Ollama tag
- ministral-3:8b
- Revision
- 2512
- Layers
- 34
- Hidden size
- 4,096
- KV heads
- 8
- Head dim
- 128
How much VRAM does Ministral 3 8B need for local deployment?
| Quantization | WeightsGB | 8K contextGB | 32K contextGB | 128K contextGB | 256K context (max)GB |
|---|---|---|---|---|---|
| 1-bit | 2.4GGUF | 4.6 | 8 | 21.7 | 40 |
| 2-bit | 3.6GGUF | 5.8 | 9.2 | 22.9 | 41.2 |
| 3-bit | 4.4GGUF | 6.6 | 10 | 23.7 | 42 |
| 4-bit | 5.2GGUF | 7.4 | 10.8 | 24.5 | 42.8 |
| 5-bit | 6.1GGUF | 8.3 | 11.7 | 25.4 | 43.7 |
| 8-bit | 9GGUF | 11.2 | 14.6 | 28.3 | 46.6 |
| BF16/FP16 | 17GGUF | 19.2 | 22.6 | 36.3 | 54.6 |
Full calculation rules are on the Methodology page.
Can a single GPU run Ministral 3 8B?
A 12GB card is enough: 4-bit with 32K of context is about 10.8 GB, and Mistral says its official weights fit in 12GB as published. If you want image input, the vision part ships as a separate file, so leave a bit under 1 GB extra for it. Where cards really differ is context. The model supports up to 256K, but long context is memory-hungry, and every extra stretch of it claims more VRAM. A 24GB RTX 3090 at 8-bit reaches about 110K, enough length and speed for most long documents; to use all 256K you need a 48GB RTX A6000 or a 128GB unified-memory machine.
| Device | Price | Speed t/s | Max context |
|---|---|---|---|
 Tesla V100 16GB16 GB | $37024 listings | 134.6 | 87K |
| GeForce RTX 5060 Ti 16GB16 GB | $86958 listings | 74 | 87K |
| Tesla V100 32GB32 GB | $89518 listings | 134.6 | 208K |
| Radeon RX 7900 XTX24 GB | $1,25029 listings | 141.8 | 147K |
| GeForce RTX 309024 GB | $1,799107 listings | 139 | 147K |
| GeForce RTX 409024 GB | $3,75868 listings | 147.5 | 147K |
| Ryzen AI Max+ 395 128GBWhole system · 128 GB | $4,46420 listings | 44.3 | 256K |
| RTX A600048 GB | $4,89916 listings | 118.2 | 256K |
| Mac Studio M5 Ultra 96GBWhole system · 96 GB | $7,27412 listings | 172 | 256K |
| MacBook Pro M5 Max 128GBWhole system · 128 GB | $7,54920 listings | 97.7 | 256K |
| DGX Spark 128GBWhole system · 128 GB | $8,24022 listings | 47 | 256K |
| GeForce RTX 509032 GB | $8,395123 listings | 225.9 | 208K |
| RTX PRO 600096 GB | $22,50011 listings | 225.9 | 256K |
| Device | Price | Speed t/s | Max context |
|---|---|---|---|
| Tesla V100 16GB16 GB | $37024 listings | 102.6 | 76K |
| GeForce RTX 5060 Ti 16GB16 GB | $86958 listings | 55 | 76K |
| Tesla V100 32GB32 GB | $89518 listings | 102.6 | 196K |
| Radeon RX 7900 XTX24 GB | $1,25029 listings | 108.4 | 136K |
| GeForce RTX 309024 GB | $1,799107 listings | 106.1 | 136K |
| GeForce RTX 409024 GB | $3,75868 listings | 113 | 136K |
| Ryzen AI Max+ 395 128GBWhole system · 128 GB | $4,46420 listings | 32.5 | 256K |
| RTX A600048 GB | $4,89916 listings | 89.4 | 256K |
| Mac Studio M5 Ultra 96GBWhole system · 96 GB | $7,27412 listings | 133.1 | 256K |
| MacBook Pro M5 Max 128GBWhole system · 128 GB | $7,54920 listings | 73.3 | 256K |
| DGX Spark 128GBWhole system · 128 GB | $8,24022 listings | 34.6 | 256K |
| GeForce RTX 509032 GB | $8,395123 listings | 178.8 | 196K |
| RTX PRO 600096 GB | $22,50011 listings | 178.8 | 256K |
| Device | Price | Speed t/s | Max context |
|---|---|---|---|
| Tesla V100 16GB16 GB | $37024 listings | 65.6 | 49K |
| GeForce RTX 5060 Ti 16GB16 GB | $86958 listings | 34.2 | 49K |
| Tesla V100 32GB32 GB | $89518 listings | 65.6 | 170K |
| Radeon RX 7900 XTX24 GB | $1,25029 listings | 69.5 | 110K |
| GeForce RTX 309024 GB | $1,799107 listings | 67.9 | 110K |
| GeForce RTX 409024 GB | $3,75868 listings | 72.6 | 110K |
| Ryzen AI Max+ 395 128GBWhole system · 128 GB | $4,46420 listings | 20 | 256K |
| RTX A600048 GB | $4,89916 listings | 56.7 | 256K |
| Mac Studio M5 Ultra 96GBWhole system · 96 GB | $7,27412 listings | 86.6 | 256K |
| MacBook Pro M5 Max 128GBWhole system · 128 GB | $7,54920 listings | 46.1 | 256K |
| DGX Spark 128GBWhole system · 128 GB | $8,24022 listings | 21.2 | 256K |
| GeForce RTX 509032 GB | $8,395123 listings | 119.6 | 170K |
| RTX PRO 600096 GB | $22,50011 listings | 119.6 | 256K |
Only common GPUs and systems are listed, one set for models under 50B and another for larger ones, sorted by price from low to high.
What to know before running Ministral 3 8B locally
The official weights are FP8
The Instruct version on Hugging Face is released directly in FP8, and Mistral says it fits in 12GB of VRAM; vLLM needs 0.12.0 or later. For llama.cpp, download from Mistral's official GGUF repository, which only has four quants: Q4_K_M, Q5_K_M, Q8_0 and BF16. For lower quants, look to third parties such as Unsloth.
Images need one more file
It consists of an 8.4B language model plus a 0.4B vision component, which lives in a separate mmproj file (about 0.8 GB for BF16); llama.cpp has to load it too for images to work. The VRAM figures in the table only count the language model, so add this part for image use. Mistral recommends images with an aspect ratio close to 1:1; crop very long or narrow images first.
Keep temperature below 0.1
Mistral recommends
temperaturebelow 0.1 for everyday and production use, going higher only for creative work. That's much lower than most front ends' defaults, so remember to change it in Ollama or Open WebUI. The Ollama tag is ministral-3:8b and requires version 0.13.1 or later.Where the 256K comes from
Its native training length is only 16K; the config already applies YaRN to stretch it 16x to 256K, so you don't need to add any parameters yourself. But vLLM allocates VRAM for 256K by default, so on a single card, remember to lower it with
--max-model-len.
Launch commands
Ollama
- Windows
- macOS
- Linux
ollama run ministral-3:8bllama.cpp
- NVIDIA
- AMD
- Mac
- CPU
llama-server -hf mistralai/Ministral-3-8B-Instruct-2512-GGUF:Q4_K_M \ -c 8192
vLLM
- Linux
- NVIDIA
- AMD
vllm serve mistralai/Ministral-3-8B-Instruct-2512MLX
- Apple Silicon
Vision models run on mlx-vlm: on a Mac, pip install mlx-vlm first; this command downloads the MLX weights from Hugging Face and starts a server.
mlx_vlm.server --model mlx-community/Ministral-3-8B-Instruct-2512-4bitFile names follow that week's model-library snapshot; the definitions are in the methodology. Methodology
Other models in the Mistral family
| Model | Total params | Active params | 4-bit size |
|---|---|---|---|
| Mistral 7B v0.3Dense | 7.3B | 7.3B | 4.4 GB |
| Mistral NemoDense | 12.3B | 12.3B | 7.5 GB |
| Ministral 3 14BDense | 13.5B | 13.5B | 8.2 GB |
| Mistral Small 3.2 24BDense | 24B | 24B | 14.3 GB |