How to Run Mistral Large 4 Locally

• Min VRAM at 4-bit (8K context)607 GB
• Parameters1,000B
• Active params52B
• Max context1M

Architecture

MoE

Experts offloaded

16 GB VRAM + 592 GB RAM

License

Mistral Ecosystem Open-Weight License

Launch date

Oct 29, 2026

Vendor

Mistral AI

Hugging Face repo

mistralai/Mistral-Large-4-1T-A52B

GGUF repo

unsloth/Mistral-Large-4-1T-A52B-GGUF

NVFP4 repo (Blackwell)

mistralai/Mistral-Large-4-1T-A52B-NVFP4

Layers

72

Hidden size

6,144

KV heads

64

Head dim

128

Full calculation rules are on the Methodology page.

Only common GPUs and systems are listed, from cheapest to most expensive.

Device	Cards	Total price	Total VRAM	Speed t/s	Total power
Mac Studio M3 Ultra 512GB512 GB per unit	2 units	$45,600	1,024 GB	17.7	960 W
RTX PRO 600096 GB per card	8 cards	$160,800	768 GB	35.2	4,800 W
A100 80GB PCIe80 GB per card	8 cards	$168,000	640 GB	37.5	2,400 W

Mac Studios link up over Thunderbolt 5 and run with MLX or exo; vLLM doesn't support them. Full calculation rules are on the Methodology page.

Detailed calculation rules are on the Methodology page.

As an eBay Partner, VRAMGlass may be compensated if you buy through the eBay links on this page. The price you pay stays the same.

1. 1

   llama.cpp has merged it, but there's no release build yet

   llama.cpp support was merged into mainline on release day (PR #31376), but as of October 29 no release build includes it, so you need to build the latest source before the GGUFs will load; Unsloth also suggests using their own build for now. Ollama only has a cloud version (mistral-large-4:cloud) that doesn't run locally. vLLM support isn't in a release or nightly yet (PR #64534), so the official recipe uses the dedicated vllm/vllm-openai:mistral-large-4 image. On Apple silicon, mlx-lm support is still an open PR as well, though mlx-community has already uploaded a text-only 2-bit build.

2. 2

   A built-in draft head roughly doubles output speed

   The weights include a DSpark draft head: a small 3-layer network guesses the next 3 tokens and the full model checks them in one pass, so the output is exactly the same as without it. Mistral measured up to 1.9x higher throughput, and Unsloth says single-user chat runs about twice as fast. In llama.cpp, load the roughly 2.2 GB draft file separately (-md dspark-Mistral-Large-4-1T-A52B-Q8_0.gguf --spec-type draft-dspark --spec-draft-n-max 3); in vLLM, add --speculative-config '{"method": "dspark", "num_speculative_tokens": 3}'. Our speed estimates assume the draft head is off.

3. 3

   You can set how hard it thinks, but not turn thinking off

   Set thinking depth with reasoning_effort: low, medium or high, with high as the default. Mistral recommends high for complex prompts and agent work, medium for everyday use and low for quick replies, and ggml-org notes that reasoning can't be switched off entirely. The recommended sampling for all three is temperature 1.0 and top_p 0.95. Image input needs a separate projector file of about 3.4 GB (--mmproj mmproj-BF16.gguf), which you can skip if you only use text.

4. 4

   A different license from the rest of the family

   It uses Mistral's new Mistral Ecosystem Open-Weight License, a modified MIT license: you can use it commercially for free, fine-tune it and redistribute it, as long as you keep the copyright notice. The extra clause targets companies that run a Model-as-a-Service business, meaning they give third parties access to a model through an API or hosted endpoint with real control over inputs and parameters. Such a company has to sign a separate commercial license with Mistral before using it for any commercial purpose. Internal use and use through Mistral's own products are exempt. The other Mistral versions on this site are all Apache 2.0, with no such clause.

llama-server -hf unsloth/Mistral-Large-4-1T-A52B-GGUF:UD-Q4_K_XL \
  --jinja -c 8192

docker run --runtime nvidia --gpus all \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -p 8000:8000 \
  --ipc=host \
  vllm/vllm-openai:mistral-large-4 \
  --model mistralai/Mistral-Large-4-1T-A52B-NVFP4 \
  --tensor-parallel-size 8 \
  --kv-cache-dtype fp8_ds_mla \
  --tool-call-parser mistral \
  --reasoning-parser mistral \
  --enable-auto-tool-choice \
  --gpu-memory-utilization 0.85

File names follow that week's model-library snapshot; the definitions are in the methodology. Methodology

Model	Total params	Active params	4-bit size
Mistral 7B v0.3Dense	7.3B	7.3B	4.4 GB
Ministral 3 8BDense	8.4B	8.4B	5.2 GB
Mistral NemoDense	12.3B	12.3B	7.5 GB
Ministral 3 14BDense	13.5B	13.5B	8.2 GB
Mistral Small 3.2 24BDense	24B	24B	14.3 GB

About Mistral