How to Deploy Kimi K2.5 Locally

• Min VRAM at 4-bit623 GB · 8K
• Parameters1,000B
• Active params32B
• Max context256K

Architecture

MoE

Experts offloaded

10 GB VRAM + 615 GB RAM

License

Modified MIT

Launch date

Jan 27, 2026

Vendor

Moonshot AI

Hugging Face repo

moonshotai/Kimi-K2.5

GGUF repo

unsloth/Kimi-K2.5-GGUF

NVFP4 repo (Blackwell)

nvidia/Kimi-K2.5-NVFP4

MLX repo (Apple)

mlx-community/Kimi-K2.5

Layers

61

Hidden size

7,168

KV heads

64

Head dim

192

Full calculation rules are on the Methodology page.

Only common GPUs and systems are listed, one set for models under 50B and another for larger ones, sorted by price from low to high.

Device	Cards	Total price	Total VRAM	Speed t/s	Total power
Mac Studio M3 Ultra 512GB512 GB per unit	2 units	$45,800	1,024 GB	26.2	960 W
RTX PRO 600096 GB per card	8 cards	$180,000	768 GB	50	4,800 W

Mac Studios link up over Thunderbolt 5 and run with MLX or exo; vLLM doesn't support them. Full calculation rules are on the Methodology page.

Detailed calculation rules are on the Methodology page.

As an eBay Partner, VRAMGlass may be compensated if you buy through the eBay links on this page. The price you pay stays the same.

1. 1

   Skip the 8-bit GGUF

   The officially released expert weights are already 4-bit (INT4), and the model was adapted to 4-bit during training (the same approach as K2 Thinking). Unsloth says 4-bit or 5-bit dynamic quants are enough for original-level precision, so for K2.5 run locally, 4-bit is as high as it makes sense to go; 8-bit just stores the same data in a larger format. Starting with K2.6, Unsloth switched to an 8-bit that keeps the INT4 as-is, only about 10 GB larger than 4-bit.

2. 2

   Thinking and instant modes

   Use temperature 1.0 for thinking mode and 0.6 for instant mode, with top_p 0.95 for both. When deploying with vLLM or SGLang, pass thinking: false under chat_template_kwargs in the request to switch to instant mode.

3. 3

   Local GGUFs are basically text-only

   When llama.cpp merged K2.5 support in February 2026, it included preliminary vision code (PR #19170), but Unsloth's K2.5 GGUF repository doesn't include the projector file (mmproj) needed for images; only the GGUFs from K2.6 onward do. Video input, according to Moonshot, is currently available only through its own API.

4. 4

   The template changed on January 29

   Two days after release, Moonshot removed a default system prompt that could cause misunderstandings and fixed a typo in the chat template, changing <|media_start|> to <|media_begin|>. If you're using weights or GGUFs from the earliest conversions, make sure the template is the new one.

llama-server -hf unsloth/Kimi-K2.5-GGUF:Q4_K_M -c 8192

llama-server -hf unsloth/Kimi-K2.5-GGUF:Q4_K_M -c 8192 -ngl 99 --cpu-moe

vllm serve moonshotai/Kimi-K2.5 \
  --trust-remote-code \
  --tensor-parallel-size 8 \
  --tool-call-parser kimi_k2 \
  --enable-auto-tool-choice \
  --reasoning-parser kimi_k2 \
  --speculative-config '{"model":"lightseekorg/kimi-k2.5-eagle3-mla","method":"eagle3","num_speculative_tokens":3}' \
  --mm-encoder-tp-mode data

mlx_lm.server --model mlx-community/Kimi-K2.5

File names follow that week's model-library snapshot; the definitions are in the methodology. Methodology

Model	Total params	Active params	4-bit size
Kimi Linear 48B A3BMoE	49.1B	3B	30.1 GB
Kimi K2MoE	1,000B	32B	621.3 GB
Kimi K2.6MoE	1,000B	32B	583.7 GB
Kimi K2.7 CodeMoE	1,000B	32B	583.7 GB
Kimi K3MoE	2,800B	104B	1,508.7 GB

About Kimi