How to Deploy Kimi K2.7 Code Locally

• Min VRAM at 4-bit586 GB · 8K
• Parameters1,000B
• Active params32B
• Max context256K

Architecture

MoE

Experts offloaded

10 GB VRAM + 578 GB RAM

License

Modified MIT

Launch date

Jun 12, 2026

Vendor

Moonshot AI

Hugging Face repo

moonshotai/Kimi-K2.7-Code

GGUF repo

unsloth/Kimi-K2.7-Code-GGUF

NVFP4 repo (Blackwell)

nvidia/Kimi-K2.7-Code-NVFP4

MLX repo (Apple)

mlx-community/Kimi-K2.7-Code-4bit

Layers

61

Hidden size

7,168

KV heads

64

Head dim

192

Full calculation rules are on the Methodology page.

Only common GPUs and systems are listed, one set for models under 50B and another for larger ones, sorted by price from low to high.

Device	Cards	Total price	Total VRAM	Speed t/s	Total power
Mac Studio M3 Ultra 512GB512 GB per unit	2 units	$45,600	1,024 GB	27.6	960 W
A100 80GB PCIe80 GB per card	8 cards	$151,992	640 GB	55.4	2,400 W
RTX PRO 600096 GB per card	8 cards	$180,000	768 GB	52.3	4,800 W

Mac Studios link up over Thunderbolt 5 and run with MLX or exo; vLLM doesn't support them. Full calculation rules are on the Methodology page.

Detailed calculation rules are on the Methodology page.

As an eBay Partner, VRAMGlass may be compensated if you buy through the eBay links on this page. The price you pay stays the same.

1. 1

   Thinking mode only

   It doesn't support instant mode, and preserve_thinking is forced on and can't be turned off: in multi-turn conversations, the client has to send back the thinking content returned in each turn (in vLLM, this field may be called reasoning rather than reasoning_content). Moonshot recommends temperature 1.0 and top_p 0.95.

2. 2

   Quantization

   Unsloth offers more quants than for K2.6, with several levels between 1-bit and 4-bit; 3-bit is about 464 GB. The recommended pick is 2-bit (UD-Q2_K_XL); the 8-bit UD-Q8_K_XL keeps the 4-bit experts as-is and is the lossless version. For context, Unsloth recommends a default of 98,304, with a maximum of 262,144.

3. 3

   vLLM flags

   The vLLM recipe requires 0.19.1 or later. For tool calling and thinking parsing, add --tool-call-parser kimi_k2 and --reasoning-parser kimi_k2 respectively. For the vision part, --mm-encoder-tp-mode data runs a separate copy on each card, avoiding communication between GPUs.

4. 4

   Hooking it up to coding agents

   Moonshot says it works best with its own Kimi Code CLI. When connecting it to other agent frameworks, first make sure the framework passes thinking content back unchanged; otherwise you lose the benefit of “preserved thinking.”

llama-server -hf unsloth/Kimi-K2.7-Code-GGUF:UD-Q4_K_XL -c 8192

llama-server -hf unsloth/Kimi-K2.7-Code-GGUF:UD-Q4_K_XL -c 8192 \
  -ngl 99 --cpu-moe

vllm serve moonshotai/Kimi-K2.7-Code \
  --trust-remote-code \
  --tensor-parallel-size 8 \
  --tool-call-parser kimi_k2 \
  --enable-auto-tool-choice \
  --reasoning-parser kimi_k2 \
  --speculative-config '{"model":"lightseekorg/kimi-k2.7-coder-eagle3.1-mla","method":"eagle3","num_speculative_tokens":3}' \
  --mm-encoder-tp-mode data

mlx_lm.server --model mlx-community/Kimi-K2.7-Code-4bit

File names follow that week's model-library snapshot; the definitions are in the methodology. Methodology

Model	Total params	Active params	4-bit size
Kimi Linear 48B A3BMoE	49.1B	3B	30.1 GB
Kimi K2MoE	1,000B	32B	621.3 GB
Kimi K2.5MoE	1,000B	32B	621.2 GB
Kimi K2.6MoE	1,000B	32B	583.7 GB
Kimi K3MoE	2,800B	104B	1,508.7 GB

About Kimi