How to Deploy Kimi Linear 48B A3B Locally

• Min VRAM at 4-bit32 GB · 8K
• Parameters49.1B
• Active params3B
• Max context1M

Architecture

MoE

Experts offloaded

4 GB VRAM + 30 GB RAM

License

MIT

Launch date

Oct 30, 2025

Vendor

Moonshot AI

Hugging Face repo

moonshotai/Kimi-Linear-48B-A3B-Instruct

GGUF repo

bartowski/moonshotai_Kimi-Linear-48B-A3B-Instruct-GGUF

MLX repo (Apple)

mlx-community/Kimi-Linear-48B-A3B-Instruct-4bit

Layers

27

Hidden size

2,304

KV heads

32

Head dim

72

Full calculation rules are on the Methodology page.

Device	Price	Speed t/s	Max context
Tesla V100 32GB32 GB	$89518 listings	134.9	385K
Ryzen AI Max+ 395 128GBWhole system · 128 GB	$4,46420 listings	70.3	1M
RTX A600048 GB	$4,89916 listings	126.9	1M
Mac Studio M5 Ultra 96GBWhole system · 96 GB	$7,27412 listings	149.6	1M
MacBook Pro M5 Max 128GBWhole system · 128 GB	$7,54920 listings	115.3	1M
DGX Spark 128GBWhole system · 128 GB	$8,24022 listings	73.3	1M
GeForce RTX 509032 GB	$8,395123 listings	165	385K
RTX PRO 600096 GB	$22,50011 listings	165	1M

Only common GPUs and systems are listed, one set for models under 50B and another for larger ones, sorted by price from low to high.

Detailed calculation rules are on the Methodology page.

As an eBay Partner, VRAMGlass may be compensated if you buy through the eBay links on this page. The price you pay stays the same.

1. 1

   What linear attention means

   With standard attention, every extra token adds another entry to the cache; linear attention (KDA here) stores only a fixed-size state that doesn't grow no matter how long the context gets. Kimi Linear pairs every 3 linear-attention layers with 1 standard-attention layer, and Moonshot says this cuts the context cache by up to 75% and makes generation up to about 6x faster at 1M of context. The same design was later used in Kimi K3 and GLM 5.3 Flash.

2. 2

   Let context drive your quantization choice

   Long context is the whole point of this model, so when choosing a quant, work out how much context you need first. On a 24GB card, 2-bit leaves room for over 800K of context, while 3-bit leaves only about 96K. On a 32GB card, use 4-bit; a 48GB card can step up to 5-bit (about 35 GB), which comes to about 45 GB with the full 1M; 8-bit is about 52 GB and needs a 96GB card or a unified-memory machine.

3. 3

   Use a llama.cpp build from mid-February or later

   Mainline llama.cpp merged Kimi Linear support on February 6, 2026 (PR #18755). On February 13 it fixed a bug where state got scrambled when llama-server handled multiple requests at once (PR #19531), so use a build from after that; on March 5 another optimization landed (PR #19827), which the author says makes prompt processing about 15% faster. Ollama's official library doesn't have this model.

4. 4

   vLLM version

   The vLLM recipe recommends pinning 0.11.2, since 0.12.0 has a known issue that makes Kimi Linear fail at startup. The official example uses four GPUs with the full 1M context; for local single-GPU use, llama.cpp is simpler.

llama-server \
  -hf bartowski/moonshotai_Kimi-Linear-48B-A3B-Instruct-GGUF:Q4_K_M \
  -c 8192

llama-server \
  -hf bartowski/moonshotai_Kimi-Linear-48B-A3B-Instruct-GGUF:Q4_K_M \
  -c 8192 -ngl 99 --cpu-moe

vllm serve moonshotai/Kimi-Linear-48B-A3B-Instruct \
  --trust-remote-code \
  --tensor-parallel-size 1

mlx_lm.server --model mlx-community/Kimi-Linear-48B-A3B-Instruct-4bit

File names follow that week's model-library snapshot; the definitions are in the methodology. Methodology

Model	Total params	Active params	4-bit size
Kimi K2MoE	1,000B	32B	621.3 GB
Kimi K2.5MoE	1,000B	32B	621.2 GB
Kimi K2.6MoE	1,000B	32B	583.7 GB
Kimi K2.7 CodeMoE	1,000B	32B	583.7 GB
Kimi K3MoE	2,800B	104B	1,508.7 GB

About Kimi