How to Deploy Hy4 preview Locally

• Min VRAM at 4-bit470 GB · 8K
• Parameters770B
• Active params49B
• Max context1M

Architecture

MoE

Experts offloaded

19 GB VRAM + 452 GB RAM

License

Apache-2.0

Launch date

Aug 28, 2026

Vendor

Tencent

Hugging Face repo

tencent/Hy4-preview

GGUF repo

AngelSlim/Hy4-preview-GGUF

MLX repo (Apple)

mlx-community/Hy4-preview-4bit

Layers

78

Hidden size

6,144

KV heads

8

Head dim

64

Full calculation rules are on the Methodology page.

Only common GPUs and systems are listed, one set for models under 50B and another for larger ones, sorted by price from low to high.

Device	Cards	Total price	Total VRAM	Speed t/s	Total power
DGX Spark 128GB128 GB per unit	4 units	$33,372	512 GB	6.5	960 W
Instinct MI21064 GB per card	8 cards	$40,328	512 GB	33.9	2,400 W
RTX PRO 5000 72GB72 GB per card	8 cards	$85,776	576 GB	28.6	2,400 W
A100 80GB PCIe80 GB per card	8 cards	$163,192	640 GB	38.9	2,400 W
RTX PRO 600096 GB per card	8 cards	$178,800	768 GB	36.5	4,800 W

Mac Studios link up over Thunderbolt 5 and run with MLX or exo; vLLM doesn't support them. Full calculation rules are on the Methodology page.

Detailed calculation rules are on the Methodology page.

As an eBay Partner, VRAMGlass may be compensated if you buy through the eBay links on this page. The price you pay stays the same.

1. 1

   llama.cpp needs a patch

   Mainline llama.cpp merged Hy4 preview support on September 4 (PR #28127), but AngelSlim's GGUF notes state that these files can't run on stock llama.cpp: you have to patch a specific version as described and then build it, and the STQ1_0 version needs a second patch on top. For chat you must add --jinja. Keep the GGUFs on a local drive; on network storage, loading time stretches from a minute to several hours.

2. 2

   Only three GGUFs

   The standard Q4_K_M, at about 467 GB, is AngelSlim's recommended default. If you're short on memory, use UD-IQ1_M (about 235 GB) or the slightly smaller STQ1_0, which cut the size in half; the trade-off is that the main expert weights drop to only about 1.3 to 2 bits, so quality suffers.

3. 3

   Deep thinking is on by default

   Unlike Hy3, Hy4 preview defaults to reasoning_effort high; for direct answers, pass no_think. Recommended sampling is the same as Hy3: temperature 0.9 and top_p 1.0. Tencent also acknowledges in the model card that this version tends to think too long and keep double-checking itself, which uses up a lot more output tokens.

4. 4

   Going to 1M adds only a little over 90 GB

   Its attention design compresses context heavily: at the full 1M, 4-bit totals about 563 GB, only a little over 90 GB more than at 32K. In the official vLLM recipe, eight MI355X cards are validated at 256K by default, and the FP8 version can also go to 1M.

./build-cuda/bin/llama-cli -m Hy4-preview-Q4_K_M.gguf -ngl 99 -c 8192 \
  --jinja -st -f prompt.txt

docker run --gpus all \
  -p 8000:8000 \
  --ipc=host \
  -e VLLM_ENABLE_HPC_OPS=1 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:hy4-preview tencent/Hy4-preview-FP8 \
    --tensor-parallel-size 8 \
    --speculative-config '{"num_speculative_tokens":3,"method":"mtp"}' \
    --attention-backend FLASHMLA_SPARSE \
    --tool-call-parser hy_v4 \
    --reasoning-parser hy_v4 \
    --enable-auto-tool-choice \
    --port 8000 \
    --served-model-name hy4-preview

mlx_vlm.server --model mlx-community/Hy4-preview-4bit

File names follow that week's model-library snapshot; the definitions are in the methodology. Methodology

Model	Total params	Active params	4-bit size
Hy3MoE	295B	21B	182.2 GB

About Hunyuan