How to Deploy Hy3 Locally

• Min VRAM at 4-bit186 GB · 8K
• Parameters295B
• Active params21B
• Max context256K

Architecture

MoE

Experts offloaded

11 GB VRAM + 177 GB RAM

License

Apache-2.0

Launch date

Jul 6, 2026

Vendor

Tencent

Hugging Face repo

tencent/Hy3

GGUF repo

bartowski/Hy3-GGUF

NVFP4 repo (Blackwell)

RedHatAI/Hy3-NVFP4-FP8

Layers

80

Hidden size

4,096

KV heads

8

Head dim

128

Full calculation rules are on the Methodology page.

Device	Price	Speed t/s	Max context
Mac Studio M3 Ultra 512GBWhole system · 512 GB	$21,95015 listings	36.4	256K

Only common GPUs and systems are listed, one set for models under 50B and another for larger ones, sorted by price from low to high.

Device	Cards	Total price	Total VRAM	Speed t/s	Total power
Tesla V100 32GB32 GB per card	8 cards	$7,160	256 GB	32.1	2,000 W
Arc Pro B7032 GB per card	8 cards	$13,168	256 GB	22.8	1,840 W
DGX Spark 128GB128 GB per unit	2 units	$16,686	256 GB	10.9	480 W
Instinct MI21064 GB per card	4 cards	$20,164	256 GB	52.1	1,200 W
RTX A600048 GB per card	8 cards	$39,192	384 GB	28	2,400 W
RTX PRO 5000 72GB72 GB per card	4 cards	$42,888	288 GB	44.7	1,200 W
GeForce RTX 509032 GB per card	8 cards	$64,000	256 GB	55.7	4,600 W
A100 80GB PCIe80 GB per card	4 cards	$81,596	320 GB	58.9	1,200 W
RTX PRO 600096 GB per card	4 cards	$89,400	384 GB	55.7	2,400 W
GeForce RTX 4090 48GB48 GB per card	8 cards	No price	384 GB	35.4	3,600 W

Full calculation rules are on the Methodology page.

Detailed calculation rules are on the Methodology page.

As an eBay Partner, VRAMGlass may be compensated if you buy through the eBay links on this page. The price you pay stays the same.

1. 1

   GGUFs and server versions

   bartowski has made a full set of GGUFs from 1-bit to 8-bit; the 4-bit weights are about 182 GB and 2-bit about 107 GB. For servers, besides the official BF16 and FP8 versions, the vLLM recipe also lists an NVFP4 version from Red Hat that fits on four 80GB H100s.

2. 2

   Which framework to use

   bartowski's GGUFs were quantized with llama.cpp release b9993, so mainline llama.cpp runs them; Ollama's official library doesn't have it. For servers, Tencent recommends vLLM (the recipe requires 0.28.0 or later) or SGLang; in vLLM, both the tool-call parser and the reasoning parser are called hy_v3.

3. 3

   Thinking toggle and sampling parameters

   By default it doesn't think (reasoning_effort is no_think); for complex tasks like math and code, switch it to high, and there's also a low level in between. Tencent recommends temperature 0.9 and top_p 1.0.

4. 4

   Long context takes a lot of VRAM

   It uses standard attention, so the context cache grows linearly with length: at 4-bit, 256K takes about 83 GB more than 8K (about 269 vs. 186 GB). Factor that in when sizing hardware, and if you don't need long context, don't max it out.

5. 5

   The license is now Apache 2.0

   Tencent's earlier Hunyuan models (such as Hunyuan-A13B) used its own community license, which explicitly did not apply in the EU, the UK and South Korea; Hy3 switched to Apache 2.0, with no such regional restriction.

llama-server -hf bartowski/Hy3-GGUF:Q4_K_M --jinja -c 8192

# Switch to trtllm backend to work-around mnnvl workspace size issue.
export VLLM_FLASHINFER_ALLREDUCE_BACKEND=trtllm
vllm serve tencent/Hy3 \
  --tensor-parallel-size 8 \
  --speculative-config.method mtp \
  --speculative-config.num_speculative_tokens 2 \
  --tool-call-parser hy_v3 \
  --reasoning-parser hy_v3 \
  --enable-auto-tool-choice \
  --port 8000 \
  --served-model-name hy3

File names follow that week's model-library snapshot; the definitions are in the methodology. Methodology

Model	Total params	Active params	4-bit size
Hy4 previewMoE	770B	49B	467.3 GB

About Hunyuan