Run Hy4 Flash Locally: VRAM & GPU Requirements
Hy4 Flash is Tencent Hunyuan's 220B multimodal MoE from November 2026, using about 12B per token and 168 GB at 4-bit. Because about 63 GB of that is a lookup table that can stay on disk and be read on demand, Tencent's benchmarks show a 128GB Mac or DGX Spark can run 4-bit. Support in llama.cpp is not merged into mainline yet, so you have to compile the official branch yourself, and Ollama does not have it.
Hy4 Flash details
- Min VRAM at 4-bit (8K context)170 GB
- Parameters220B
- Active params12B
- Max context1M
- Architecture
- MoE
- Experts offloaded
- 8 GB VRAM + 164 GB RAM
- License
- Tencent Hy License
- Launch date
- Nov 18, 2026
- Vendor
- Tencent
- Hugging Face repo
- tencent/Hy4-flash
- GGUF repo
- AngelSlim/Hy4-flash-GGUF
- Layers
- 42
- Hidden size
- 3,072
- KV heads
- 32
- Head dim
- 64
How much VRAM does Hy4 Flash need?
Hy4 Flash's 4-bit file is about 168 GB, and loading the whole model into VRAM with 32K context takes about 170 GB; 2-bit is about 135 GB and 3-bit is about 155 GB. All of those numbers include an Engram lookup table of about 63 GB. Generation looks up only a few dozen rows per token, so that table can sit in system RAM or stay on disk, leaving about 105 GB of 4-bit weights that actually need to stay resident. Context is very light, adding only 14 GB when scaling from 32K all the way to 1M.
| Quantization | WeightsGB | 32K contextGB | 128K contextGB | 256K contextGB | 1M context (max)GB |
|---|---|---|---|---|---|
| 1-bit | 102.8est. | 104.4 | 105.7 | 107.5 | 118.4 |
| 2-bit | 135GGUF | 136.5 | 137.9 | 139.7 | 150.6 |
| 3-bit | 155.4GGUF | 156.9 | 158.3 | 160.1 | 171 |
| 4-bit | 168.3GGUF | 169.8 | 171.2 | 173 | 183.9 |
| 5-bit | 176.8est. | 178.4 | 179.7 | 181.5 | 192.4 |
| 8-bit | 237.7est. | 239.2 | 240.6 | 242.4 | 253.2 |
| BF16/FP16 | 392.2est. | 393.8 | 395.1 | 396.9 | 407.8 |
Full calculation rules are on the Methodology page.
Can a single GPU run Hy4 Flash?
It depends on the quantization and on where you keep the lookup table. If the entire file stays in memory, a 128GB machine fits only 2-bit, while 4-bit needs a 512GB Mac Studio M3 Ultra. Leaving the ~63 GB lookup table on disk to be read on demand lets 4-bit run in 128GB: Tencent's AngelSlim team measured about 44 to 48 tokens/s on a 128GB M5 Max, and about 26 to 33 tokens/s on a DGX Spark using the dedicated SPARK-4Bit build. The largest discrete GPU has 96GB, which cannot hold the ~105 GB resident portion of 4-bit without multiple cards or MoE offloading to system RAM.
| Device | Price | Speed t/s | Max context |
|---|---|---|---|
| Ryzen AI Max+ 395 128GBWhole system · 128 GB | $4,19933 listings | 30.3 | 95K |
| MacBook Pro M5 Max 128GBWhole system · 128 GB | $7,34934 listings | 60.6 | 95K |
| $7,99931 listings | 32 | 95K | |
| Mac Studio M3 Ultra 512GBWhole system · 512 GB | $25,0009 listings | 73.8 | 1M |
| Device | Price | Speed t/s | Max context |
|---|---|---|---|
| Mac Studio M3 Ultra 512GBWhole system · 512 GB | $25,0009 listings | 56.8 | 1M |
| Device | Price | Speed t/s | Max context |
|---|---|---|---|
| Mac Studio M3 Ultra 512GBWhole system · 512 GB | $25,0009 listings | 38.3 | 1M |
Only common GPUs and systems are listed, from cheapest to most expensive.
How many GPUs do you need to run Hy4 Flash?
Running Hy4 Flash at 4-bit across multiple GPUs takes at least two 96GB RTX PRO 6000 cards, four 48GB cards, or eight 24GB consumer cards. That assumes splitting the entire file evenly across the GPUs; moving the lookup table to system RAM saves another 63 GB or so of VRAM. Multi-GPU inference with quantized files uses llama.cpp. The official vLLM recipe runs the full-precision and FP8 weights only on datacenter GPUs like the H100, H200 and B200, and puts the lookup table in system RAM by default.
| Device | Cards | Total price | Total VRAM | Speed t/s | Total power |
|---|---|---|---|---|---|
| 8 cards | $9,992 | 256 GB | 77 | 2,000 W | |
| Radeon RX 7900 XTX24 GB per card | 8 cards | $11,200 | 192 GB | 80.2 | 2,840 W |
| 2 units | $15,998 | 256 GB | 31.2 | 480 W | |
| 8 cards | $16,000 | 192 GB | 78.9 | 2,800 W | |
| 8 cards | $17,224 | 256 GB | 58.9 | 1,840 W | |
| 2 cards | $18,994 | 144 GB | 97.5 | 600 W | |
| 4 cards | $19,800 | 192 GB | 69.4 | 1,200 W | |
| Instinct MI21064 GB per card | 4 cards | $20,176 | 256 GB | 108.1 | 1,200 W |
| 8 cards | $29,200 | 192 GB | 82.6 | 3,600 W | |
| 2 cards | $32,000 | 160 GB | 116.9 | 600 W | |
| 2 cards | $38,000 | 192 GB | 112.8 | 1,200 W | |
| 8 cards | $54,312 | 256 GB | 112.8 | 4,600 W | |
| 4 cards | No price | 192 GB | 82.6 | 1,800 W |
| Device | Cards | Total price | Total VRAM | Speed t/s | Total power |
|---|---|---|---|---|---|
| 8 cards | $9,992 | 256 GB | 59.9 | 2,000 W | |
| Radeon RX 7900 XTX24 GB per card | 8 cards | $11,200 | 192 GB | 62.7 | 2,840 W |
| 2 units | $15,998 | 256 GB | 22.6 | 480 W | |
| 8 cards | $16,000 | 192 GB | 61.6 | 2,800 W | |
| 8 cards | $17,224 | 256 GB | 44.5 | 1,840 W | |
| 4 cards | $19,800 | 192 GB | 53.3 | 1,200 W | |
| Instinct MI21064 GB per card | 4 cards | $20,176 | 256 GB | 88.6 | 1,200 W |
| 8 cards | $29,200 | 192 GB | 64.9 | 3,600 W | |
| 4 cards | $37,988 | 288 GB | 78.6 | 1,200 W | |
| 2 cards | $38,000 | 192 GB | 93.3 | 1,200 W | |
| 8 cards | $54,312 | 256 GB | 93.3 | 4,600 W | |
| 4 cards | $64,000 | 320 GB | 97.3 | 1,200 W | |
| 4 cards | No price | 192 GB | 64.9 | 1,800 W |
| Device | Cards | Total price | Total VRAM | Speed t/s | Total power |
|---|---|---|---|---|---|
| 8 cards | $9,992 | 256 GB | 41 | 2,000 W | |
| 2 units | $15,998 | 256 GB | 14.4 | 480 W | |
| 8 cards | $17,224 | 256 GB | 29.5 | 1,840 W | |
| Instinct MI21064 GB per card | 4 cards | $20,176 | 256 GB | 64.4 | 1,200 W |
| 4 cards | $37,988 | 288 GB | 55.9 | 1,200 W | |
| 8 cards | $39,600 | 384 GB | 36 | 2,400 W | |
| 8 cards | $54,312 | 256 GB | 68.5 | 4,600 W | |
| 4 cards | $64,000 | 320 GB | 72.1 | 1,200 W | |
| 4 cards | $76,000 | 384 GB | 68.5 | 2,400 W | |
| 8 cards | No price | 384 GB | 44.9 | 3,600 W |
Full calculation rules are on the Methodology page.
What PC or server build do you need to run Hy4 Flash locally?
If you only have one or two regular GPUs, putting Hy4 Flash's experts and lookup table in system RAM is the practical approach: at 4-bit the GPU holds only about 8 GB of shared weights used on every token, paired with 128GB to 256GB of RAM. One RTX 3090 in a dual-socket EPYC DDR4 server reaches an estimated 40 tokens/s, while a dual-channel desktop maxed out at 128GB with two RTX 5070 cards fits only 2-bit at an estimated 29 tokens/s. These speeds are estimated from memory bandwidth, as there are no public MoE offloading benchmarks for this model yet.
2 × Tesla V100 32GB + Dual Xeon E5 DDR4
BudgetGeForce RTX 3090 + Single EPYC DDR4
Best valueGeForce RTX 5090 + Single EPYC DDR5
Flagship2 × Tesla V100 32GB + Dual Xeon E5 DDR4
BudgetGeForce RTX 3090 + Dual EPYC DDR4
Best valueGeForce RTX 5090 + Single EPYC DDR5
FlagshipTesla V100 32GB + Dual Xeon E5 DDR4
BudgetGeForce RTX 3090 + Dual EPYC DDR4
Best valueGeForce RTX 5090 + Dual EPYC DDR5
FlagshipDetailed calculation rules are on the Methodology page.
As an eBay Partner, VRAMGlass may be compensated if you buy through the eBay links on this page. The price you pay stays the same.
What to know before running Hy4 Flash locally
You have to compile the official llama.cpp branch
Support for Hy4 Flash in llama.cpp is in PR #32517, which was not merged as of November 18. AngelSlim's GGUF guide gives two options: build the branch from that PR directly, or apply the patches included in the repository to a specified older commit, with separate patches for Mac and DGX Spark. Pass
--jinjawhen chatting.Why the parameter count says 220B
The 220B total on the model card includes about 63.5B in the Engram lookup table, which the model queries using combinations of two or three adjacent tokens like a built-in phrase memory. The backbone is 155.5B and each token uses about 12B, so while the file size is in the 200B+ class, generation speed scales like a 12B model.
Which quantization to pick
The official GGUF repository offers three standard quantizations. Q4_K_M is about 168 GB and is AngelSlim's recommended default; drop to Q3_K (about 155 GB) or IQ2_AH (about 135 GB) if memory is tight. The lookup table is the same size in all three and stepping down only compresses the experts, so going from 4-bit to 2-bit saves only about 33 GB. There are also two SPARK-4Bit builds for DGX Spark that use NVIDIA's 4-bit floating-point format.
Thinking mode is on by default
reasoning_effortdefaults to high, meaning thinking is enabled; passno_thinkif you want direct answers. The officially recommended sampling parameters aretemperature0.9 andtop_p1.0. The model supports vision, but reading images in llama.cpp requires loading a separate mmproj file, which had not been uploaded to the GGUF repository as of November 18.1M context adds only 14 GB
Only 12 of the 42 layers attend to the full context while the other 30 layers look only at the most recent 512 tokens, keeping the KV cache small. Tencent says the context cache is 1/24 the size of Hy3's. AngelSlim's recommended setup for DGX Spark sets the context length straight to about 196K.
The license is no longer Apache 2.0
Hy4 Flash switches to Tencent's own Tencent Hy License, which takes Apache 2.0 and adds one commercial restriction: companies offering external model API services with more than 50 million US dollars in annual global revenue, or using the model in products with over 100 million monthly active users, must get written authorization from Tencent first. Personal use and internal company use are not affected by this clause.
Launch commands
llama.cpp
- NVIDIA
- AMD
- Mac
- CPU
Follow the deployment guide to prepare the required runtime and hardware.
./build/bin/llama-server -m Hy4-flash-Q4_K_M-00001-of-00002.gguf \ -c 8192 -ngl 99 --jinja --port 8080
vLLM
- Linux
- NVIDIA
- Docker
Follow the deployment guide to prepare the required runtime and hardware.
docker run --gpus all \ -p 8000:8000 \ --ipc=host \ -v ~/.cache/huggingface:/root/.cache/huggingface \ vllm/vllm-openai:hy4-flash tencent/Hy4-flash-FP8 \ --tensor-parallel-size 8 \ --speculative-config '{"num_speculative_tokens":5,"method":"dfly"}' \ --attention-backend FLASHMLA_SPARSE \ --tool-call-parser hy_v4 \ --reasoning-parser hy_v4 \ --enable-auto-tool-choice \ --port 8000 \ --served-model-name hy4-flash
Ollama
- Windows
- macOS
- Linux
The Ollama library has no official tag for it yet.
File names follow that week's model-library snapshot; the definitions are in the methodology. Methodology
Other models in the Hunyuan family
| Model | Total params | Active params | 4-bit size |
|---|---|---|---|
| 295B | 21B | 182.2 GB | |
| 770B | 49B | 467.3 GB |