VRAMGlass contains affiliate links. As an eBay Partner, we may be compensated if you make a purchase. As an Amazon Associate I earn from qualifying purchases.We may earn from eBay and Amazon links.Learn more

GPUs tracked64Price history78 daysPrice points (24h)32,149 pointsBiggest 24h dropA100 80GB PCIe-15.8%Biggest 24h riseRadeon RX 6950 XT+9.4%Updated
$ USD

Run Hy4 Flash Locally: VRAM & GPU Requirements

Hy4 Flash is Tencent Hunyuan's 220B multimodal MoE from November 2026, using about 12B per token and 168 GB at 4-bit. Because about 63 GB of that is a lookup table that can stay on disk and be read on demand, Tencent's benchmarks show a 128GB Mac or DGX Spark can run 4-bit. Support in llama.cpp is not merged into mainline yet, so you have to compile the official branch yourself, and Ollama does not have it.

Hy4 Flash details

  • Min VRAM at 4-bit (8K context)170 GB
  • Parameters220B
  • Active params12B
  • Max context1M
Architecture
MoE
Experts offloaded
8 GB VRAM + 164 GB RAM
Launch date
Nov 18, 2026
Vendor
Tencent
Hugging Face repo
tencent/Hy4-flash
Layers
42
Hidden size
3,072
KV heads
32
Head dim
64

How much VRAM does Hy4 Flash need?

Hy4 Flash's 4-bit file is about 168 GB, and loading the whole model into VRAM with 32K context takes about 170 GB; 2-bit is about 135 GB and 3-bit is about 155 GB. All of those numbers include an Engram lookup table of about 63 GB. Generation looks up only a few dozen rows per token, so that table can sit in system RAM or stay on disk, leaving about 105 GB of 4-bit weights that actually need to stay resident. Context is very light, adding only 14 GB when scaling from 32K all the way to 1M.

QuantizationWeightsGB32K contextGB128K contextGB256K contextGB1M context (max)GB
1-bit102.8est.104.4105.7107.5118.4
2-bit135GGUF136.5137.9139.7150.6
3-bit155.4GGUF156.9158.3160.1171
4-bit168.3GGUF169.8171.2173183.9
5-bit176.8est.178.4179.7181.5192.4
8-bit237.7est.239.2240.6242.4253.2
BF16/FP16392.2est.393.8395.1396.9407.8

Full calculation rules are on the Methodology page.

Can a single GPU run Hy4 Flash?

It depends on the quantization and on where you keep the lookup table. If the entire file stays in memory, a 128GB machine fits only 2-bit, while 4-bit needs a 512GB Mac Studio M3 Ultra. Leaving the ~63 GB lookup table on disk to be read on demand lets 4-bit run in 128GB: Tencent's AngelSlim team measured about 44 to 48 tokens/s on a 128GB M5 Max, and about 26 to 33 tokens/s on a DGX Spark using the dedicated SPARK-4Bit build. The largest discrete GPU has 96GB, which cannot hold the ~105 GB resident portion of 4-bit without multiple cards or MoE offloading to system RAM.

eBay
DevicePriceSpeed t/sMax context
Mac Studio M3 Ultra 512GBWhole system · 512 GB$25,0009 listings56.81M

Only common GPUs and systems are listed, from cheapest to most expensive.

How many GPUs do you need to run Hy4 Flash?

Running Hy4 Flash at 4-bit across multiple GPUs takes at least two 96GB RTX PRO 6000 cards, four 48GB cards, or eight 24GB consumer cards. That assumes splitting the entire file evenly across the GPUs; moving the lookup table to system RAM saves another 63 GB or so of VRAM. Multi-GPU inference with quantized files uses llama.cpp. The official vLLM recipe runs the full-precision and FP8 weights only on datacenter GPUs like the H100, H200 and B200, and puts the lookup table in system RAM by default.

eBay
DeviceCardsTotal priceTotal VRAMSpeed t/sTotal power
Tesla V100 32GB32 GB per card8 cards$9,992256 GB59.92,000 W
Radeon RX 7900 XTX24 GB per card8 cards$11,200192 GB62.72,840 W
DGX Spark 128GB128 GB per unit2 units$15,998256 GB22.6480 W
GeForce RTX 309024 GB per card8 cards$16,000192 GB61.62,800 W
Arc Pro B7032 GB per card8 cards$17,224256 GB44.51,840 W
RTX A600048 GB per card4 cards$19,800192 GB53.31,200 W
Instinct MI21064 GB per card4 cards$20,176256 GB88.61,200 W
GeForce RTX 409024 GB per card8 cards$29,200192 GB64.93,600 W
RTX PRO 5000 72GB72 GB per card4 cards$37,988288 GB78.61,200 W
RTX PRO 600096 GB per card2 cards$38,000192 GB93.31,200 W
GeForce RTX 509032 GB per card8 cards$54,312256 GB93.34,600 W
A100 80GB PCIe80 GB per card4 cards$64,000320 GB97.31,200 W
GeForce RTX 4090 48GB48 GB per card4 cardsNo price192 GB64.91,800 W

Full calculation rules are on the Methodology page.

What PC or server build do you need to run Hy4 Flash locally?

If you only have one or two regular GPUs, putting Hy4 Flash's experts and lookup table in system RAM is the practical approach: at 4-bit the GPU holds only about 8 GB of shared weights used on every token, paired with 128GB to 256GB of RAM. One RTX 3090 in a dual-socket EPYC DDR4 server reaches an estimated 40 tokens/s, while a dual-channel desktop maxed out at 128GB with two RTX 5070 cards fits only 2-bit at an estimated 29 tokens/s. These speeds are estimated from memory bandwidth, as there are no public MoE offloading benchmarks for this model yet.

eBay

2 × Tesla V100 32GB + Dual Xeon E5 DDR4

Budget
Generation speed32.9t/s
Memory bandwidth137GB/s
GPU
2 × Tesla V100 32GB25 / 42 expert layers in 64 GB VRAM
$2,498
RAM
128 GB DDR4-21338 × 16 GB sticks
$560
CPU
2 × Intel Xeon E5-2680 v428 cores / 56 threads · 8-channel in total
$38
Board
Supermicro X10DRi16 DIMM slots
$275
PSU
Corsair RM1200x SHIFT1,200 W
$150
Reference build price$3,521

GeForce RTX 3090 + Dual EPYC DDR4

Best value
Generation speed39.5t/s
Memory bandwidth341GB/s
GPU
GeForce RTX 30907 / 42 expert layers in 24 GB VRAM
$2,000
RAM
256 GB DDR4-266616 × 16 GB sticks
$1,584
CPU
2 × AMD EPYC 753264 cores / 128 threads · 16-channel in total
$576
Board
Supermicro H12DSi-N616 DIMM slots
$1,100
PSU
Corsair RM1200x SHIFT1,200 W
$150
Reference build price$5,410

GeForce RTX 5090 + Single EPYC DDR5

Flagship
Generation speed63.6t/s
Memory bandwidth461GB/s
GPU
GeForce RTX 509011 / 42 expert layers in 32 GB VRAM
$6,789
RAM
192 GB DDR5-480012 × 16 GB sticks
$6,600
CPU
AMD EPYC 935432 cores / 64 threads · 12-channel
$1,890
Board
Supermicro H13SSL-N12 DIMM slots
$820
PSU
Corsair RM1200x SHIFT1,200 W
$150
Reference build price$16,249

Detailed calculation rules are on the Methodology page.

As an eBay Partner, VRAMGlass may be compensated if you buy through the eBay links on this page. The price you pay stays the same.

What to know before running Hy4 Flash locally

  1. You have to compile the official llama.cpp branch

    Support for Hy4 Flash in llama.cpp is in PR #32517, which was not merged as of November 18. AngelSlim's GGUF guide gives two options: build the branch from that PR directly, or apply the patches included in the repository to a specified older commit, with separate patches for Mac and DGX Spark. Pass --jinja when chatting.

  2. Why the parameter count says 220B

    The 220B total on the model card includes about 63.5B in the Engram lookup table, which the model queries using combinations of two or three adjacent tokens like a built-in phrase memory. The backbone is 155.5B and each token uses about 12B, so while the file size is in the 200B+ class, generation speed scales like a 12B model.

  3. Which quantization to pick

    The official GGUF repository offers three standard quantizations. Q4_K_M is about 168 GB and is AngelSlim's recommended default; drop to Q3_K (about 155 GB) or IQ2_AH (about 135 GB) if memory is tight. The lookup table is the same size in all three and stepping down only compresses the experts, so going from 4-bit to 2-bit saves only about 33 GB. There are also two SPARK-4Bit builds for DGX Spark that use NVIDIA's 4-bit floating-point format.

  4. Thinking mode is on by default

    reasoning_effort defaults to high, meaning thinking is enabled; pass no_think if you want direct answers. The officially recommended sampling parameters are temperature 0.9 and top_p 1.0. The model supports vision, but reading images in llama.cpp requires loading a separate mmproj file, which had not been uploaded to the GGUF repository as of November 18.

  5. 1M context adds only 14 GB

    Only 12 of the 42 layers attend to the full context while the other 30 layers look only at the most recent 512 tokens, keeping the KV cache small. Tencent says the context cache is 1/24 the size of Hy3's. AngelSlim's recommended setup for DGX Spark sets the context length straight to about 196K.

  6. The license is no longer Apache 2.0

    Hy4 Flash switches to Tencent's own Tencent Hy License, which takes Apache 2.0 and adds one commercial restriction: companies offering external model API services with more than 50 million US dollars in annual global revenue, or using the model in products with over 100 million monthly active users, must get written authorization from Tencent first. Personal use and internal company use are not affected by this clause.

Launch commands

llama.cpp

  • NVIDIA
  • AMD
  • Mac
  • CPU

Follow the deployment guide to prepare the required runtime and hardware.

Terminal
./build/bin/llama-server -m Hy4-flash-Q4_K_M-00001-of-00002.gguf \
  -c 8192 -ngl 99 --jinja --port 8080

vLLM

  • Linux
  • NVIDIA
  • Docker

Follow the deployment guide to prepare the required runtime and hardware.

Terminal
docker run --gpus all \
  -p 8000:8000 \
  --ipc=host \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:hy4-flash tencent/Hy4-flash-FP8 \
    --tensor-parallel-size 8 \
    --speculative-config '{"num_speculative_tokens":5,"method":"dfly"}' \
    --attention-backend FLASHMLA_SPARSE \
    --tool-call-parser hy_v4 \
    --reasoning-parser hy_v4 \
    --enable-auto-tool-choice \
    --port 8000 \
    --served-model-name hy4-flash

Ollama

  • Windows
  • macOS
  • Linux

The Ollama library has no official tag for it yet.

File names follow that week's model-library snapshot; the definitions are in the methodology. Methodology

Other models in the Hunyuan family

ModelTotal paramsActive params4-bit size
Hy3MoE295B21B182.2 GB
Hy4 previewMoE770B49B467.3 GB

About Hunyuan