Run Big Models Cheaply on a Used Server with DDR4: How Fast a Dual-3090 Build Goes and Where the Money Goes (2026)

Why can a server with DDR4 run a 300B model?

Because these models are MoE: each token uses only a small share of the parameters, and that small share can be read from RAM. The software that does this is llama.cpp and its fork ik_llama.cpp. DeepSeek V4 Flash has 284B parameters and 256 experts, and each token activates only 6 experts plus the attention layers, 13B in total. GLM 5.3 Flash is 320B with 288 experts and activates 18B per token.

llama.cpp splits the two kinds of weights. Attention layers, shared experts and KV cache go on the graphics card and take a few GB. The 256 experts go in RAM and take 100 to 200 GB. For every token generated, the card computes its own part, then reads 6 to 8 experts from RAM. How fast that read runs is how fast generation runs.

Read speed comes from channel count times clock:

A desktop can do it too, but an 8-channel server is more than twice as fast, and a desktop takes 128GB at most, which cannot hold V4 Flash at Q4. That is why you buy a server: not for the CPU, for the 8 memory slots.

Why two RTX 3090s?

One 3090 does three jobs: hold the attention layers and shared experts, hold the KV cache, and process prompts in batches. The second card doubles the KV cache, holds six more expert layers and speeds up prompt processing, while generation gains only about ten percent. First, where the 24GB of a single card goes:

Most of the generation time sits in the RAM portion. Each token reads 3.7 GB of experts (V4 Flash) or 4.9 GB (GLM) from RAM, which takes 30 to 40 milliseconds on 8-channel DDR4-2666. The GPU portion takes only 5 to 8 milliseconds. So the six extra expert layers on the second card only shorten the RAM portion by a little over ten percent, moving generation from 25 to 27 tokens/s.

What the second card is really worth is the other two jobs:

• Context. KV cache goes from the dozen or so GB left on one card to more than forty GB. V4 Flash can then run at 128K, and multi-turn agent sessions do not need constant clearing.
• Prompt processing. Reading the prompt happens in batches. llama.cpp sends a whole batch of tokens to the GPU, temporarily moving even the RAM-resident experts over to compute. More cards and bigger batches make this step faster: 65 to 500 tokens/s in practice, depending on batch size and context length.

Picking the 3090 over a 4090 or 5090 is simple. The graphics card is not the bottleneck in this build, and 24GB at about $67 per GB is the cheapest VRAM you can buy. Two 3090s cost the same as a single 4090 and bring 48GB instead of 24GB. The two cards do not need NVLink.

The counterexample belongs up front too. One user on 4-channel DDR4 got only 5 to 6 tokens/s with two 3090s and 9.5 after going back to one card. Another, with four 3090s and 256GB of DDR4, reported "6 at most". A badly tuned multi-GPU split is slower, so get one card working first and add the second afterwards (Unsloth V4 Flash discussion).

How fast does it go, and how much VRAM and RAM does each precision need?

Q4 is the precision you can live with: V4 Flash needs 155 GB and GLM 5.3 Flash 200 GB of RAM. Generation on two 3090s with 8-channel DDR4 runs about 27 and 21 tokens/s. Dropping to 3-bit is twenty percent faster, 2-bit fifty percent. First the file sizes at each precision. Both models use Unsloth's UD dynamic quants, and accuracy is Unsloth's top-1 measured on GLM 5.3 Flash (V4 Flash GGUF, GLM 5.3 Flash GGUF):

Take the file size, subtract the few GB that go on the card, and leave twenty percent of headroom. Context length mostly affects the VRAM share. The table below is estimated live from the model database by parameter count, giving total requirements for 3-bit and 4-bit at 32K, 128K and 256K context; the offload column shows the split between card and RAM:

Generation speed comes from this site's expert-offload formula (methodology) at 32K context:

The formula adds three segments per token: the card reads the attention layers, shared experts and whatever expert layers sit in VRAM at 936 GB/s; RAM supplies the remaining experts at 171 GB/s for 8-channel DDR4-2666; then 4 milliseconds of routing overhead. Both bandwidths are multiplied by 0.7 efficiency. DDR4-3200 is ten percent faster and 12-channel DDR5 seventy percent faster.

Check the formula against published measurements. Three 3090s with DDR4 running GLM 5.3 Flash at IQ3_XXS measured 14.6 tokens/s of generation against a formula value of 25, sixty percent. A dual-socket EPYC with 8-channel DDR4-2666 running V4 Flash at Q8_K_XL measured 20, against 27 for the formula on a Q4 basis, seventy percent (ik_llama.cpp PR #2376, Unsloth V4 Flash discussion). So read the table as 60 to 100 percent, and 18 to 27 for V4 Flash Q4 is normal.

Two things pull real numbers away from that table:

• Prompt processing is a different number. Do not mix them up. llama.cpp prints two speed lines after each answer. prompt eval is the speed of reading the prompt, and 65 to 500 tokens/s is all normal with batch size at 4096. eval is the speed of generating the answer, which is the number in the speed table above. Posts online claiming "over 100" are usually quoting the first line.
• Longer context is slower. On one machine with a single 3090 and 8-channel DDR4-3200, feeding GLM 5.3 Flash a 69K prompt left only 6.1 tokens/s of generation, because every token has a much larger attention computation.

Where does the money go, and how much cheaper is this than multi-GPU or a Mac?

Buy DDR4 only, in one of two tiers: 8 × 16GB for 128GB at about $3,600, which reaches 3-bit; or 8 × 32GB for 256GB at about $5,600, which reaches Q4, with V4 Flash even fitting the lossless Q8_K_XL. Memory prices are live data, computed for a single socket with all 8 channels filled:

Both tiers at current eBay prices:

The CPU and board line is not something this site tracks. It is derived from September 2026 eBay prices of $2,700 to $3,300 for complete dual EPYC 7642 servers with 512GB. The 128GB tier is about $2,000 cheaper, and the price is precision stopping at 3-bit: GLM 5.3 Flash scores 82% at 3-bit against 92% at Q4, and you feel it in code and long reasoning. Reaching 256GB with 16GB sticks would take 16 of them, more than a single-socket board can hold.

Now the other builds that also hold V4 Flash Q4:

Three conclusions:

• GPU-only builds have no middle tier. The 96GB of four 3090s cannot even hold V4 Flash at 2-bit (97 GB). Holding Q4 takes 8 cards, $12,800 in cards alone, plus a board with 8 slots, riser cables and a 3000W supply. Two RTX PRO 6000 come to about $34,000.
• The Mac is half again as fast and nearly four times the price. The 819 GB/s unified memory of the M3 Ultra 512GB gives an estimated 43 tokens/s on V4 Flash Q4 and runs the lossless version, quietly, efficiently and with no fiddling, for $21,250.
• The Ryzen AI Max+ 395 is the rival to the 128GB tier. A $4,429 mini PC whose 128GB of unified memory runs at only 256 GB/s, giving an estimated 16 tokens/s on V4 Flash at 3-bit, half the speed of the 128GB DDR4 server. Size and noise are in another world.

DDR4 costs about $6.3 per GB, one sixth the price of DDR5 server sticks and one tenth the price of 3090 VRAM, which is why DDR4 is the only version of this route that works in 2026. The bad news is that DDR4 is in its end-of-life cycle: Micron issued an end-of-life notice in June 2026, and Samsung and SK Hynix have only delayed the phase-out to 2026 (TrendForce). In January 2025 someone bought 512GB of DDR4-2400 for $400 (Digital Spaceport); in September 2026 the same amount costs $3,200 on eBay. Buy the memory first.

Three rules for the rest of the parts:

• Buy an EPYC 7003, or a 7002, but avoid the 7232P, 7252, 7262, 7272 and 7282. Those five have only two CCDs and half the memory bandwidth (ServeTheHome). 32 cores is plenty and more brings no return.
• The board must be single-socket with 8 slots. The Gigabyte MZ32-AR0 and Supermicro H12SSL are common picks. Buy a dual-socket board only once you are sure you want more than 16 sticks.
• Use one kind of memory throughout. RDIMM and LRDIMM cannot be mixed. When 2666 and 3200 are close in price, buy 3200 for the twenty percent more bandwidth.

How do you start it once it is built?

Use llama-server from llama.cpp, put all attention layers on the card and all expert weights in RAM, then move experts onto the card once it runs. Step one, all experts in RAM:

./llama-server -m DeepSeek-V4-Flash-0731-UD-Q4_K_XL-00001-of-00004.gguf \
  -ngl 99 --n-cpu-moe 43 -t 32 -c 32768 -fa on -b 4096 -ub 4096 \
  --host 0.0.0.0 --port 8080

Three parameters decide everything. -ngl 99 hands every layer to the card. --n-cpu-moe 43 then pushes the expert weights of 43 layers back to RAM; V4 Flash has 43 layers and GLM 5.3 Flash takes 45. -b 4096 -ub 4096 sends the prompt to the card in batches of 4096 tokens, which is the key to prompt processing in the hundreds; without it you get tens. Set -t to the physical core count.

Step two, watch the VRAM headroom in nvidia-smi and lower --n-cpu-moe. Each step down moves one layer of experts (about 3.7 GB for V4 Flash) onto the card and speeds generation up a little. One 3090 gets to about 39 and two to about 33. How much headroom to leave for KV cache is something you test yourself. With two cards, add -ts 1,1 first and confirm both cards' VRAM is in use before touching this number.

ik_llama.cpp is a fork of llama.cpp with faster MoE kernels on the CPU, which suits GLM 5.3 Flash at long context. The command from the three-3090 measurement above was:

./llama-server -m GLM-5.3-Flash-UD-IQ3_XXS-00001-of-00003.gguf \
  --flash-attn on -mla 1 -ctk f16 -ctv f16 --dsa -ngl 999 -ot exps=CPU \
  -amb 512 -b 8192 -ub 2048 -c 32768

-ot exps=CPU and --n-cpu-moe do the same thing, using a regex to point all expert tensors at the CPU. To keep some layers on the card, write something like -ot "blk\.(3[0-9]|4[0-4])\.ffn.*exps=CPU" that matches only the later layers.

What are the traps?

Three of them, ordered by cost: dual-socket NUMA, dense models, and the multi-GPU split.

1. A dual-socket server is not twice as fast. Each CPU has its own 8 channels, so theoretical bandwidth doubles, but the cost of cross-node memory reads in llama.cpp is still an open problem (llama.cpp Issue #1437), and you get 1.3 to 1.6 times in practice. When 8 × 32GB on a single socket is not enough, switch to 64GB sticks before considering dual socket. If you do go dual, add --numa distribute.
2. Do not take this route for dense models. A fully activated model like Qwen3.8 27B reads 16 GB from RAM per token, which is only 8 tokens/s on 8-channel DDR4-3200, and more CPU cores do not help. Put dense models back on the card; the arithmetic is in not enough VRAM: does adding RAM help.
3. A badly tuned multi-GPU split is slower. The example above of two 3090s losing to one is exactly this. With multiple cards, set the ratio by hand with -ts first, confirm both cards' VRAM is in use, then touch --n-cpu-moe.

Conclusion

For one person, targeting 300B-class MoE models, willing to live with twenty or thirty tokens/s of generation, the $5,600 machine in the 256GB tier is the cheapest answer in 2026. If you only want to try it out or your budget stops near $3,600, take the 128GB tier at 3-bit. If you need more than 50 tokens/s, or your main models are dense, do not buy it.

• To run DeepSeek V4 Flash and GLM 5.3 Flash at Q4: single-socket EPYC, 8 × 32GB DDR4, two 3090s.
• To run Qwen3.8 Flash Next, gpt-oss 120B, or 300B models at 3-bit: 8 × 16GB and one 3090 is enough.
• To run the 763B DeepSeek V4.1 Flash or the 1T Kimi K2.6: swap the memory for 8 × 64GB to reach 512GB and leave the rest unchanged.
• If you only have a desktop and target 100B-class MoE: skip the server. 96GB of RAM and one 16GB card is enough. See not enough VRAM: does adding RAM help.
• If you want half again the speed and do not care about money: Mac Studio M3 Ultra 512GB. See Mac unified memory versus discrete GPUs.