Not Enough VRAM: Does Adding RAM Help? What Local LLM Offloading Actually Moves (2026)

What separates VRAM from RAM?

A PCIe bus and a bandwidth gap of more than tenfold: VRAM delivers 900 to 1800 GB/s, dual-channel DDR5 about 70 GB/s, and PCIe 4.0 x16 about 32 GB/s. Generating each token requires reading all the weights involved in its computation. Where those bytes are read from, and which path they take, determines speed.

GPU bandwidth comes from each model's specifications. RAM and PCIe figures use values for typical desktops: dual-channel DDR5-5600 has a theoretical 89.6 GB/s and achieves 70–80% in practice; PCIe 4.0 x16 has a theoretical one-way 31.5 GB/s, and PCIe 5.0 doubles that to 63 GB/s (PCI Express bandwidth by generation, DDR5 specifications). This site's speed formula uses 70 GB/s for system RAM multiplied by 0.7 efficiency; see the methodology.

So “16GB VRAM plus 64GB RAM” is not 80GB VRAM. For every token, the weights in RAM must take a path more than ten times slower, while the fast portion waits. The whole point of offloading is to minimize the bytes taking that slow path.

What does offloading actually move?

The three methods move different things: dense-layer offloading moves weights needed for every token; MoE expert offloading moves experts of which only a small fraction are needed per token; diffusion block offloading moves weights needed at every step, but only once per step. All are called offloading, but their costs differ by an order of magnitude.

1. Dense-layer offloading. llama.cpp uses -ngl to specify how many layers go on the GPU; the remaining layers run on the CPU. Every token in a dense model passes through every layer, so CPU-resident layers run at RAM bandwidth. This is the most common and least worthwhile form of offloading.
2. MoE expert offloading. Each MoE layer has dozens to hundreds of experts, but each token activates only a few. llama.cpp's --n-cpu-moe places all expert weights in RAM while attention layers, shared experts and KV cache stay on the GPU. Each token reads only the activated experts from RAM, so the bytes taking the slow path are only a few percent of total weights. gpt-oss 120B has 117B total parameters but reads only 3.6B of experts per token, which is why it runs even on a 16GB card.
3. Diffusion block offloading. ComfyUI and Diffusers keep blocks of UNet or DiT weights in RAM, move them onto the GPU in sequence at each inference step, and replace each block after computation. Every step transfers the whole model across PCIe, but diffusion has substantial computation per step and a fixed step count, so the extra transfer time typically adds seconds to tens of seconds per image. The model offload and sequential offload in Diffusers documentation are these two granularities (Diffusers memory optimization).

Of these three, only the second makes adding RAM a serious solution. Here is the breakdown.

Dense-layer offloading: how much speed do you lose per offloaded layer?

Offload 10% of the layers and speed halves; offload 30% and only a quarter remains. Using this site's formula for Qwen3 32B at 4-bit on an RTX 3090, here is what happens as the share of weights on the GPU drops from 100%:

Speed drops so quickly because each token's time is the sum of two portions: the GPU portion reads at 900 GB/s and the RAM portion at 49 GB/s. Put 1.8 GB in RAM and that 1.8 GB takes 37 milliseconds per token, longer than the 26 milliseconds the GPU takes to read the remaining 16.5 GB. As soon as the RAM portion appears, it becomes the bottleneck; however fast the GPU is, it must wait.

In a real GPU-buying scenario, a 16GB RTX 5070 Ti running Qwen3 32B at 4-bit has 18.3 GB of weights plus 2 GB of KV cache and can keep only 66% on the GPU, for an estimated 7 tokens/s. The same model fits wholly in VRAM on a 24GB RTX 3090, at an estimated 32 tokens/s. Gemma 4 31B fares worse: its large KV cache leaves room for only 37% of its weights on a 16GB card, at an estimated 4 tokens/s.

The conclusion is straightforward: when a dense model exceeds VRAM, adding RAM buys “runnable, but not usable.” To make it usable, first reduce context, then move to 3-bit, and finally switch to a 24GB card; see the sequence in what models run on 16GB. You can tolerate 2 tokens/s for an occasional 70B run, but not for daily use.

MoE expert offloading: where extra RAM truly helps

Once MoE experts are in RAM, VRAM requirements fall to just 4 to 7 GB, speed is determined by RAM bandwidth, and any GPU with 16GB or more is much the same. The following table is calculated live from the model database; its offload column gives two figures, VRAM plus RAM:

The three size tiers call for different judgments:

• 30B-class MoE (Qwen3.6 35B A3B, Gemma 4 26B A4B, GLM 4.7 Flash): the whole model at 4-bit needs 17 to 22 GB VRAM. A 24GB card holds it directly, at an estimated 110 to 145 tokens/s. Offloading on a 16GB card uses 4 to 6 GB VRAM and 13 to 19 GB RAM, at an estimated 38 to 54 tokens/s. 32GB RAM is enough, and speed is sufficient for everyday chat and coding.
• 100B-class MoE (gpt-oss 120B, GLM 4.5 Air): the whole model needs 62 to 67 GB VRAM, beyond any single consumer card. Offloading needs 4 to 7 GB VRAM and 56 to 65 GB RAM, at an estimated 12 to 21 tokens/s. This is where extra RAM is most worthwhile: a 16GB card plus 96GB RAM runs a model that would otherwise need an RTX PRO 6000 to fit in full.
• 300B-class MoE (DeepSeek V4 Flash): RAM requirements exceed 155 GB, at an estimated 12 tokens/s. It runs, but installed RAM itself starts at 192GB; see the assessment of the full model in the DeepSeek GPU guide.

The most counterintuitive point is that GPU model barely affects speed in offload mode:

An asterisk marks offload speed. gpt-oss 120B runs at 21 tokens/s on RTX 5070 Ti, RTX 3090 and RTX 5090 alike, because most of each token's time is the more than 40 milliseconds spent reading 3.6B active experts from RAM; the GPU portion takes only one or two milliseconds. The same model wholly in the 96GB VRAM of an RTX PRO 6000 reaches 153 tokens/s, and in the unified memory of a Mac Studio M4 Max 128GB reaches 68 tokens/s. To speed up a 100B-class MoE, either fit the whole model in VRAM or use Mac unified memory; see the Mac unified-memory comparison. Buying a more expensive GPU for offloading is wasted money.

What should you check before adding RAM?

Confirm RAM bandwidth, capacity and llama.cpp parameters before ordering. Check these three things in order:

1. Channel count determines bandwidth. Speeds above assume dual-channel DDR5 at about 70 GB/s. Laptops and mini PCs often have only single-channel or soldered memory; halve bandwidth and offload speed halves too, while a quad-channel workstation platform doubles it. Install memory modules in pairs: using the wrong slots and ending up with single-channel memory is the most common waste.
2. Capacity should be expert weights plus 20%. The OS, runtime buffers beyond KV cache and your browser all consume RAM. For gpt-oss 120B's 65 GB expert weights, use 96GB; for GLM 4.5 Air's 56 GB, 64GB is tight and 96GB comfortable; for Qwen3.6 35B A3B's 19 GB, 32GB is enough. When RAM runs out, the OS swaps to disk and speed falls below 1 tokens/s, which is more frustrating than not running at all.
3. Use the correct parameters. Add --n-cpu-moe N when starting llama.cpp. N is the number of layers whose experts go on the CPU; start at the model's layer count and work down, reducing it when VRAM headroom allows more layers on the GPU. -ngl 99 ensures all attention layers go on the GPU. LM Studio has a model-loading switch to put expert weights on the CPU; for other frontends, follow their version's documentation (llama.cpp repository).

Windows users face another trap: NVIDIA drivers enable system-memory fallback by default. When VRAM fills, data automatically spills into RAM, the application reports no error, and speed falls straight to single digits. This is not deliberate offloading; it is the driver doing the worst kind for you. Under Manage 3D Settings in NVIDIA Control Panel, change CUDA – Sysmem Fallback Policy to Prefer No Sysmem Fallback. Insufficient VRAM will then produce an error, letting you tell whether the model actually fits.

When should you replace the GPU instead of adding RAM?

If your usual model is dense and exceeds VRAM by more than 20%, replace the card; if it is MoE, add RAM. The two routes cost roughly the same but differ tenfold in effect.

For a dense model like Qwen3 32B, a 16GB RTX 5070 Ti plus 64GB RAM gives 7 tokens/s after offloading; a 24GB RTX 3090 holding the whole model in VRAM gives 32 tokens/s. A used 3090 is cheaper than a 5070 Ti, and the extra VRAM is worth much more than extra RAM; see 3090 versus 5070 Ti.

Now consider gpt-oss 120B: an RTX 5090 gives 21 tokens/s after offloading, and so does an RTX 5070 Ti. The cards differ in price by around CNY 10,000, with no difference at all for this model. Spend that money on 96GB RAM and more than half remains.

There are just three rules:

• If your usual model is dense and the VRAM shortfall is within 20%, first reduce quantization precision or context; above 20%, replace the card.
• If your usual model is MoE, add RAM to 1.2 times the expert-weight size; a 16GB GPU is enough.
• For a large model you run only occasionally, tolerate offloading and do not buy anything specifically for it.

How does offloading differ for image and video generation?

Diffusion offloading costs a few extra seconds of transfers per step rather than a tenfold speed drop for every token, so it is easier to tolerate; but peak VRAM often occurs during VAE decoding, where offloading does not help. LLM decoding reads the weights once per token; diffusion reads them once per step for dozens of steps. The calculation is different.

When VRAM is insufficient, ComfyUI automatically places weights in RAM and loads them in blocks. Waiting seconds to tens of seconds longer per image is acceptable to most users. The official Wan 2.2 documentation's statement that “8GB VRAM can run the 5B model” is based on this native offloading (ComfyUI Wan 2.2 tutorial). Note that this means the 5B version with offloading; A14B is not covered by that threshold, and the two versions' VRAM figures must not be mixed.

What actually blocks video generation is often VAE decoding rather than weights: decoding dozens of latent frames into pixels at once can produce a VRAM peak higher than generation itself. At this stage weights are small and intermediate tensors large, so offloading weights is pointless. If VRAM runs out only at the final generation step, use tiled decoding, lower resolution or fewer frames, rather than adding RAM.

The advice for image and video users is also simple: extra RAM can make a previously impossible workflow run, at the cost of waiting longer per image. Cutting that wait still requires VRAM, and video generation's upper VRAM requirements are much higher than those of LLMs. See the capacity tiers in what models run on 16GB VRAM.