Local LLM GPU Ranking

What is VRAM, and how much do you need for local LLMs?

VRAM is the fast memory on a graphics card. Local LLM inference needs space for model weights, the context (KV) cache and runtime buffers. Quantization can reduce weight memory, while longer prompts and more simultaneous users need additional memory. Once a model fits, generation speed comes down mostly to memory bandwidth.

• Compare model VRAM requirements

How to plan a local LLM deployment

Choose a model and quantization, set the context length, and check both GPU memory and system RAM. CPU or expert offloading can make a large model fit with less VRAM, but it depends on RAM bandwidth and can be much slower. Check CUDA, ROCm or Metal support before buying hardware.

Read the rankings and price data

Compare VRAM capacity first, then bandwidth, power and software support. Our prices are median asking prices, not completed sales; sample counts, observation dates and stale-price labels help you judge their usefulness. Generation speeds on each GPU page are calibrated against public llama.cpp benchmarks.

• Price sources and calculation methods