A100 40GB PCIe Price & Used Prices: Which Local LLMs Can It Run?

• Memory40 GB · HBM2
• Bandwidth1,555 GB/s
• FP16 / BF16312 TFLOPS
• INT8624 TOPS

Bus width

5,120-bit

Architecture

Ampere (GA100)

Cores

6,912

Tensor cores

432

Ecosystem

CUDA

Interface

PCIe 4.0 x16

NVLink

600 GB/s

Power

250 W

Power connector

8-pin EPS

Slots

2

Length

267 mm

Type

Data center

Launch date

Jun 22, 2020

Model	Size	Cards	Total price	Total VRAM	Speed t/s	Total power
Qwen3.8 Flash Next176B (6B active)	111.3 GB	4 cards	$31,000	160 GB	121.7	1,000 W
DeepSeek V4 Flash284B (13B active)	155.1 GB	8 cards	$62,000	320 GB	93.2	2,000 W
Hy3295B (21B active)	182.2 GB	8 cards	$62,000	320 GB	50	2,000 W
GLM 5.3 Flash320B (18B active)	199.7 GB	8 cards	$62,000	320 GB	70	2,000 W

Full calculation rules are on the Methodology page.

1. 1

   Watch out for converted SXM4 cards

   On the used market, some sellers mount server-only SXM4 modules on third-party adapter boards and sell them as PCIe cards, often with “SXM4 converted” in the listing title. Cooling, power delivery and stability on these cards depend entirely on how well the adapter board is built, and there is no factory warranty. To tell them apart, install the card and check the model name in nvidia-smi: the factory PCIe version shows “A100-PCIE-40GB”, while an SXM4 module shows “A100-SXM4-40GB”. In lspci, the device IDs are 20F1 and 20B0 respectively.

2. 2

   Power needs an adapter cable, and cooling is up to you

   The power input is an 8-pin CPU connector (EPS), which has a different plug from the 8-pin PCIe connector graphics cards usually use. With a regular power supply, you need an adapter cable that combines two PCIe 8-pin leads into one CPU 8-pin. The card has no fan, just a very dense heatsink designed to be cooled by powerful server fans. In a regular case, you need to add a high-static-pressure fan or switch to a water block, because ordinary case fans can't push air through it.

3. 3

   When MIG is useful

   MIG is the A100's partitioning feature: it splits one card into several “mini GPUs” whose memory and compute don't affect each other, which suits several people sharing a card. Practical splits are two 20GB slices, each running a 4-bit 27B, or three 10GB slices, each running a 4-bit 12B. Splitting all the way to seven slices leaves only 5GB each, enough only for small models like Gemma 4 E4B at 2-bit. If one person is running one model, you won't need it.

Spec	A100 40GB PCIe	A100 80GB PCIe	CMP 170HX 40GB	L40S
Memory	40 GB	80 GB100% more	40 GBSame	48 GB20% more
Bandwidth	1,555 GB/s	1,935 GB/s24% higher	1,560 GB/sSame	864 GB/s44% lower
BF16 / FP16 compute	312 TFLOPS	312 TFLOPSSame	202 TFLOPS35% lower	362 TFLOPS16% higher
INT8 compute	624 TOPS	624 TOPSSame	404 TOPS35% lower	733 TOPS17% higher
Qwen3.8 27B speed (2-bit)	101.6 t/s	122.2 t/s20% faster	101.9 t/sSame	60.3 t/s41% slower
Largest dense model (4-bit)	65B	136B109% more	65BSame	79B22% more
Power	250 W	300 W20% higher	250 WSame	350 W40% higher
GPU interconnect	NVLink 600 GB/s	NVLink 600 GB/s	Not supported	Not supported
Interface	PCIe 4.0 x16	PCIe 4.0 x16	PCIe 2.0 x4	PCIe 4.0 x16
Released	2020-06	2021-06	2021-09	2023-08
eBay median	$7,750Buy A100 40GB PCIe	$18,999Buy A100 80GB PCIe	$2,934Buy CMP 170HX 40GB	$8,974Buy L40S

As an eBay Partner, VRAMGlass may be compensated if you buy through the eBay links on this page. The price you pay stays the same.