Quick Answer: For most people building a local-LLM rig in 2026, a used RTX 3090 (24 GB) is the best value — it fits 32B models at Q4 and costs a fraction of anything new. If you want the fastest single card available and can stomach the price, the RTX 5090 (32 GB) is the only consumer GPU with enough VRAM for 70B-class models (with some offload). On a tight budget, the RTX 5060 Ti 16 GB handles 7B–20B models well. The number that decides everything is VRAM, so size the card to the model, not the other way around.
The one rule: match VRAM to the model
Every other spec is secondary to how much the model weights take up in memory. At 4-bit (Q4) quantization — the sweet spot most local runners use — a model needs roughly 0.5–0.6 GB of VRAM per billion parameters, plus a little extra for the context window. Go over your VRAM budget and the model either won’t load or spills into system RAM and crawls.
| Model size | Q4 weights | Practical VRAM (with context) | Example cards |
|---|---|---|---|
| 7B | ~5 GB | 8 GB | RTX 5060 Ti / 4060 Ti |
| 13B | ~9 GB | 12 GB | RTX 5060 Ti 16 GB |
| 32B | ~21 GB | 24 GB | RTX 3090 / 4090 / 5090 |
| 70B | ~42 GB | 48 GB | 2× 24 GB, or 5090 + offload |
| 120B-class | ~70 GB | 96 GB+ | Multi-GPU or unified-memory box |
A 70B model at Q4 is about 42–44 GB — more than any single consumer card here. If 70B is your goal on one device, a high-bandwidth GPU with partial offload works but is slow; a 128 GB unified-memory mini PC or Mac is often the saner route.
Our top picks at a glance
| GPU | VRAM | Best for | Reality check |
|---|---|---|---|
| RTX 5090 | 32 GB | Fastest single card | Sells well over MSRP, often out of stock |
| Used RTX 3090 | 24 GB | Best value | Used-only, a few years old |
| RTX 4090 | 24 GB | Fast 24 GB, warranty | Pricey, also hard to find new |
| RTX 5060 Ti 16 GB | 16 GB | Best budget entry | Caps you at ~20B models |
A note on prices: GPU pricing in 2026 is volatile — the RTX 5090 has traded well above its MSRP and goes in and out of stock constantly. We quote capability and price class rather than a sticker number that’s wrong by the time you read it. Always check the live price, and keep a second in-stock card in mind.
1. RTX 5090 (32 GB) — Best Overall, If You Can Get One
NVIDIA GeForce RTX 5090 (32 GB)
- The only consumer card with 32 GB of VRAM — enough for 32B models with long context, and 70B Q4 with some offload.
- Highest memory bandwidth of any single card here, so tokens generate fast on anything that fits.
- The catch: it regularly sells above MSRP and sells out — have a backup pick ready.
The RTX 5090 is the fastest single card most people can buy and the only consumer GPU with 32 GB, which is exactly why it’s the pick when you want headroom. Running a workstation for a team? A free Amazon Business account unlocks VAT invoices and quantity pricing on GPUs and complete rigs. If the 5090 is sold out or gouged, the used 3090 below is the pragmatic fallback.
2. Used RTX 3090 (24 GB) — Best Value
NVIDIA GeForce RTX 3090 (24 GB, used)
- 24 GB of VRAM — enough for 32B models at Q4 — for a fraction of a new flagship.
- Discontinued new, so this is a used-market buy; prefer sellers with returns.
- Pair two of them for 48 GB and real 70B capability on a budget.
Pound for pound, the 3090 remains the local-LLM community’s favourite because of its 24 GB. It’s a few years old and used-only, but the VRAM is what matters, and two of them is the cheapest honest path to 70B.
3. RTX 4090 (24 GB) — Fast 24 GB With a Warranty
NVIDIA GeForce RTX 4090 (24 GB)
- Same 24 GB as the 3090 but faster, and available new with warranty where stock exists.
- Great middle ground if you want 32B performance without the 5090's price.
4. RTX 5060 Ti 16 GB — Best Budget Entry
NVIDIA GeForce RTX 5060 Ti (16 GB)
- 16 GB gets you 7B–20B models at good speed — ideal for a first local-LLM setup.
- Buy the 16 GB variant specifically; the 8 GB version is too small for anything but 7B.
How to choose
- Pick your target model first. Decide whether you want to run 7B, 32B, or 70B-class models, then read the VRAM table above and buy the card that clears it with a few GB to spare for context.
- Buy the VRAM, not the benchmark. A card that can’t load your model is useless no matter how fast it is. Capacity is the gate; speed is the tiebreaker.
- Have an in-stock backup. Flagship cards sell out and spike in price. If the 5090 isn’t available at a fair price, a used 3090 (or two) gets you most of the way for far less.
- Consider unified memory for the biggest models. If you want 70B+ on one box, a Strix Halo mini PC or Mac can hold bigger models than any single GPU — just slower.
The bottom line
For most people, a used RTX 3090 is the smartest local-LLM GPU in 2026: 24 GB of VRAM for 32B models at a fraction of flagship pricing. Step up to the RTX 5090 only if you want the fastest single card and can buy one at a fair price — and if you can’t, don’t overpay; the 3090 (or a pair) is the honest fallback. Starting small? The RTX 5060 Ti 16 GB is the budget on-ramp.