RTX 5090 vs RTX 4090 for Local LLMs: What 32GB and 78% More Bandwidth Really Buy

Here is the matchup a lot of local-LLM builders are stuck on in 2026. The RTX 4090 is a discontinued card now, used-only, and it still costs around two thousand dollars because AI buyers will not let go of its 24GB. The RTX 5090 is newer, has 32GB and much more memory bandwidth, and sells for well above its paper price when you can find one at all. So the question is not "which is faster." The 5090 is faster. The question is whether the newer card earns the roughly 1.5x price jump for the models you run at home day to day.

We have not benchmarked these cards ourselves. What follows is built from NVIDIA's own Blackwell architecture whitepaper, owner-measured token rates from llama.cpp benchmarkers, and a read through the r/LocalLLaMA threads where people who own one (or both) compare notes. Every number is sourced at the end.

The spec sheet, side by side

Both cards are built on the same TSMC 4N node. The gap is memory and generation, not process.

Sources: NVIDIA RTX Blackwell GPU Architecture whitepaper (Appendix A) and both NVIDIA product pages. TDP and bandwidth are firm official specs.

The one number to keep in your head is bandwidth. The 5090 moves 1,792 GB/s versus the 4090's 1,008 GB/s. That is 1.78x, about 78% more. As we explain in Bandwidth, Not TFLOPS, memory bandwidth, not raw compute, is what sets how fast a local model spits out tokens. That 1.78x is the ceiling on how much faster the 5090 can generate text, and real gains land a bit below it.

What fits: 24GB vs 32GB

The extra 8GB does not unlock a whole new class of model on its own. What it removes is the squeeze between weights and context that makes 24GB frustrating on 30-to-32B models.

Weight sizes from Hardware Corner author measurements. Fit assumes room left for KV-cache/context.

The pivotal row is 32B dense at Q4. Its weights are about 18.6GB. On the 4090 that leaves only four or five gigabytes for the KV cache, so you either cap your context short or lean on KV-cache quantization tricks to stretch it. On the 5090 the same model runs with a generous context window and no gymnastics. The genuine 5090-only wins are narrower than the marketing implies: a 30-to-34B model at Q8, a 13B-or-larger model at full FP16, and 32B at Q4 with real context. Useful, but specific.

One myth to kill: neither card runs a 70B at Q4 without help. At roughly 38-40GB, a 70B overflows 24GB and 32GB alike, so both cards fall back on slow CPU/RAM offload or a second GPU. If a 70B is your target, capacity is the whole game, and two used 3090s (48GB combined) or a big unified-memory box matter more than either card here. Our guide on how much VRAM a 70B really needs walks through the math.

How much faster is the 5090, really?

For single-user chat (one conversation at a time, which is what most people run at home), token generation is bandwidth-bound. So the 5090's advantage should track its 1.78x bandwidth edge, minus overhead. Here is what owners measured on each card, all Q4, all llama.cpp.

5090 figures: Hardware Corner (Allan Witt), llama.cpp, batch 1. 4090 figures: LocalScore crowdsourced medians and owner recipe repos. Different testers, so treat as ballpark.

Those columns come from different people on different setups, so do not divide them blindly. The cleanest apples-to-apples row is the 32B dense case (both single-user, both the same model class), and it lands at roughly 1.4 to 1.6x, right on the bandwidth-limited expectation. Across models that fit both cards, plan on the 5090 generating tokens about 1.3 to 1.6x faster for solo chat. Not the 2-to-3x you will see quoted.

Where do the big multipliers come from? Batched server throughput. When you serve many simultaneous requests through vLLM, the 5090 pulls ahead 2x or more, helped by its 32GB allowing bigger batches. That is a real advantage if you are hosting a model for a team, but it is a different measurement from the one-user-at-a-time speed you feel in a chat window.

A methodology note, since it matters: the crowdsourced LocalScore page for the 5090 shows decode numbers lower than the 4090, which is physically impossible with 78% more bandwidth. That entry is a throttled or misconfigured single submission dragging the median down, so we did not use it for the 5090 and pulled author-controlled benchmarks instead. If you see a chart claiming the 5090 is slower than a 4090 at inference, that is the outlier you are looking at.

Prefill and the FP4 question

The 5090 pulls further ahead on prefill (prompt processing), which is compute-bound rather than bandwidth-bound. Owner benchmarks put its prompt-processing edge closer to 1.5-1.8x, and it grows when the new FP4 kernels kick in. If your workload is long prompts, RAG, or agentic coding where time-to-first-token dominates, that is the 5090's strongest felt advantage, more than raw chat speed.

FP4 is the 5090's headline new trick (the 4090 has no FP4 path at all), and it is worth being precise about. As of mid-2026, NVFP4 kernels are merged in llama.cpp and working, and they measurably speed up prefill (2 to 3x on prompt processing in the best cases). They do close to nothing for decode, because a 4-bit weight is the same size in memory whether or not the tensor cores can crunch it faster, and decode is waiting on memory, not math. vLLM and TensorRT-LLM support FP4 checkpoints; the click-to-run front-ends (Ollama, LM Studio) had not fully wired it up yet. Owners are split on how much it matters, which we get into below.

The price and power reality

This is where the decision turns, because the sticker prices are nothing like the MSRPs.

Prices are ranges across StockMaid, eBay ASP, r/hardwareswap and retail snapshots, mid-2026. They move; check the live price index.

So the real gap most buyers face is roughly a $2,000-$2,300 used 4090 against a $3,000-plus 5090, and the 5090 side should carry a $150-$250 power-supply upgrade too, since 575W wants a 1000W-class unit. Note also that the used 5090 is no bargain right now: it averages around $4,000, basically the same as new. And a "new" old-stock 4090 at $2,500-$3,700 is worse value than a clean used one.

What owners are saying

The r/LocalLLaMA consensus is more skeptical than the spec sheet. u/Kirys79 rented a 5090, ran a benchmark suite, and described the mechanism better than most reviews:

The 5090 is "only" 50% faster in inference than the 4090 (a much better gain than it got in gaming). I've noticed that the inference gains are almost proportional to the ram speed till the speed is <1000 GB/s then the gain is reduced. (r/LocalLLaMA)

The sharpest for-balance take came from u/Nicholas_Matt_Quail, who owns a 5090, a 4090 and a 4080:

even though my speeds on 5090 are around 20-30 t/s higher than 4090, it's not such a revolution in terms of VRAM that I'd move to higher models. Right now, I'm using Mistral 24B and Qwen/QWQ/Gemma 27-32B. (r/LocalLLaMA)

The value counter-position comes up in nearly every thread. As u/armadeallo put it, "3090s still the king of price performance/value with the big caveat only available used now" (r/LocalLLaMA). In the same thread, u/AppearanceHeavy6724 explained why stacking cards does not buy speed: "2x3090 has exactly same bandwidth as single 3090. the only rare case when 2x3090 will be faster is MoE with 2 experts active" (r/LocalLLaMA). More VRAM lets you hold a bigger model; it does not make each token faster.

On FP4, owners are split. u/Ok_Run_1823 sees the upside: "the RTX 5090's FP4 throughput is roughly double its FP8 rate... yet it's often ignored in LLM-inference threads" (r/LocalLLaMA). u/Dependent-Passion-54 is more guarded on quality: "Fp4 only runs on Nvidia transformer engine and it does have some degradation... I rather run 12B model in FP8 or int8/bf16 than 30B model in fp4" (r/LocalLLaMA). And the availability gripe was universal at launch. u/sleepy_roger, who scored one, admitted: "I got lucky with a Bestbuy drop on release day" (r/LocalLLaMA).

Why decode tracks bandwidth, not TFLOPs

The reason owner speed-ups keep landing near the 1.78x bandwidth ratio, and not the 2.5x compute number on NVIDIA's slides, is structural. During autoregressive decoding a model re-reads all of its weights from VRAM to produce each single token, which is a low arithmetic-intensity operation: lots of memory traffic, little math. The 2024 survey LLM Inference Unveiled: Survey and Roofline Model Insights (arXiv:2402.16363) applies the roofline model to show exactly this, decode sits in the memory-bound region while prefill is compute-bound. That is the textbook version of what u/Kirys79 saw empirically: below about 1 TB/s, speed scales with bandwidth. It is also why FP4 helps prefill and not decode.

So which should you buy?

Get a used RTX 4090 (or used 3090s) if your budget is the constraint and you mostly run models up to 24B, plus 30B-class MoE. You will pay around $2,000, reuse your 850W PSU, and give up perhaps 30-50% single-user speed you may never miss. The used 3090 remains the value king for pure capacity per dollar.

Get an RTX 5090 if you specifically need one of its real wins: a 30-to-34B dense model at Q8 or at Q4 with a big context window, a 13B-plus model at full FP16, the fastest single-GPU chat you can buy, or heavy prefill work (RAG, long prompts, agents) where its Blackwell tensor cores and FP4 prefill pull ahead. You will pay $3,000-plus, probably a new power supply, and you will hunt for stock.

Do not buy either expecting to run a 70B at Q4 natively. That needs 48GB-plus, which is a two-card or unified-memory decision, not a 4090-versus-5090 one. If you are cross-shopping the prosumer route, see RTX 5090 vs RTX Pro 6000.

Not sure a given model fits your card? Run it through our Can I run it? calculator, and use the quant picker to choose the right GGUF file for your VRAM.

Sources and how we researched this

• Specs: NVIDIA RTX Blackwell GPU Architecture whitepaper (Appendix A spec table) and NVIDIA's 5090 and 4090 product pages.
• Owner token rates: Hardware Corner (5090, llama.cpp), LocalScore crowdsourced medians (4090), and owner recipe repos for 32B-class on the 4090.
• FP4/NVFP4 status: merged llama.cpp pull requests (generic NVFP4 kernel, April 2026) and the Blackwell tensor-core path that followed.
• Pricing: StockMaid 5090 tracker, eBay average selling prices, r/hardwareswap sentiment, and retail snapshots, mid-2026.
• Owner sentiment: linked r/LocalLLaMA threads above.
• Mechanism: LLM Inference Unveiled: Survey and Roofline Model Insights (arXiv:2402.16363).
• We have not tested these cards first-hand; figures are owner-measured and vendor-published, cited inline.

Related: Bandwidth, Not TFLOPS · Is the RTX 5090 worth it? · GGUF vs GPTQ vs AWQ · Mixture-of-Experts, explained · Hardware cheat-sheet

Shopping now: a new RTX 5090 on Amazon, or a used RTX 4090 on eBay. Prices move fast, so cross-check both before you commit.