📎 Run a site or newsletter? Use the Cite or Embed buttons just above to link to this tool or embed the live version on your own page, free, no signup, just keep the credit.
One step earlier: not sure you should buy hardware at all? Our cost calculator compares buying vs renting cloud GPUs vs just paying for an API, with break-even math for your usage.
Two ways to use it: leave "Your machine" empty to shop across everything we track, or pick the hardware you already own (or enter its memory) to get a personal verdict, including, when it doesn't fit, the exact quant, context, or KV-cache change that would make it fit.
How the estimate works
The tool uses the same math from our guides, shown in the open because that's the point of this site. A model's memory cost has three parts:
- Weights, parameters × bits-per-weight ÷ 8. A 70B model at Q4_K_M (~4.8 bits/weight) is about 42 GB. Quantization choices are covered in our plain-English quantization guide.
- KV cache, grows with every token of context. We assume a GQA-typical attention shape and an FP16 cache; the KV-precision selector in the tool shows exactly what a Q8 or Q4 cache saves. Full math in The KV cache, explained.
- Overhead, a flat ~1.5 GB buffer for the runtime and activations.
For Mixture-of-Experts models, memory follows total parameters but speed follows active parameters, that's why a 120B MoE can be fast on a box that would crawl on a dense 70B. The one-line rule: buy memory for the total, expect speed from the active (MoE, explained).
The "gen ceiling" column is memory bandwidth ÷ bytes streamed per token, a theoretical upper bound from the fact that token generation is bandwidth-bound, not compute-bound (why that is). Real speeds come in below it.
Real limits
These are estimates, not lab measurements. Real usage varies by runtime (llama.cpp vs vLLM vs MLX), KV-cache precision, batch settings, and model architecture. Unified-memory machines share RAM with the OS, so we subtract an 8 GB reserve; discrete GPUs lose ~1 GB to the desktop. When a result says "tight fit," believe it, within 10% of capacity means long context or background apps will push you over. Hardware listings come from our methodology; affiliate links never influence what appears or how it ranks.
Worked example: a 70B model on a 24GB card, and what fits instead
Here is the exact arithmetic the calculator runs, for the question we get most. Take Llama 3.3 70B at Q4_K_M on a used RTX 3090 (24GB of VRAM) with 8,192 tokens of context.
| Line item | How it is computed | Result |
|---|---|---|
| Weights | 70B parameters × 4.8 bits per weight ÷ 8 | 42.0GB |
| KV cache at 8K context | 2 × 80 layers × 8 KV heads × 128 dims × 2 bytes × 8,192 tokens | 2.7GB |
| Runtime overhead | fixed allowance | 1.5GB |
| Total needed | 46.2GB | |
| Usable on a 24GB card | 24GB minus a 1GB reserve | 23GB |
| Verdict | 46.2GB needed against 23GB available | Does not fit |
No quant rescues this on one 24GB card: even a 2-bit build of a 70B is around 26GB before the cache. The real answers are a 48GB-plus machine, two cards, or a different model. Run the same numbers for a sparse Mixture-of-Experts model on the same card and the picture flips:
| Qwen3-30B-A3B on the RTX 3090 | How it is computed | Result |
|---|---|---|
| Weights | 30.5B × 4.8 ÷ 8 | 18.3GB |
| KV cache at 8K context | 2 × 48 layers × 8 KV heads × 128 × 2 bytes × 8,192 | 1.6GB |
| Runtime overhead | 1.5GB | |
| Total needed | 21.4GB of 23GB usable | |
| Speed ceiling | 936 GB/s ÷ (3.3B active × 4.8 ÷ 8, plus the 1.6GB cache) | about 260 tok/s theoretical; owners measure roughly 110 to 120 |
| Verdict | Fits, and it is fast |
That single comparison is why the move to sparse models changed the buying math: a 24GB card that cannot touch a dense 70B runs a 30B-A3B at over a hundred tokens a second. For the dense 70B, a 64GB Mac mini M4 Pro fits it (56GB usable after the 8GB system reserve) but the ceiling is 273 GB/s divided by 44.7GB streamed per token, about 6 tokens per second, which owners confirm at 4 to 7.5. That is the difference between "loads" and "usable", and it is exactly what the calculator's verdict line reports. Our guide to the cheapest way to run a 70B walks through the alternatives.
How much VRAM do you need to run a 70B model?
About 46GB at Q4 with 8K of context, per the arithmetic above. A single 48GB card is tight, 64GB is comfortable, and a 24GB card cannot do it without spilling to system RAM, where owners report about 2 tokens per second. The full breakdown, quant by quant, is in how much VRAM a 70B needs.
Can you run a local LLM on 16GB of RAM?
Yes, for 7B to 8B models at Q4 with room for context, and up to about 13B to 14B with short context. On a 16GB Mac the GPU only gets about 11GB by default, so a 30B-class MoE does not fit there. The tier-by-tier answer, from 16GB to 512GB, is in how much RAM a local LLM needs.
Why does context length change whether a model fits?
Because the KV cache grows in a straight line with tokens. The 70B above costs 2.7GB at 8K and 10.7GB at 32K, four times as much, while the weights do not change. Quantizing the cache to 8-bit halves that. The calculator's context field is not cosmetic; it is often the difference between fitting and not. The KV cache, explained covers the mechanism.
VRAM or unified memory: which number do you enter?
For a discrete card, enter its VRAM. For a Mac or a Strix Halo mini PC, enter the total unified memory; the calculator subtracts an 8GB system reserve because macOS only wires about 70 to 75 percent of installed memory to the GPU by default (you can raise that with sudo sysctl iogpu.wired_limit_mb). Why the two kinds of memory behave differently is in unified memory, explained, and the current Macs by memory tier are in which Mac for local LLMs.
Where the numbers come from
Weights use 4.8 bits per weight for Q4_K_M and 4.25 for models shipped in native MXFP4, matching the file sizes on Hugging Face. Layer counts and KV head counts come from the model cards. The tokens-per-second column is a bandwidth ceiling, and wherever an owner has published a measurement for that model class on that machine (Hardware Corner, ServeTheHome, Level1Techs, the llama.cpp benchmark threads), the tool shows the measured figure with its source instead. The same engine powers our methodology page, the quant picker, and the public MCP server and API, so an AI agent asking the same question gets the same answer as this page. We have not benchmarked these machines ourselves, and we say so wherever a number is a ceiling rather than a measurement.