📎 Run a site or newsletter? Use the Cite or Embed buttons just above to link to this tool or embed the live version on your own page, free, no signup, just keep the credit.
How to read the table
It balances three things at once: quality (more bits = better), context (bigger files leave less room for the KV cache), and speed (bigger files stream slower). Tell it the context you need and it picks the highest-quality quant that fits, then shows the approximate tokens/sec each quant runs on your machine, so you can trade quality for speed deliberately. It also points you to the community-trusted GGUF makers (bartowski, unsloth) and the smaller I-quants you’ll see in their repos.
Every GGUF model ships in multiple quantization levels, same model, different precision, different file size. The trade is simple: more bits = better quality = bigger file = less room left for context. This tool does the arithmetic for your exact machine: file size per quant, then whatever memory remains becomes your context budget (the KV cache eats it per token).
The recommendation logic is the community consensus from our quantization guide: take the highest quant that still leaves ≥8k of context. Q6/Q5 are near-lossless, Q4_K_M is the sweet spot, and below Q3 quality falls off fast, if you're forced down there, you usually want a smaller model instead (a bigger model at Q4 beats a smaller one at Q8, but a Q2 of anything beats very little).
Real limits
File sizes are computed from bits-per-weight, not scraped from Hugging Face, real files vary a little by quantizer version (K-quants vs I-quants, imatrix variants). The KV-cache math assumes a GQA-typical architecture; exotic models differ. And max context here is what fits, models also have their own context limits, and quality at extreme context is its own story. Treat the numbers as a reliable guide, not a contract.
The tool family
Shopping rather than downloading? Can I run it? finds hardware that fits a model. Wondering if you should buy hardware at all? The cost calculator compares buying vs renting vs the API.
Worked example: which Qwen3.8-27B file to download
Take the most-downloaded dense model of the moment, Qwen3.8-27B, and the Unsloth GGUF ladder for it. File size decides which machine each fits; independent testing decides how much quality each gives up.
| Quant | File size | Fits comfortably on | Quality, independently measured |
|---|---|---|---|
| IQ1_M | 6.7GB | any 8GB card | Collapses in testing. Avoid. |
| UD-Q2_K_XL | 9.8GB | 12GB to 16GB card | About 88% on GPQA Diamond and 70% on Terminal-Bench: the usable floor |
| Q4_K_M | 16.5GB | 24GB GPU, 32GB Mac | About 90% and 75%: the same as Q8 and BF16 within noise |
| Q6_K | 22GB | 32GB Mac, 48GB card | Indistinguishable from Q4_K_M in testing |
| Q8_0 | 29GB | 48GB card, 64GB Mac | Reference quality |
| BF16 | 54GB | 64GB-plus Mac, two 48GB cards | Full precision, no measured gain over Q8 |
The quality column comes from Quesma's benchmark of every Qwen3.8-27B quant, which found BF16, Q8_0 and Q4_K_M scoring the same within noise on GPQA Diamond and Terminal-Bench, while only the 1-bit builds fell apart. That pattern holds across most modern models and is the whole case for Q4_K_M as the default: a 24GB card runs the model at near-full quality. We have not run these tests ourselves; the picker links the evidence per model where it exists.
Which GGUF quant should you download?
Q4_K_M unless you have a reason not to. Step up to Q5_K_M or Q6_K when the file plus your context still leaves about 30 percent headroom, and to Q8_0 when memory is not a constraint, since it is close to lossless. Step down to a 2-bit or 3-bit build only when nothing else fits, and never to 1-bit for anything that has to be correct. The reasoning, and what GPTQ and AWQ are for, is in the plain-English quantization guide.
What do K_M, K_S, IQ and UD mean?
K-quants keep the most sensitive tensors at higher precision than the rest; M (medium) and S (small) are how much of that extra precision you keep. IQ quants use an importance matrix calibrated on real text, which is why IQ2 and IQ3 files hold up better than a plain 2-bit or 3-bit would. UD is Unsloth's dynamic scheme, choosing the bit width layer by layer, which is why a UD-Q4_K_XL is a little bigger than a Q4_K_M and usually a little better. When two files are within a gigabyte of each other, the picker prefers the one with the better measured quality.
Does a smaller quant run faster?
Yes, and roughly in proportion. Token generation is bound by memory bandwidth, so every token costs about one full read of the weights; a 16.5GB Q4 file streams almost twice as fast as a 29GB Q8 file on the same card. Prompt processing barely changes, because it is compute-bound. If you want more speed without dropping precision, speculative decoding is the free lever.
How much context will you have left?
Usable memory, minus the file, minus about 1.5GB of runtime overhead, is your cache budget. For Qwen3.8-27B's shape that is about 0.2GB per 1,000 tokens at 16-bit, so a 24GB card running the 16.5GB Q4_K_M has room for roughly 25,000 tokens; quantizing the cache to 8-bit doubles that. The picker reports this number for every row. Why the cache costs what it does is in the KV cache, explained.
Where the numbers come from
File sizes are read from the actual repositories (Unsloth, bartowski, the model authors), not estimated. Speeds are bandwidth ceilings unless an owner has published a measurement for that machine and model class, in which case the picker shows the measurement and its source. The engine is the same one behind Can I run it? and the public API, so answers never disagree across the site. Method and named sources are on the methodology page.