Qwen3.8-27B: The Local Model Most People Should Run in 2026

The short version: Qwen3.8-27B is the local model most people should try first in late 2026. It is a dense 27-billion-parameter vision model under a clean Apache 2.0 license, it fits a single 24GB GPU or a 32GB Mac at 4-bit, and independent testing says its quality barely drops at that quant. The catch is that it thinks for a very long time by default, sometimes tens of thousands of tokens before it answers, so the first thing you should learn is how to turn that down. Here is what it is, what the community found, and exactly what runs it.

We have not benchmarked this model first-hand. This draws on the model card, an independent quantization study, and owner reports, all linked below. Alibaba's own benchmark numbers are reported here as the maker's claims, not verified results.

What Alibaba says it is

Qwen3.8-27B shipped on August 14, 2026 as a dense, natively multimodal model (text, image, and video in) under Apache 2.0, which means you can use it commercially with no strings. Alibaba's claimed benchmarks put it near the top of the open-weight pack: Terminal-Bench 2.1 73.0, SWE-bench Pro 61.7, GPQA Diamond 89.2, LiveCodeBench v6 90.3. Artificial Analysis, an independent aggregator, scored it 52 on its Intelligence Index, high for a 27B. Treat the maker's per-benchmark figures as claims (there is no technical report for the 27B, only a blog post), but the independent score and the download numbers both point the same way: this became the default local model of the month, with millions of pulls across the BF16 and GGUF repositories.

Under the hood: mostly linear attention

The architecture is the interesting part, and it ties directly to why the model is cheap to run at long context. Its 64 layers follow a 3-to-1 pattern: three Gated DeltaNet layers (a form of linear attention) for every one full-attention layer, so 48 of the 64 layers use the cheap linear variant. That is what lets it carry a 262,144-token native context (extensible to a million with YaRN) without the KV cache exploding the way a classic transformer's would. It also ships a multi-token-prediction head for optional speculative decoding.

The quant ladder: which file fits your machine

Because it is dense, sizing is simple: the whole model is read every token, so pick the largest quant that fits with room for context. Verified GGUF sizes (Unsloth) and where they land:

The good news for the 24GB tier: an independent study by Quesma (Piotr Migdał) that spent thousands of dollars of GPU time comparing quants found that BF16, Q8_0, and Q4_K_M all score roughly the same, about 90% on GPQA Diamond and 75% on Terminal-Bench, while only the 1-bit builds fall apart. In other words, the 16.5GB Q4_K_M you run on a 3090 gives up very little to the full model. Confirm the fit for your exact card in our Can I run it? calculator and pick the file with the quant picker.

Real owner speeds

Reported tokens per second, all attributed, all from owners (not us):

• RTX 3090 / 4090 / 5090 at Q4_K_S, 4K context: roughly 40 / 46 / 75 tok/s in llama.cpp, dropping with longer context.
• Mac Studio (M3 Max, 64GB): about 40 tok/s (owner report).
• MacBook Pro M5 Max, 48GB: "Qwen3.8 27B runs at 30 tk/s," per one Hacker News commenter.
• Mac Studio, Q6_K with speculative decoding: "~24-27 tok/s generation between 0-16k context," another owner reports.

Speeds conflict a bit (one M3 Ultra owner measured only ~14 tok/s on Ollama Q4_K_M, about half the speed of the older Qwen3.6-27B), which usually points at runtime settings and the thinking overhead below rather than the hardware.

The overthinking problem (read this before you run it)

The single biggest complaint is not quality, it is verbosity. The model ships with reasoning set to xhigh by default, and it will churn through enormous chains of thought before answering. One owner running it on a 128GB box watched it sit "generating for 45 to 80+ minutes, churning through 20,000 to 55,000+ tokens of internal thought before writing a single line of code." Artificial Analysis measured it emitting about 3.3 times the median number of output tokens of other models on its index. The fix is one setting: drop reasoning_effort to medium or low. As one commenter put it, "switching from xhigh to medium roughly halved the time it took." Do that first.

One more launch wrinkle worth knowing: the initial chat template nested empty thinking blocks and truncated multi-turn history, which broke some tool-calling and agent setups. It has been patched in the runtimes, but if you pulled it early, update your template.

What the community likes, and who is still waiting

The consensus is positive: strong coding and agentic behavior for the size, genuinely useful vision, and a permissive license. The recurring wish is for a smaller sibling; the Qwen3.8 open lineup jumped straight from this 27B to giant MoEs, and there is no 4B or 9B version, so laptop and phone users keep asking for one. A fair critique from the benchmark-skeptical camp: a newer MoE like Ornith-1.5 posts similar scores at far higher speed on the same hardware, so the dense 27B is not automatically the fastest path to a given quality, just the most broadly compatible one.

Runtime support and the settings that matter

Day-one support is broad: Ollama (already past a million pulls), LM Studio, llama.cpp, MLX on Apple Silicon, and vLLM with NVFP4 on NVIDIA. Use the maker's recommended sampling: in thinking mode, temperature 1.0, top-p 0.95, top-k 20; in non-thinking mode, temperature 0.7, top-p 0.8. Our sampling settings guide explains what each dial does if you want to tune further.

Our take

If you have a 24GB GPU or a 32GB-plus Mac and you want one capable local model, this is the safe default: Apache 2.0, multimodal, near-lossless at 4-bit, and supported everywhere. Turn the reasoning depth down to medium, run the Q4_K_M, and it will handle most coding and assistant work locally. If you are on a smaller machine, wait for a smaller Qwen or reach for a sparse MoE instead. Size it against your hardware first.

Sources and how we researched this

• Model card, architecture, and claimed benchmarks: Qwen/Qwen3.8-27B on Hugging Face (the 27B has no separate technical report).
• Independent quantization quality: Quesma (Piotr Migdał), "Benchmarking Qwen3.8 27B quantizations"; independent intelligence score from Artificial Analysis.
• Quant file sizes from the Unsloth and mlx-community repositories; owner speeds and the overthinking and template issues from Hacker News and the Hugging Face discussions. We have not tested this model first-hand.

Related: The attention rebuild (linear attention) · The plain-English quantization guide · Sampling settings explained · Which Mac for local LLMs