Kolibri 1: Aleph Alpha's 78B Open Model Reads Only 3.5B Per Token. What It Takes to Run It

Aleph Alpha released Kolibri 1 on 3 October 2026: a 78.1-billion-parameter mixture-of-experts model that reads only 3.46 billion parameters per token, trained for German and English, with a 1M-token context and a plain Apache 2.0 licence. The launch thread on Hacker News passed 700 points. For local runners it is an unusual shape: big enough on disk to need 48–64GB of memory at 4-bit, light enough per token to generate quickly once it fits, and built so that long context costs very little memory. The catch is software. A week after release, no mainstream local runtime loads it without a patch. Here is what Aleph Alpha claims, what the architecture does, what owners have measured, and what it takes to run.

What Aleph Alpha says

The model card and the technical report give the shape: 50 transformer blocks, 384 routed experts per layer with 6 chosen per token plus one shared expert, a 128,000-token vocabulary, and a native context of 262,144 tokens that the company says it has validated to 1,048,576. Training ran on 24 trillion tokens across pre-training, mid-training and long-context extension, with German making up more than 20% of the mix; the report says post-training used reinforcement learning on more than 1.2 million internally curated tasks. The official weights ship in FP8 at about 78GB, and the card's stated minimum hardware is data-centre class: two A100 80GB cards, two H100s, or one H200, B200 or B300.

The headline claim, repeated in the launch blog, is that Kolibri "sits on the Pareto frontier for quality versus serving cost, for both English and German." In Aleph Alpha's own evaluation table, all models run through its eval-framework with Kolibri at high reasoning effort, the results read like this:

So on its maker's harness, Kolibri edges the best small-active MoE models and loses clearly to a dense 27B model that reads about eight times as many parameters per token. That last row did not escape Hacker News: amoshebb pointed out that "Qwen3.8 27B beats Kolibri 79.9 vs 70.8 in German in Kolibri's harness on Kolibri's benchmark." Inside the table, Kolibri's clearest strengths are grounding and abstention. Its non-hallucination rate on AA-Omniscience is 44.0 against 11.1 for Qwen3.5 35B-A3B, and it scores 38.1 on the Tau3 banking agent test, where every other small-active model sits between 5 and 16. Its weakest spots are multi-turn tool calling (BFCL v4 multi-turn 47.5, against 58–65 for most peers) and fact-checking inside retrieved text (34 on RGB error correction, last in the table). These are self-reported numbers; no independent leaderboard had published results when we wrote this.

The research behind it, and why a local runner should care

Most layers only look back 512 tokens. Four of every five blocks use sliding-window attention over the previous 512 tokens; every fifth block attends to the whole context. The report says the hybrid "supports sequences four times as long" as a full-attention model at matched training compute, and the idea follows Longformer-style windowed attention. For memory, the consequence is large. Only the 10 full-attention layers keep a cache that grows with context. Using the published dimensions (4 KV heads of 128 dimensions), that is about 20KB per token at 16-bit, or roughly 5.4GB for the full 262,144-token context and half that at 8-bit. The 40 windowed layers hold a fixed cache of about 42MB. A conventional model with this shape and full attention everywhere would need about 27GB at 262K. That is our arithmetic, and the community has confirmed the effect: the Hob-forge GGUF page reports serving the full 262,144-token context with only 6.4GB of VRAM used on a 12GB RTX 5070, with the experts kept in system RAM. (Our KV cache explainer covers why this usually dominates long-context memory.)

Experts are balanced without a tuning knob. Kolibri spreads tokens across its 384 experts with Exact Quantile Balancing and Load-Error Injection, a method from Aleph Alpha researchers published as Neitemeier et al. (2026, arXiv:2609.28053). The paper computes exact global quantiles to keep every expert in use, and feeds local imbalance straight into the router's gradients. For a local runner the relevance is indirect but real: a model whose experts are all well trained degrades more gracefully when quantizers squeeze rarely used experts, and the Hob-forge page notes that a few "rarely-routed experts got partial coverage" during importance-matrix calibration.

Trained to say "I don't know". The report describes a reinforcement-learning environment built on the Merlin–Arthur protocol from Deiseroth et al. (arXiv:2512.11614), in which an adversary hides parts of the retrieved context and the model is rewarded for abstaining when the evidence is missing. That is the likely source of the strong abstention and non-hallucination scores above, and it matters if your use is answering questions over your own documents.

A German tokenizer. Against the GPT-5 tokenizer, the report says Kolibri's needs 11.2% fewer tokens on German web text, and places 65% of German token boundaries on morpheme boundaries against 47%. Fewer tokens means faster generation and more effective context for German text on the same hardware.

The runtime problem

Kolibri uses a new architecture, and support lags. Aleph Alpha's own route is vLLM with its aleph-alpha-inference plugin. As of this writing:

• llama.cpp: no upstream support; feature request 29922 is open. Community GGUFs work with a patch or a prebuilt fork, which the Hob-forge page documents, including what the patch implements and a check against a float64 reference.
• MLX: community conversions exist, but support is an open request in mlx-lm, so the quants require a fork.
• Ollama and LM Studio: will not load it yet. An MLX-based Kolibri port is open as Ollama pull request 18780.

Expect this to change within weeks, but if you want a one-command install today, Kolibri is not that model.

What owners report

Speed reports are consistent with a 3.5B-active model, and the measurements below come from the people who ran them:

One report we cannot place: a commenter in the model's llama.cpp thread claims 90 tok/s with a 4-bit GGUF on a single RTX 3090 using an experimental expert-caching branch. A 47GB file does not fit in 24GB, so most experts must live in system RAM; without the full setup, treat that figure as unconfirmed.

The quirks owners raise are consistent. martianvoid found it "spends way too many tokens on overthinking stuff even if it's able to catch the correct approach", so set reasoning effort to low or none for routine work. Two tool-calling problems are open on Hugging Face: a parser failure on multi-line code diffs in coding-agent harnesses, and a report that after 10–15 cached tool turns in an n8n workflow the model reasons but stops returning an answer. On the positive side, a run of tool-eval-bench scored 86 out of 100 and an independent BEAM evaluation on an RTX Pro 6000 found the full 262K window usable. One German user on HN tested it on bureaucracy and found it cited the superseded Telemediengesetz, a reminder that a June 2026 knowledge cutoff and 3.5B active parameters limit what it knows unaided. Its design target is retrieval over your documents, where it is meant to abstain rather than guess.

For context on the "sovereign" framing that dominated the thread: Cohere and Aleph Alpha signed a merger agreement in September, with the combined company to use the Cohere name. The Apache 2.0 weights are unaffected by that: once downloaded, they are yours.

The quant ladder

The Hob-forge page measured each GGUF against a full-precision reference on one prompt: Q8_0 and Q6_K agreed on 67 of 67 top predictions, Q4_K_M on 63 of 67. Myric reports that its 2.4-bit build passed the same 42 agentic coding tasks as a 4.6-bit build on a DGX Spark, but that prose quality "suffers first". The Q4_K_M is the sensible default where it fits; see the quantization guide for the trade-offs.

What runs it

At Q4_K_M, each token reads about 2.4GB of weights (the 3.46B active parameters plus the output layer), so bandwidth sets the ceiling the way our MoE explainer describes: roughly 105 tok/s on a 256GB/s Strix Halo box and 115 on a 273GB/s Mac mini M4 Pro or DGX Spark, before real-world losses.

• 128GB Strix Halo box (GMKtec EVO-X2, Framework Desktop): fits everything up to Q8_0 with room for the full context. The cheapest machine that runs Kolibri without compromise, via the patched llama.cpp and Vulkan or ROCm. No owner has published a Strix Halo number yet.
• 64GB Mac: Q4_K_M or MLX 4-bit fits with about 8GB to spare after macOS's reserve. See which Mac to buy.
• 48GB Mac: the 3-bit MLX build, around 70 tok/s on an M5 Pro per the table above.
• DGX Spark: FP8 through vLLM at about 48 tok/s, the easiest officially supported path on a desk. See the DGX Spark, according to owners.
• 24GB GPU (used RTX 3090): workable with experts in system RAM, which needs 64GB or more of DDR5; expect high-teens tok/s by Hob-forge's numbers.
• CPU only with 64GB+ RAM: about 12 tok/s, usable for batch work.

Check your own machine in Can I Run It, and pick the file size in the Quant Picker.

Verdict: worth downloading?

If you work in German, answer questions over your own documents, or need a long context on modest memory, yes, once you are comfortable building a patched llama.cpp or running vLLM. The windowed attention makes a 262K context affordable on hardware that cannot hold one for most models, and the abstention training suits retrieval. If you write English code or run multi-turn agents, our earlier pick still holds: Qwen3.8 27B beats Kolibri on Aleph Alpha's own table, and Kolibri's tool-calling bugs are not fixed yet. Everyone else should wait for upstream llama.cpp and Ollama support, which is when a 3.5B-active, 78B-capacity model with cheap long context becomes an easy recommendation for 64GB machines.

Sources and how we researched this

Creator documents: the Kolibri 1 model card, the technical report ("Kolibri: A Sovereign European Model on the Pareto Frontier") and the launch blog. Research: Neitemeier et al., Exact Quantile Balancing and Load-Error Injection for Mixture-of-Experts (2026); Deiseroth et al., Bounding Hallucinations: Merlin-Arthur Protocols for Mutual-Information Bounds in Language Models (2025). Quantizations and measurements: Hob-forge GGUF, Myric APEX GGUF, audreyt MLX 6-bit, here-be-dragons-ai MLX 3-bit, and the model's Hugging Face discussions. Runtime status: llama.cpp, mlx-lm, Ollama. Community: the Hacker News launch thread; merger context from SiliconANGLE. KV-cache sizes and speed ceilings are our arithmetic from the published architecture. We have not run the model; the site is non-commercial.

Related: MLX vs llama.cpp on a Mac · Strix Halo vs the Mac · The 2026 model licence map