Software & Tools

Honest guides to the software that runs local LLMs — inference engines, GUIs, quantization, serving, and RAG tooling. Mostly free, open-source tools, covered because they matter to the niche, not for affiliate links.

Software & Tools

Local Embedding Models Explained: The Other Model Your RAG Setup Needs

LLM Sampling Settings Explained: Temperature, Top-P, Top-K, and Min-P

How Much RAM Do You Need to Run a Local LLM in 2026?

The M5 Mac and Local LLMs: Apple's New Matmul Hardware Finally Attacks Prompt Processing

The Best Open LLM You Can Actually Run Right Now, by VRAM Tier (August 2026)

What "Open Weights" Lets You Do: The 2026 Model License Map, Read From the Actual Texts

The Attention Rebuild: How 2026's Open Models Made 1M Context Fit on Machines You Own

Speculative Decoding, Explained: The Free Speed Toggle Your Local LLM Is Probably Not Using

Bandwidth, Not TFLOPS: What Sets Your Local LLM Speed (and Why the Newest Card Isn't Always Fastest)

GPT-5.6 is here, and you can't run it. Here's what you can run instead.

Serving a Local LLM as an API: From Ollama's Endpoint to vLLM Throughput (and When to Rent Instead)

RAG on a Local LLM, Explained: Give Your Model Your Documents Without Drowning in Context

Prompt Processing vs Generation: Why Your Box Is Fast at One and Slow at the Other

The KV Cache, Explained: Why Long Context Eats Your VRAM (and How to Fit More)

How Much VRAM Do You Actually Need to Run a 70B Model Locally?

Ollama vs LM Studio vs llama.cpp: Which Local LLM Runtime Should You Actually Use?

Mixture-of-Experts (MoE), Explained: Why “Active Parameters” Decide What Runs on Your Machine

Your Local AI Model Folder Is a Mess: Taming a Multi-Terabyte Model Hoard on Apple Silicon

GGUF vs GPTQ vs AWQ: The Plain-English Guide to LLM Quantization (and Which One to Pick)

Get the Vetted Consumer newsletter