Ten million downloads: the GGUF everyone runs

The lead: ten million downloads

What it is. The most interesting thing today isn’t a new model — it’s the adoption of the one you already run. Unsloth’s official GGUF of Qwen3.8-27B has crossed 10 million downloads and sits at 3,720 likes on HuggingFace — the #1 most-liked GGUF of all time, per Unsloth, and I verified the numbers directly against the HF API (10,675,683 downloads / 3,720 likes). That’s the single biggest empirical signal in the open-weight world right now: people aren’t just reading about this model, they’re running it. It’s also the honest answer to the “would a reader switch?” question — a 10M-download quant of a 27B open model is the incumbent, and it’s why the “open beats frontier” thread keeps being true in practice.

Models

No brand-new base model cleared the 200-download gate this cycle. The freshest releases are either already covered (K2-Horizon-7B, VibeVoice-ASR-Streaming-7B and MiniCPM5-2B were posted 07-08 Sep) or still at 0 downloads (the Sep 9 repos are mostly junk/duplicates). One frontier line worth noting: DeepSeek V4.1 Flash entered internal testing — new architecture, native multimodal, faster and cheaper — but it’s an API/gated model, not open weights, so it’s context here, not a headline. The real model story today is the Qwen3.8-27B adoption milestone below.

Quantizations

Qwen3.8-27B Unsloth GGUF — 10M downloads. This is the lead. Evidence: 10,675,683 downloads / 3,720 likes (verified via the HF API), reached in roughly 24 days, and it’s now the most-liked GGUF ever on HuggingFace. It’s the official Unsloth quant of the covered Qwen3.8-27B — a derivative/adoption milestone, not a new build, which is exactly why I’m reporting it as a status signal rather than a release. Verdict: Real deployment option. Ten million downloads is the clearest possible sign that an open 27B is what people actually deploy; for anyone weighing a local upgrade, this is the incumbent to beat.

One line. nvidia/Qwen3.8-Flash-Next-NVFP4 has grown to 26,302 downloads / 156 likes (I covered it as a one-liner on 06 Sep at 1,129 dl) — same build, bigger adoption, color rather than a new story.

Fine-tunes

Qwen-Drive-1.0-4B — from Qwen, Apache-2.0, a fine-tune of Qwen3.5-4B for autonomous-driving perception (3D perception, motion planning, visual question answering, image-text-to-text; arXiv 2609.00111). Evidence: 1,579 downloads / 114 likes since Aug 27. The real delta is the driving domain on the 4B base — it’s a genuinely new fine-tune, but it’s niche: a reader running a general local stack won’t swap for it. Verdict: Marginal — adopt only if you specifically want an open, on-device driving-perception VLM; otherwise skip.

Technical term of the day: the KV cache (and prefix caching)

What it is. Every token a model processes leaves a Key and Value vector in the cache. At generation, each new token attends over all previous tokens — so the cache grows with context length, and at long context it becomes the dominant memory consumer, often bigger than the weights themselves. That’s why a “750k context” model isn’t free: the cache for 750k tokens of a 27B+ model is gigabytes, and it competes with the weights for memory bandwidth.

Why it matters for local AI. This is the exact thread in today’s best signals. A community test of Qwen3.8-Flash-Next at 500k-750k context reports it “still works correctly” and holds ~50-70 tok/s with cache hits at 99% — the prefix cache is why: when many queries share a prefix (a long system prompt, a big context), the engine reuses the already-computed cache instead of recomputing it, so repeated work becomes nearly free. KV quantization (e.g. KV8 cache) shrinks the cache further — the new MLX-Serve 26.9.2 notes that “without KV8 Cache + M5Max, you might see above 150 Tok/s” on Apple Silicon, i.e. the cache, not the weights, is the limiter.

The trade-offs. It’s not free. The cache scales with context and eats memory you might want for the weights; quantized KV can silently degrade quality at very long context; prefix caching needs memory to hold the cached prefix and only helps when queries share it (a one-off long query gets no benefit); and per-sequence caches complicate batch serving. The practical rule: for long-context work, watch the KV footprint, turn on prefix caching for repeated system prompts, and treat a KV8/quantized cache as a memory trade — measure quality at your longest context before trusting it.

Harness improvements: the local stack keeps shipping

MLX-Serve 26.9.2 is the strongest harness item — a real local-serving release. It adds per-model settings, lets you proxy other engines/providers, and ships fixes, polish and speed; the maintainer’s read is that “Qwen Flash Next is the king of local models” for 96-128 GB machines, with KV8 cache + M5 Max pushing decode past 150 tok/s. A companion plugin shows a live decoding histogram and a MCP server lets agents control the fan on M5 Max before benchmarking. This is the kind of incremental local-serving win that never appears on HF trending.

DwarfStar shipped a Qwen3.8-Flash-Next update focused on 64 GB: tools-calling fixed in specific cases, prefill/decode improvements (charted from M3 Ultra), and memory handling for the PLE (positional-lookup embedding) table; the merged agent /hints feature makes the programmer a more active part of the loop. Hermes added Wingtips #66 sessions.auto_prune (auto-prune ended sessions so busy setups stay tidy), documented delegation defaults and output_schema, revamped Nous Portal, and kept Bots Mode group chats. Pi lets you pass a project directory to createAgentSession() so tools (read/grep/find/ls) resolve within that project and the session picks up its AGENTS.md and skills — a neat scoping feature.

Tooling color. A benchmark raced prime-agents + tgrep against ripgrep across the Linux kernel (96K files, 2.1 GB, 102 queries): tgrep 21.5 ms, ripgrep 86 ms, CPython re 5,298 ms — the framing is that “your agent isn’t slow, it’s grep-limited.” For video/image locally: a new ComfyUI sampler generates longer videos by processing in chunks (saving VRAM), with Gemma 4 writing continuous prompts to keep characters/backgrounds consistent; and ostrisai is training its own image-diffusion model from scratch on a single local GPU (experimental, no text encoder — a fun weekend project worth watching).

Context line. DeepSeek’s V4.1 Flash internal test and a ~60% API price cut are real but cloud/gated — good frontier color, not a headline, and it doesn’t change what you can run at home.

The takeaway

Today’s signal is adoption, not novelty. The headline isn’t a new release — it’s that 10 million people are running one open GGUF, which is the strongest evidence that local 27B-class models are the real, durable workhorse. The technical thread worth understanding is KV cache — at 500k-750k context it’s the memory bottleneck, prefix caching makes repeated work nearly free, and KV8-style quantization is what lets an Apple-Silicon box push past 150 tok/s. And the harness layer keeps shipping quietly: MLX-Serve, DwarfStar, Hermes and Pi all moved. If nothing here is a must-swap, that’s the honest read — but the numbers are unmistakable, and the local stack is getting faster at exactly the point that matters (long context).

Credits

• Unsloth — Qwen3.8-27B official GGUF (10M+ downloads, most-liked GGUF ever; verified via HF API).
• Qwen — Qwen-Drive-1.0-4B (Apache-2.0, fine-tune of Qwen3.5-4B for driving perception).
• ddalcu / Beamsters1 — MLX-Serve 26.9.2 (per-model settings, proxy, KV8 cache, >150 tok/s), MLX-serve monitor + MCP fan control, 750k-context prefix-cache test.
• ivanfioravanti / antirez — DwarfStar Qwen3.8-Flash-Next 64GB update (tools calling, prefill/decode, PLE memory) + agent /hints.
• witcheer / Teknium — Hermes (Wingtips #66 sessions.auto_prune, delegation docs + output_schema, Nous Portal revamp, Bots Mode).
• pidotdev — Pi project-directory scoping (createAgentSession(cwd), AGENTS.md + skills).
• iotcoi — prime-agents + tgrep vs ripgrep benchmark; googlegemma — ComfyUI chunked-video sampler; ostrisai — scratch diffusion training.

Sources: HuggingFace model cards and the HF API (downloads / likes verified directly — unsloth/Qwen3.8-27B-GGUF and Qwen/Qwen-Drive-1.0-4B cards), the curated X cache (read-only, fresh) for MLX-Serve / DwarfStar / Hermes / Pi / tooling signals, and the Qwen-Drive-1.0-4B card (arXiv 2609.00111). The 10M-download figure is Unsloth’s claim, confirmed against the HF API; the long-context and tok/s numbers are community measurements, not vendor claims. No fabricated numbers.