Frontier on a conveyor belt — calibrated quants and the thinking-token bill

The lead: the calibrated Q2/Q4 for DeepSeek-V4.1-Flash

What it is. On Sep 11, antirez — the Redis author who has been spending his weekends making absurdly large MoEs run on Apple Silicon — published antirez/deepseek-v4.1-flash-gguf: official-grade calibrated weights for the ds4.1flash branch of DwarfStar, the inference engine behind last week’s “V4.1 Flash streams on a 128 GB M5 Max” posts. This is not another mradermacher-style re-quant: it is the engine author’s own quantization pass, documented like a lab notebook. Q2: a 340.60 GiB file whose main weights are 151.77 GiB — IQ2_XXS on routed gate/up experts, Q2_K on down experts, and Q8 kept for attention projections, shared experts and output. Q4: 482.98 GiB file, 294.15 GiB main, Q4_K across routed experts. Both keep the 188.83 GiB of native FP8 Engram tables unquantized — DwarfStar reads the rows it needs straight off disk and never makes the table resident — and both were cut from the same 8,192-token activation imatrix, not requantized from each other. A separate 0.90 GiB vision encoder completes the multimodal story.

The deployment math, straight off the card. One 128 GB Mac runs Q2 with SSD streaming (keep the GGUF on fast local storage); Q2 fits fully resident only across two 128 GB Macs over RDMA, or on a larger single Mac. Q4’s resident target is a 512 GB Mac; smaller machines stream, and Q4 does not fit resident tensor-parallel across two 128 GB boxes. One practical wrinkle: Q4 exceeds HuggingFace’s single-file limit, so it ships as two parts that the DwarfStar downloader joins and verifies (budget +37 GiB of free disk while it joins). And a real caveat, not a nit: these weights are DwarfStar-specific — “not interchangeable with DeepSeek V4 Flash weights or its DSpark drafter” — so this buys you into the ds4 ecosystem, not into llama.cpp.

The evidence. 1,086 downloads / 21 likes in ~2 days — modest in absolute terms, enormous in meaning: this is the repo where people chasing the Sep 12 story go to actually run the model. The surrounding wave is the bigger number. As of today the base DeepSeek-V4.1-Flash repo sits at 140,636 downloads / 2,058 likes — nearly double the 75,774 we logged yesterday, in about 24 hours — and at least 24 community builds now clear 200 downloads across four runtimes: GGUF (vcruz305 2,733→8,855, apetersson MixedQ2 2,905→5,524, AMAImedia FP8 2,133→5,028), MLX with MTP (Vontra 1,140→2,087, OpensourceWTF’s streaming-Q2 at 755, Jundot’s oQ4e-MTP at 794), EXL3 (bot-lab-21 at 3.5 bpw, diffbot’s 2.0 bpw twin targeting 2× RTX PRO 6000), and NVFP4. Curious detail for a Sunday: unsloth still hasn’t shipped a V4.1-Flash GGUF — their newest DeepSeek build remains the V4-Flash-Vision one from Aug 31. The calibrated one came from the engine instead.

Verdict: Real deployment option — if you own a 128 GB+ Mac and accept engine lock-in to DwarfStar, this is the most trustworthy way to run the open frontier at home today: known quant recipe, known memory math, author-maintained. If you’re on 16–32 GB or live in llama.cpp/Ollama, this one is not for you yet; the generic GGUF wave’s Marginal verdict from Sep 12 still stands.

Models

Nothing new clears the gate today. The HF newest/trending sweep surfaced no previously-uncovered base model above 200 downloads — the freshest lab releases are all covered (DeepSeek-V4.1-Flash, the Sep 12 lead), and everything else created in the last day is at zero downloads, still being born. On a Sunday that’s the honest Models section. What did move is adoption, which is the other half of the story:

MiniCPM5-2B keeps compounding. The OpenBMB edge model we made a Real deployment option on Sep 8 has gone from 2,879 downloads / 601 likes to 102,334 downloads / 1,276 likes in five days, with its official GGUF at 87,316. A 35× adoption in under a week says the 2B-permissive slot has a real owner. Verdict unchanged — and now with numbers.

K2-Horizon-MoVA-36B-A4B went from the 0 downloads we flagged on Sep 4 to 5,616 / 299 likes; GLM-5.3-CYBERSECURITY-FP8 doubled from 15,648 to 30,068 / 416. Both verdicts from their original posts stand: K2-Horizon stays Marginal — independent evals still haven’t landed; the cyber FP8 stays a specialist’s pick, not a daily driver. For texture: the WarmBloodAban MiniMax H3 Singularity build has doubled to 114,141 downloads, and nvidia’s Qwen3.8-Flash-Next NVFP4 went 78,737 → 89,924. The tape is green where the gates are old and quiet.

Quantizations

The 665,911-download elephant. DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-…-NEO-CODER-MAX-MTP-GGUF — yes, that is the filename — has reached 665,911 downloads / 578 likes, the llama.cpp build of the uncensored “Turbo” fusion family we’ve tracked since Sep 4. The build itself: both regular and MTP speculative-decoding GGUF quants using a dual-imatrix (“DI-MATRIX”) pass, Apache-2.0, fine-tuned on consumer hardware with Unsloth. The advertised delta is throughput-shaped: thinking tokens cut to 1/2–1/10 of base Qwen3.8-27B while (their word) keeping output quality. The advertised benchmarks are DavidAU’s own non-standard “735 ARC-C” framing, which we flagged on Sep 4 and flag again — the download number is the real data; the quality board is his. For scale of the family’s pull: the Qwopus3.8-27B-Flash GGUF we noted at 10,680 downloads on Sep 6 is now at 260,350. Community GGUFs of the Qwen3.8-27B line are eating the official builds’ lunch. Verdict: Marginal — adopt only if you want a fast, uncensored Qwen3.8-27B for creative and role-play workloads; it is not a coder’s upgrade over the base model.

Fine-tunes

Minimax-H3-ComfyUI (Alissonerdx) — an Apache-2.0 rank-64 LoRA for MiniMax H3 (ref2va base), built for the Comfy-Org H3 weights, with a ready ComfyUI workflow and before/after videos in the repo. The LoRA — “LMS”, its self-description is a little more sharpness — sharpens a source video while keeping it photorealistic. The interesting part is how: it conditions on your source clip as a latent guide aligned frame-to-frame with the target — same temporal origin, same spatial grid, and crucially it does not advance the model’s reference clock, so the guide-to-target correspondence is handed to attention instead of searched for. That’s the same in-context arrangement LTX video LoRAs use, now pointed at H3. Evidence: 10,390 downloads / 127 likes since Sep 6, still climbing (updated Sep 12). Verdict: Marginal — adopt only if you run H3 in ComfyUI and want a photoreal enhancer; the guide-latent trick is the reason to watch this builder.

LTX-2.5 IC-LoRAs (Lightricks) — Sep 10, four new in-context control LoRAs for the covered LTX-2.5-22b: Ingredients (239 downloads / 20 likes), plus Cinemagraph, Colorization and Decompression (51 / 46 / 41 downloads — under the gate). The Ingredients one is the only one with pull so far. Verdict: Marginal — niche control handles on a model you already run; too early to judge the family.

Below the gate, one line. dealignai’s DeepSeek-V4.1-Flash-UNCENSORED-FP8 has grown from the 171 downloads we noted yesterday to 943 / 69 likes — the day-one abliteration pipeline now outlives its own novelty. Skip unless you specifically want it.

Technical term of the day: the reasoning budget (what thinking tokens actually cost)

What it is. Reasoning models answer in two phases: they emit thinking tokens — private deliberation, chain-of-thought — and only then the answer. The reasoning budget is how much of that first phase you pay for: a hard cap on thinking tokens, an on/off switch, or a graded dial like DeepSeek-V4.1-Flash’s reasoning_effort 1–100. Every local deployer sets this dial; almost nobody sets it by measurement.

A measurement, not a vibe. This week a community builder ran the cleanest experiment of the month: the same 198 GPQA-diamond questions, same RTX 5090, same llama.cpp server, greedy decoding — and moved only the thinking-token ceiling (4k → 8k → 16k):

• Qwen3.8-27B (Q6_K): 60.1% → 70.2% → 79.3%
• Ornith-1.5-35B-A3B (Q4_K_M): 70.2% → 77.3% → 81.8%

Two things live in those numbers. First, a tight budget hides your model’s quality: the 27B was running 19 points below its own 16k-budget score at 4k, purely because the operator capped the thinking. Second, architectures want different budgets: the sparse MoE started 10 points higher at 4k (70.2 vs 60.1) but gained less from quadrupling the ceiling (+11.6 vs +19.2 points) — it reaches for the answer sooner and wastes fewer tokens getting lost. Caveats, honestly: one run per configuration, one quant per model, 198 questions, one card. It’s a direction, not a leaderboard — but it’s more than any vendor table tells you about your budget question.

Why it matters for local AI. On an API, wasted thinking tokens cost fractions of a cent; on your own box, every thinking token is billed at your decode speed. 8,000 thinking tokens at 40 tok/s is 200 seconds per answer — before the answer starts. That’s why the ecosystem keeps orbiting this dial: the Sep 8 measurement on Qwopus3.8-27B-Flash counted ~4,229 tokens per correct GPQA answer as an efficiency metric, DavidAU’s TURBO family sells 1/2–1/10 the thinking tokens as its headline feature (vendor claim, same flag as always), and frontier releases now ship effort dials as a first-class feature. The correct local move is to measure tokens-per-correct on your own 50-prompt sample of your actual workload: cap the budget until accuracy starts bending down, then stop one notch earlier.

The trade-offs. Returns diminish non-linearly — the 4k→8k jump bought ten points, 8k→16k bought nine, and the next doubling will buy less; the shape of that curve differs per model and per quant, so a Q4 model can spend budget on confused deliberation in ways the Q8 never does. Watch benchmark tables that don’t state their budget: a score quoted at a 4k thinking cap is a different model than the same checkpoint at 16k, and nobody wants to say that in a press release. And remember the counter-pressure: thinking tokens are decode-time compute — the same wall speculative decoding attacks for other reasons. The practical rule: set your reasoning budget like you set context size — from a measurement on your machine, not from the default.

Harness improvements: the local stack keeps moving

DwarfStar is becoming an ecosystem, not a script. In the ~week since the V4.1 SSD-streaming posts: carlzulauf/ds4.nix packages the engine as a Nix flake, Baekpica/ds4-dfm-rs is a day-old Rust-host continuation of the project, and two crowd efforts put one-click DwarfStar launch scripts on a single NVIDIA DGX Spark. A third-party inference-deploy repo adds systemd integration for ds4. Engine authors get fans; engine ecosystems are rarer and worth noting.

llama.cpp shipped the boring plumbing that agent tooling eats. Across b10931–b10938 (Sep 12–13): a new common_schema internal representation with a JSON-schema optimizer that the json-schema-to-grammar path now runs on — structured-output groundwork; improved tool-call parsing for qwen3-coder (the model family half the community runs agents on); OpenCL row-alignment fixes reaching q4_K/q5_K/q8_0 kernels; and a Vulkan workaround for a driver-level queue-submit bug. Nothing you’d tweet; all of it lands under your agent stack.

Hermes Desktop news. A community kit cleans up Bots mode: the bot’s main chat shows only messages, with approvals and inter-bot traffic moved to their own lanes — multi-agent chatter stops drowning the conversation. And Collective Wisdom shipped: Hermes can now notice that a skill you wrote is genuinely useful, review it — including a security check — and propose sharing it with your team, where others discover and install it. Skill libraries are starting to grow social layers; watch whether the review step holds up. On the calendar: Nous x Berlin, Sep 23 — fifty Hermes users, a room in Kreuzberg, open floor. If you’re in Germany, apply.

One for the eval-skeptics. An Earendil engineer published a piece this week on why “sloppiness” in code is so hard to quantify, which code-quality evals don’t survive contact with reality, and which metrics actually matter. When code generation is abundant, quality measurement is the bottleneck — the article is the clearest thing we read all weekend on the subject.

The takeaway

Sunday tape: the model pipeline paused, and the build layer is where the action was — which is itself the story of this week. DeepSeek-V4.1-Flash doubled its downloads in a day and now has 24+ community builds past 200 downloads plus the engine author’s own calibrated Q2/Q4: for the 128 GB+ club that’s a Real deployment option, engine lock-in and all. The DavidAU Turbo GGUF at 665k downloads and the H3 ComfyUI LoRA at 10k are both Marginal — adopt only for their niches, but both tell you where hobbyist compute actually goes. And if you take one dial home: your reasoning budget is the cheapest quality knob you own — the measured curves say a 4k cap can cost you ten points, and nobody put that in the model card.

Credits

• antirez — calibrated DeepSeek-V4.1-Flash Q2/Q4 GGUFs for DwarfStar (model card, quant recipe, memory math); his public X posts on V4.1 Flash on Apple Silicon.
• DeepSeek-AI — DeepSeek-V4.1-Flash (MIT), the base of the wave.
• Builders — vcruz305, apetersson, AMAImedia, Vontra, Jundot, OpensourceWTF, bot-lab-21, diffbot, dealignai et al. — the maturing quant wave; DavidAU and Jackrong — the 27B GGUF builds with six-figure downloads.
• Alissonerdx — the H3 ComfyUI LoRA + workflow; Lightricks — LTX-2.5 IC-LoRA family; OpenBMB — MiniCPM5-2B at 100k downloads.
• Community measurements — witcheer’s GPQA thinking-budget sweep and RTX-5090 fit work (via public X posts); the ds4.nix / ds4-dfm-rs / DGX-Spark crowd; Sebastian Baye (Earendil) on code-quality evals; tonysimons_, Teknium, witcheer for Hermes Desktop updates.

Sources: HuggingFace model cards and the HF API (all downloads / likes figures verified directly on 13 Sep 2026), the curated X cache (read-only, refreshed this morning) for the GPQA measurement and harness signals, model-card READMEs for every quant/fine-tune claim quoted, and llama.cpp release notes b10931–b10938. Vendor- and author-reported benchmarks are flagged as such; the thinking-budget curve is one community run (198 questions, one card, one quant per model) and is labeled as a direction, not a leaderboard. No fabricated numbers.