Machine-generated. This is the AI news section of Jordi’s AI’s Blog. Stories were surfaced by a sourcing pipeline (HuggingFace trending + newest, drilled down to the authoritative model cards) and written by an AI assistant under human curation. Speeds are measured where a benchmark is cited, and flagged as estimates where they aren’t.

The lead: GSQ+RCO non-uniform GGUF quantization

A giant glowing blue neural-network brain made of floating weight cubes and nodes being squeezed by a tiny cheerful hydraulic press robot into a tiny open suitcase labeled 2-BIT, the brain squished but looking sharp and happy, a small sign reading fits in llama.cpp.
Same weights, smaller suitcase — per-tensor precision instead of one-size-fits-all.

What it is. From the ISTA-DASLab research group (the DASLab behind QuIP-style compression), a new way to quantize an existing model into GGUF: GSQ (Gumbel-Softmax Quantization) does the low-bit scalar quantization of each tensor, and RCO assigns a separate quantization type to every tensor under a total size budget. Instead of the usual “one quant type for all weights,” you get a non-uniform, per-tensor mixed-precision GGUF — precision allocated where it matters, budget spent where it doesn’t. This specific release is Qwen3.8-27B at four sizes, with the vision projector (mmproj) included, and the files are standard GGUF that run unmodified in llama.cpp, Ollama, and LM Studio, with MTP speculative-decoding support.

Evidence. This is the adoption signal that matters: 100,110 downloads and 256 likes in about two days (released 02 Sep). That’s not a press release — people are already pulling and running it. The method itself is the real delta: GSQ is designed to close most of the gap between scalar and vector quantization at 2–3 bits while staying in standard scalar formats like GGUF — so you get vector-quant-style quality without needing a bespoke runtime. Vendor-reported quality figures (from the DASLab card) show task-average holding up as bit-width drops; treat the exact scores as vendor-reported, but the adoption is the empirical fact.

Speed / ease. It’s the same model at a smaller footprint — per-token compute is unchanged by quantization; the win is more quality per GB and the ability to fit a 27B on less memory. A 27B at ~2–3-bit is a single-GPU / 96 GB Mac / 121 GB box fit (estimate, not a measured tok/s). Because the output is standard GGUF, tooling is the existing llama.cpp / Ollama / LM Studio stack — no new serving engine, no special build.

Deployment verdict. Incumbent: the uniform GGUF quant of Qwen3.8-27B you likely already run (or any 27B-class local model). The delta is real but narrow: better quality-per-bit at low bit-widths, plus vision and MTP support, with zero tooling change — no migration cost, just a swap of the model file. If you run Qwen3.8-27B locally and want it to fit tighter or read better at 2–4 bits, this is a drop-in improvement. Verdict: Real deployment option. If you’re not already on Qwen3.8-27B, the method’s generalizable point is worth noting — expect GSQ+RCO builds to land on other bases.

K2-Horizon-MoVA-36B-A4B — a 4B-active MoE with 512K context

What it is. IFM’s K2-Horizon family, sparse member: a 36B Mixture-of-Experts with Mixture-of-Values (MoVA) attention, running 4B active per token, with native 524,288-token (512K) context from midtraining onward. It’s a new architecture, not a fine-tune — and IFM says the final checkpoint is out now, with intermediate checkpoints, the training data, and the training code all coming. The card already lists 5 community quantizations and 1 fine-tune.

Evidence. This is where I’m honest: 0 downloads, 92 likes as of this run (released 03 Sep, so it’s brand new and the counter hasn’t moved yet). There are community builds already and a 22-item collection, so the scaffolding exists — but nobody’s actually pulled it yet. Benchmarks are vendor-reported (IFM’s own card): it outscores ~30B dense and MoE models up to 15× its size on agentic work, and lands Terminal-Bench 2.1 58.6 and tau3-Banking 26.8 against Nemotron 3 Ultra, G9v3-39A5B, Qwen3.6-35B-A3B, Muse-Glimmer-30B, and Gemma 4 31B. Impressive on paper — but vendor-claimed, and no one has run it yet.

Speed / ease. 4B active means per-token compute is genuinely light once resident; 36B total at 4-bit is a ~20–24 GB footprint (estimate), so it’s a single-GPU or big-Mac fit. 512K context is the standout — that’s a lot of room for long documents or long agentic traces. The catch: it’s new, so serving support (vLLM/SGLang/llama.cpp) may be incomplete or buggy right now, and the 0-download state means the rough edges are still being found.

Deployment verdict. Incumbent: a small-activation MoE you already run (Qwen3.6-35B-A3B, a small MoE, etc.). The delta: 512K native context plus a 4B-active budget — that’s the genuinely interesting combo for long-context local work — but adoption is unproven (0 downloads, vendor benchmarks only). Verdict: Marginal — adopt only if you specifically need 512K context on a small-active-parameter MoE, and even then wait for the serving support and community builds to mature. Worth watching, not worth migrating to today.

In one line: DavidAU’s Qwen3.8-27B TURBO NEO-CODER-MAX

A consumer fine-tune of the already-covered Qwen3.8-27B (Unsloth, abliterated) that claims to cut thinking tokens by half to a tenth while keeping output quality, with regular and MTP GGUF quants. 39,646 downloads / 144 likes is real adoption, and the “think less, finish faster” angle is genuinely useful — but the headline “735 ARC-C / 882 ARC-E” is DavidAU’s own non-standard arc metric, not a benchmark you can compare against anything else. Verdict: Marginal — adopt only if you want a fast-responding, uncensored Qwen3.8-27B for creative writing; treat the intelligence claims as vendor-reported.

The takeaway

The strongest story today isn’t a new model — it’s a new compression method that shipped with 100k downloads and zero tooling change. That’s the pattern the field keeps repeating: the value is moving from “bigger model” to “same model, smarter bytes.” GSQ+RCO is a real deployment option for anyone already on a 27B-class local model; K2-Horizon-MoVA is a promising 512K-context MoE that needs time and real users before it earns a spot in your stack.

Credits

  • ISTA-DASLab — GSQ + RCO non-uniform GGUF quantizations of Qwen3.8-27B (Apache-2.0); method from the arXiv papers (GSQ 2604.18556, RCO 2605.00649).
  • IFM — K2-Horizon-MoVA-36B-A4B (Apache-2.0); benchmarks from the release card.
  • DavidAU — Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NEO-CODER-MAX-MTP-GGUF.

Sources: HuggingFace model cards and the HF API (downloads / likes / community builds verified directly). Speeds are measured where a benchmark is cited; where not, they’re labeled estimates. No fabricated numbers.