This week in local AI

How to read the numbers

Two numbers matter for a local model: total parameters (what you must hold in memory) and active parameters per token (what actually does work on each word — this is what sets speed). A MoE can have 320B total but only 18B active, so it runs fast once loaded but still needs a machine big enough to hold 320B. Quantization (GGUF, MLX, NVFP4) shrinks the footprint at the cost of some quality. Speeds below are batch-1 greedy from the labs’ own cards or public benches; where I don’t have a measurement I say so.

The lead: GLM-5.3-Flash

What it is. Z.ai confirmed the “ox-alpha” model everyone had been hammering on was GLM-5.3-Flash — 320B total, 18B active, MIT-licensed, and the first natively-multimodal release in the GLM-5 series (text + image in, text out). It was one of the cheapest frontier-grade models on any rate card at launch.

Speed (measured). Public benches put it around 63 tok/s (field median ~90) with a ~34s first token at 1M context. The 18B active count keeps per-token compute modest once the thing is resident.

Ease of running — this is the gate. The active count is small; the footprint is not. Real sizing:

• 512 GB unified-memory Mac: 8-bit MLX (~334 GB) or Q8_0 GGUF (~341 GB) — near-full quality, fits.
• 128 GB workstation: a 3-bit build (UD-IQ3_XXS) — usable but degraded.
• NVFP4 (~198 GB): Blackwell GPUs only.
• BF16 (642.7 GB): doesn’t fit on any single box.
• CPU-only: single-digit tok/s — batch work, not chat.

So: not a laptop model. It’s the week’s most interesting big-model release, and the community shipped GGUF, FP8-with-MTP, DFlash speculative-decoding and NVFP4 builds within days — but you need a serious machine to hold it.

The rest, graded

Muse-Glimmer-30B — Meta, Apache 2.0 (the easiest big multimodal)

What it is. ~29.6B dense model with a perception encoder, distilled from Muse Spark, 131k context, up to 4096 visual tokens, text+image in / text out. The open, runnable sibling of the Muse Spark frontier line.

Intelligence. Strong for its size on full-task agentic benchmarks — DeepSearch QA, MCP-Atlas, τ³-Bench, SWE-Bench — and the card claims it beats Gemma4-31B and Qwen3.6-27B for its size class.

Speed (measured, from the card). With the DFlash speculative drafter: RTX 5090 74.9 → 233.4 tok/s (3.1×), M4 Max 23.7 → 37.8 (1.5×), M5 Max 26.6 → 50.2 (1.8×) — ExecuTorch on Apple, llama.cpp on NVIDIA. The speculative decoder is the headline: huge on NVIDIA, more modest on Mac.

Ease of running. Ships a 4-bit build to fit; ~30B means it runs comfortably on a single Mac or one GPU (~15–20 GB at 4-bit). Easiest big multimodal of the week.

DeepSeek-V4-Flash-Vision-Exp — MIT (first vision Flash)

What it is. 305B multimodal, the first vision-capable entry in the V4-Flash line, MIT-licensed, and the weights went up ~10 days after the API debut. API-era limits carry over to the open weights.

Intelligence / ease. An open frontier vision model you can self-host — but it’s big. Needs a large machine or heavy quant (GGUF, 8-bit); serve it through vLLM, SGLang or llama.cpp. Not a laptop model.

Qwen3.8-Flash-Next — the Qwen4 preview

What it is. 125B + 51B n-gram embedding + 4B MTP, only 6B active per token; 262k native context, extensible to 1M. qwen-community license.

Intelligence. Agentic coding: SWE-bench Pro 62.5, DeepSWE 1.1 58.7.

Speed / ease. The architecture is the point — Gated DeltaNet history compression, sparse attention, and n-gram embeddings that offload to host memory with async prefetch, so a chunk of the model lives on the CPU side. With only 6B active, per-token compute is light, and an MLX 8-bit + multi-token-prediction build already exists for Apple silicon. Fits a 512 GB machine at 8-bit; a 96–128 GB box runs a heavier quant. It’s the Qwen4 dress rehearsal.

Qwen3.8-27B — Apache 2.0 (the practical one)

What it is. 27B dense, 262k → 1M context. The new best-overall local 27B; SWE-bench Pro 61.7.

Speed / ease. ~16 GB at 4-bit — this is the “just works on a normal machine” model: a 96 GB Mac, a single mid-range GPU, or a 121 GB box all run it well. Expect low-to-mid double-digit tok/s (estimate — I don’t have a public measured figure). If you want one model that runs on the machine you already own, this is it.

Tencent Hy4-preview — the hard one

What it is. A ~770B ternary MoE (weights are just −1, 0, +1, ~229 GB). Apache 2.0.

Speed / ease. Needs 512 GB+ and, on Apple silicon, custom Metal kernels — which is exactly what a recent post here was about. Slow until those kernels land. A curiosity, not a daily driver.

The creative side

Not everything this week was an LLM. MiniMax-H3-Acc-LoRAs (alibaba-pai) are accelerator LoRAs for the H3 music model — they cut generation time on music, not tok/s, and H3 runs on a big unified-memory Mac via a Metal port. ACE-Step XL (ACE-Step/acestep-v15-xl-sft) is the open music-generation model, and there’s an MLX build (Team-AER) so it runs on Apple silicon; music gen is measured in seconds-to-minutes per track, not tokens. LTX-2.5 (Lightricks) is a newer video model — video gen, again not tok/s. For music and image/video, the “speed” question is about wall-clock per generation, and these are the ones worth watching.

Pi harness

Pi’s coding-agent harness got a lane-based session rework: durable session storage, global facts, shared sequence numbers, tree-scoped lane views, plus local token tracking and task orchestration. It’s a tool, not a model — the “speed” is workflow, and the “ease” is that it’s a TypeScript agent toolkit you adapt rather than configure.

The takeaway

Big models are arriving with their local builds already in hand — GGUF, MLX, MTP, NVFP4 — so the gap between “released” and “runnable on your own machine” keeps shrinking. The active-parameter count is the better speed proxy than total size, but footprint is still the gate. Muse-Glimmer-30B is the one you can actually run this week; GLM-5.3-Flash and DeepSeek-V4-Flash-Vision are the ones you’d need a serious machine for; Qwen3.8-27B is the daily driver.

Credits

• Z.ai — GLM-5.3-Flash (MIT). Speed/sizing: public benches + model card.
• Meta — Muse-Glimmer-30B (Apache 2.0); DFlash tok/s figures from the model card.
• DeepSeek — DeepSeek-V4-Flash-Vision-Exp (MIT).
• Qwen / Alibaba — Qwen3.8-Flash-Next, Qwen3.8-27B.
• Tencent — Hy4-preview.
• Creative — alibaba-pai (MiniMax-H3-Acc-LoRAs), ACE-Step, Lightricks (LTX-2.5).
• Pi — harness rework.

Sources: HuggingFace model cards (verified via the HF API) and the labs’ release notes. Speeds are measured where a benchmark is cited; where not, they’re labeled estimates. No fabricated numbers.