A model that remembers nothing

The lead: an attention-free model

What it is. Today has no headline model — nothing that beats what you already run — but it has real signal, and the sharpest thread is attention-free decoding. The freshest example is RWKV7-G1j-13.3B from fla-hub: the RWKV-7 “Goose” model repackaged in Flash Linear Attention (FLA) format, Apache-2.0, Transformers-compatible. RWKV-7 is an attention-free recurrent language model: instead of a KV cache that grows with every token, it keeps a constant-size recurrent state, so decoding does constant work per token regardless of sequence length. Training stays parallelizable (via FLA); only the decode path is recurrent.

Evidence. The 13.3B checkpoint sits at 355 downloads / 0 likes since Aug 31 — thin. The family (1.5B, 2.9B, 7.2B, 13.3B) is the same story. And a key caveat on the card: this is a base model, not a safety-aligned assistant — the chat template is provided, but the card states it may not follow instructions consistently. So treat it as an architecture signal, not a deployment choice yet: attention-free is the idea worth tracking, RWKV7-G1j is the latest proof-of-concept.

Models

K2-Horizon-7B — from IFM, Apache-2.0. A 7B-core dense decoder with native 524,288-token context (512K) from the midtraining stages onward, and the genuinely notable part: it is fully open — pretrain and midtrain datasets, the recipe, the training code and evaluation resources are all public, and intermediate checkpoints are released so you can watch capability change across training rather than at one snapshot. Benchmarks are vendor-reported: SWE-bench Verified 70.6 (vs Gemma 4-12B 30.6, Qwen3.5-9B 50.8, Granite 4.2-8B 47.7), HMMT Feb 2026 73.3, Terminal-Bench 2.1 39.1, BrowseComp 59.0 (using DeepSeek-V3.2’s Discard-all@95k context protocol — note the harness may differ from comparison models). Evidence: 1,656 downloads / 81 likes since Sep 1 — modest. Verdict: Marginal — adopt only if you want a fully-open 7B with 512K context; the dense-7B field is crowded and adoption is small, but the open-data + long-context combo is the real differentiator.

VibeVoice-ASR-Streaming-7B — from Microsoft, MIT. A unified streaming ASR model that transcribes Who (speaker) said What (content) as speech arrives, with customized hotwords (names, technical terms) and 10 languages (zh, en, fr, de, it, ja, ko, pt, ru, es). Technical report at arXiv 2609.02812. Evidence: 889 downloads / 128 likes since Sep 2. It’s a genuinely useful shape — streaming, speaker-attributed, MIT — but a 7B ASR is heavy next to the Whisper-class models most people run for transcription. Verdict: Marginal — adopt only if you want streaming speaker-attributed local ASR; MIT + streaming + speaker labels is useful for voice pipelines, and 889 downloads is real but small.

One-liner. inclusionAI/LLaDA-Image (Apache-2.0) — a fully-open 6B image generator: a diffusion transformer paired with a frozen vision-language model, with the entire training recipe published. Evidence: 228 downloads / 46 likes — clears the gate but thin. Verdict: Marginal — adopt only if you want a fully-open image-gen base with the recipe; adoption is still small.

Quantizations

No new quant build cleared the 200-download gate today. The freshest quant candidates (all Sep 7, e.g. ngquocvinh/K2-Horizon-7B-GGUF, cstr/vibevoice-asr-streaming-1.5b-GGUF, Devlin-Alpha-22B-A3B-GGUF) sit at 0 downloads — brand new, no adoption signal, no headline. The covered builds (unsloth Qwen3.8-Flash-Next GGUF, ISTA-DASLab GSQ+RCO, nvidia NVFP4, Jackrong, MATLOWAI H3 INT8) are unchanged.

Fine-tunes

dealignai/GLM-5.3-CYBERSECURITY-FP8 — MIT. A cybersecurity-focused fine-tune of GLM-5.3 (base GLM-5.3 + JANGQ-AI/GLM-5.3-FP8), delivered as FP8. The real delta: cyber-domain post-training on the GLM-5.3 base we covered earlier, baked into FP8. Evidence: 15,648 downloads / 200 likes since Aug 30 — the highest-adoption genuinely-new item today. But note: the base GLM-5.3 already showed emergent cyber (CyberGym 84.5, ExploitGym 105/130 — 2× GLM-5.2), so this is a domain-tuned follow-up, not a new capability. Verdict: Marginal — adopt only if you want a cyber-defense-focused GLM-5.3; the base already leans cyber, so this narrows the target rather than adding it.

inclusionAI/Ling-3.0-flash-Fin — MIT. A finance-focused variant of the Ling-3.0-flash MoE, aimed at financial research and agentic tool use. Evidence: 390 downloads / 67 likes since Sep 3 — clears the gate, thin. Verdict: Marginal — adopt only if you need a finance-tuned Ling-3.0-flash; adoption is too thin to treat it as a standard option.

Technical term of the day: linear attention / attention-free (RWKV)

What it is. Standard transformer attention needs a KV cache that grows with every token: memory and per-token compute scale with sequence length. Feed a model 128K tokens and the cache can outgrow the weights themselves. Attention-free recurrent models (RWKV-7 is the flagship) sidestep that: they keep a constant-size state — a small fixed set of vectors — and update it once per token, so memory is flat in sequence length and decode work is constant per token. The trick is that training can still be parallelized (Flash Linear Attention / FLA), so you get the training throughput of a transformer with the flat-memory decode of a recurrent net.

Why it matters for local AI. For anyone running models on a fixed machine, the KV cache is often the long-context memory killer — the reason a 13B can’t comfortably hold a 128K prompt on a mid-range card. An attention-free model keeps memory constant as you feed more, so long-context work fits on hardware you already have. That’s the entire pitch: fixed memory budget at long context, exactly what local deployment wants.

The trade-offs. It’s not free. Quality still trails attention transformers on some tasks — and RWKV7-G1j is explicitly a base model, not instruction-tuned, so it may not follow instructions reliably. The ecosystem (quant builds, serving tooling, harness support) is thinner than the transformer path. Decode is recurrent and sequential per token — you trade the KV-memory problem for a different bottleneck (no trivially parallel decode). And the state is fixed-size, so there’s a capacity ceiling: you can’t just grow the window, the “memory” is bounded by design. The practical rule: attention-free wins where you want a flat memory budget at long context and you’re okay with a niche stack; if you want mainstream quality + tooling, the transformer path with quantization is still the safe bet.

Harness improvements: DeepSeek Harness unbundles the agent

The harness story is DeepSeek Harness (dsh) — an MIT-licensed agent runtime (developer preview, Aug 13) built on a TypeScript micro-kernel (Cordis) where models, tools, sandboxes, memory and even the UI are swappable plugins — the repo’s one-line thesis: “everything is a plugin”. It’s model-agnostic: you can drive it with DeepSeek V4, Claude, or GPT endpoints, local or remote, without touching the runtime. Four shipped profiles — Standard (shell, file editing, workspace search, planning, subagents), Code (tools exposed via a TypeScript SDK so a model emits structured scripts), Minimal (just a shell + str_replace_editor, for benchmarking), and Creator (a runtime inspector/sandbox for authoring plugins). The standout feature is an append-only trajectory log: every user message, tool call, intermediate reasoning state, token metric and subagent dispatch is recorded into one execution trail — so you can replay a run, isolate the step that broke it, and benchmark across runs. Getting started is one command: npx @deepseek-ai/dsh web → local UI at 127.0.0.1:3080 on Node 18+.

Why it matters. This is the unbundling thesis: the agent loop decoupled from any single model, the way an OS is decoupled from any single app. For local-AI people that means you keep your local runtime and swap the model without forking the harness, and the trajectory log makes every run inspectable — the same auditability push that’s winning in production. Adoption signal: ~170K GitHub stars, 18K forks, ~13K commits in about a week after reportedly clearing 125K stars in three days. Stars are cheap and it’s a fast-moving preview, so treat velocity as signal, not proof. Caveat: it’s a developer preview with compatibility-breaking changes and plugin contracts still stabilizing — don’t bet production on it yet, but watch the dsh-plugin ecosystem.

The takeaway

Today is signal, not headline. No single release is a must-swap, but the threads are real: a fully-open 7B with native 512K context and open training data, a streaming speaker-attributed ASR that’s MIT and practical, an attention-free 13.3B that never grows a cache, a cyber fine-tune of GLM-5.3, and an agent runtime that unbundles the model from the harness. The attention-free idea is the quiet engine worth understanding: fixed memory at long context is exactly what local hardware wants. If nothing here beats what you already run, that’s the honest read — but the direction is right.

Credits

• fla-hub — RWKV7-G1j-13.3B-20260831 (FLA RWKV-7 “Goose”, Apache-2.0, base model).
• IFM — K2-Horizon-7B (fully-open dense 7B, 512K context, open data/recipe/code).
• Microsoft — VibeVoice-ASR-Streaming-7B (MIT, streaming speaker-attributed ASR, arXiv 2609.02812).
• inclusionAI — LLaDA-Image (fully-open 6B image generator), Ling-3.0-flash-Fin (finance MoE variant).
• dealignai — GLM-5.3-CYBERSECURITY-FP8 (cyber fine-tune of GLM-5.3).
• DeepSeek — DeepSeek Harness (dsh) developer preview (MIT, Cordis micro-kernel, trajectory log).

Sources: HuggingFace model cards and the HF API (downloads / likes verified directly), the fla-hub RWKV7 card, the IFM K2-Horizon-7B card, the Microsoft VibeVoice card, the inclusionAI cards, the dealignai GLM-5.3-CYBERSECURITY-FP8 card, and the DeepSeek Harness developer-preview documentation. Benchmarks are vendor-reported where cited. No fabricated numbers.