Loginhe claims the released GSQ-RCO quantizations (2.40–3.50 bpw) plus a 50%-expert-pruned Coder build of the 176.9B-parameter Qwen3.8-Flash-Next MoE retain usable capability at extreme compression, establishing quantization-plus-expert-pruning as a practical route to running very large MoE models in roughly 58–84GB of memory.

state: corroboratedheat: lowuncertainty: mediumconvergesscott: lowquantization expert-pruning local-inferenceLoginhe

What is this?

ISTA-DASLab — the Data Algorithms & Systems Lab at IST Austria — released GGUF quantizations of Alibaba's open-weights Qwen3.8-Flash-Next, a 512-expert MoE (~354GB at BF16), applying two of the lab's own methods: GSQ (described in coverage as Gumbel-Softmax Quantization) for per-tensor scalar quantization and RCO for budget-constrained per-tensor type allocation, with card-listed tiers at 2.40–3.50 bpw (66.4–83.6GB) claiming near-BF16 parity — exact AIME25 match at 3.00 bpw and the 3.50 IQ3_S tier 'matches or exceeds the base model on every task' (the card table shown here lists this tier, which the case had as title-only). A second, capability-targeted Coder build removes 50% of the 512 routed experts and quantizes the remainder to 3.5 bpw — 58.4GB total, an effective 1.89 bpw of the original — with the card claiming ~98.7% coding-performance retention. The snippets confirm both artifacts ship and their benchmark tables exist, but independent quality evidence within them is mixed: the lab's dense Qwen3.8-27B sibling quants were independently reported near-full quality at ~3 bpw, yet one tester found that sibling's smallest tier producing empty outputs, and the case history carries an unanswered negative coding hands-on of the extreme pruned Coder build — so 'usable at extreme compression' still rests mainly on the card's own benchmarks. Announced by u/Loginhe on r/Qwen_AI (~104 pts, ~30 days old), the release has since drawn field deployments via Strata expert streaming and a llama.cpp fork adding Q2_0 support specifically for the format.

Why it matters to Scott

ISTA-DASLab's per-tensor non-uniform GSQ-RCO quants independently arrive where Scott's own wiki already holds positions — per-tensor non-uniform quantization vs standard GGUF k-quants, and frontier-size MoEs runnable locally as sovereignty evidence — making this a dated receipt, not a repetition. Beyond convergence, the case bears directly on his open questions: the card's near-parity claims at 2.4–3.0 bpw versus duyntnet's unusable-for-coding ~1.89-bpw pruned-Coder hands-on is fresh evidence for exactly his 'minimum bpw before coding-agent quality degrades' and 'does half the expert pool matter' positions, the 66–76GB 3.0-bpw tier is directly testable on his 80GB-class gamepc box, and the Strata/ik_llama.cpp deployment ecosystem continues the expert-streaming and ByteShape/Bartowski per-tensor lineages the radar already tracks (radar:qwen38-flash-next-commodity-local-inference is a sibling episode of the same model, not a verdict on this case).

dev:concept.hardware-aware-local-inferencedev:project.gamepcwork:technology.large-language-modelsip:framework.sovereign-software-assuranceradar:qwen38-flash-next-commodity-local-inferenceradar:byteshape-qwen38-27b-quantsradar:bartowski-gguf-tensor-layoutsradar:concept.quantizationradar:concept.moe-inferenceradar:concept.expert-streamingradar:concept.extreme-quantization

queries asked of Scott's wikis

per-tensor non-uniform quantization methods vs standard GGUF k-quants — Scott's position
minimum bpw before coding-agent / agentic task quality degrades
MoE expert redundancy and expert pruning — does half the expert pool matter
expert offloading / CPU-GPU streaming for models oversized vs VRAM
local-inference hardware envelope notes for 32–128GB consumer rigs
open-weights sovereignty argument: frontier-size MoEs runnable locally

Measured heat

now 0 pts/hpeak 119 pts/hcomments 0/hpeers p25momentum: steady2 platformsage 842h

points/hour across evidence · reading as of 2026-10-12 02:59:37.977291+11:00 · deterministic, not a model opinion

How the heat travelled

09-06 14:00	⭐ origin echo-reconstructed	The HF model card is the primary artifact the Reddit post announces: "This repository provides GGUF quantizations of Qwen3.8-Flash-Next at t ISTA-DASLab (Deep Algorithms and Systems Lab, Institute of Science and Technology Austria) on other (echo) · attributed from reddit.post.1wt4s88	—
09-29 08:40	first on r/LocalLLaMA · published · +546.7h	[Release] GSQ-RCO GGUFs for Qwen3.8-Flash-Next, plus a 50% expert-pruned Coder build at ~1.89 bpw Loginhe	—
09-29 08:40	amplified on r/LocalLLaMA	reddit.post.1wt4s88 Loginhe	peak 104 · 77 comments · 16% of case engagement
09-30 23:10	amplified on r/LocalLLaMA	reddit.post.1wujph3 rmonsurate	peak 110 · 40 comments · 14% of case engagement
09-30 23:14	amplified on r/LocalLLaMA	reddit.post.1wujtbj rmonsurate	peak 1 · 9 comments · 1% of case engagement
10-02 19:19	amplified on r/LocalLLaMA	reddit.post.1ww2lv9 Fz1zz	peak 8 · 15 comments · 2% of case engagement
10-04 20:35	amplified on r/LocalLLaMA	reddit.post.1wxpvys Bulky-Priority6824	peak 1 · 15 comments · 1% of case engagement
10-05 23:33	amplified on r/LocalLLaMA	reddit.post.1wynomx IceFog72	peak 35 · 7 comments · 4% of case engagement
6 more amplifiers in ainews.case_chain
09-29 09:20	our radar first saw it · +547.3h	discovery anchor: reddit.post.1wt4s88	—

pace: p81 vs 519 stories at the 720h mark (now 842h old) — ahead of nvidia-sol-pi-harness-efficiency (1.1x), behind coop-coding-agent-vm-isolation (1.0x)

Evidence (13) — ⭐ canonical anchor

source	object	author	score	comments
🟠 reddit	[Release] GSQ-RCO GGUFs for Qwen3.8-Flash-Next, plus a 50% expert-pruned Coder build at ~1.89 bpw LocalLLaMA Retrieved article excerpt Open article · Retrieved 2026-09-29T09:24:00.036696+00:00 # Prove your humanity We’re committed to safety and security. But not for bots. Complete the challenge below and let us know you’re a real person. [Reddit, Inc. © "2026". All rights reserved.](https://www.redditinc.com/) [User Agreement](https://www.reddit.com/help/useragreement) [Privacy Policy](https://www.reddit.com/help/privacypolicy) [Content Policy](https://www.reddit.com/help/contentpolicy) [Help](https://support.reddithelp.com/hc/en-us)	Loginhe	104	77
🟧 echo.other ⭐	The HF model card is the primary artifact the Reddit post announces: "This repository provides GGUF quantizations of Qwen3.8-Flash-Next at t	ISTA-DASLab (Deep Algorithms and Systems Lab, Institute of Science and Technology Austria)	—	—
🟠 reddit	Releasing two post quantized trained models with fine tuned draft heads: Victoria (REAP + QAD) 44% smaller Qwen Flash Next, 70% TB 2.0 avg of three runs & Maple: Victoria + Canandian dataset finetune LocalLLaMA	rmonsurate	0	9
🟠 reddit	Two open-weights releases: Victoria (Qwen3.8-Flash-Next with 44% of experts cut, 70% Terminal-Bench 2.1, GGUF included) and Maple (a Canada-first fine-tune) LocalLLaMA	rmonsurate	110	40
🟠 reddit	QFN at 262K on 32 GB RAM 48GB VRAM : 1,800 tok/s prompt, 130 tok/s decod three Strata patches. LocalLLaMA	Fz1zz	6	15
🟠 reddit	QFN llama.cpp Any juice left to squeeze? LocalLLaMA	Bulky-Priority6824	0	11
🟠 reddit	k_llama.cpp MoE Optimizations: Expert Residency, Hybrid CPU/GPU Execution, Q2_0 Support LocalLLaMA	IceFog72	30	7
🟠 reddit	Qwen3.8-27B with just 12GB VRAM + 8GB RAM - ~18 tok/sec LocalLLaMA	bodhi371	17	12
🟠 reddit	just my "how I run qwen3.8 27b on 16GB" experience and guide LocalLLaMA	randomgenericbot	2	24
🟠 reddit	Ugh I didn't want to post this... Back to Qwen3.8 27B LocalLLaMA	86obsessed	131	243
🟠 reddit	Qwen Flash Q2_0 vs IQ2_XS GSQ-RCO using Strata LocalLLaMA	dampflokfreund	0	6
🟠 reddit	Qwen 3.8 Flash Next-GSQ-RCO-IQ2_XS at ~21 tok/s on just an RTX 3060 12GB + 16GB DDR4 RAM(No gate pruning, 100% bit-exact) LocalLLaMA	zyxciss	130	34
🟠 reddit	Qwen3.8-Flash-Next-GSQ-RCO (IQ3_S): ~20-30 tok/sec decode & 300-90k tok/sec prefill on 12GB VRAM + 32GB RAM + NVME LocalLLaMA	bodhi371	47	30

Interpretation history

2026-10-10T00:28:47Z
Two new runability datapoints (IQ2_XS on RTX 3060 12GB; IQ3_S on 12GB VRAM via custom Strata fork) reinforce quantization practicality at extreme low bits, but capability uncertainty persists — no new quality evidence, no attribution resolution for hallucination/instruction-following failures at iq3–iq4. Case remains corroborated-but-cold pending systematic eval or sampling-param isolation.
2026-10-09T19:58:37Z
evidence attached: reddit.post.1x1mzsg — Additional user-reported performance for GSQ-RCO IQ3_S on 12GB VRAM via custom Strata fork; reinforces quantization practicality claims.
2026-10-09T19:58:37Z
evidence attached: reddit.post.1x1tclb — User-reported decode speeds for Qwen3.8-Flash-Next GSQ-RCO IQ2_XS on consumer hardware directly corroborates the open case's quantization claims.
2026-10-08T07:29:19Z
New Q2_0 vs IQ2_XS Strata comparison adds another extreme-low-bit runability datapoint, but the velocity spike (86obsessed thread 135/239) is repetitive amplification of the already-priced iq3–iq4 quality contest — no new capability evidence, no attribution resolution. Scott's down-vote stands; world hasn't moved. Case stays corroborated-but-cold pending systematic eval or sampling-param isolation.
2026-10-08T07:01:41Z
evidence attached: reddit.post.1x0j0ae — User compares Q2_0 vs IQ2_XS GSQ-RCO quantizations of Qwen3.8-Flash-Next using Strata runtime, directly engaging with the quantization variants claimed in the case.
2026-10-07T17:18:30Z
No new facts this look: the activity is growth inside the already-priced iq3–iq4 quality debate (86obsessed thread now 95/186, WishfulAgenda adds a same-tier negative, same confounds), i.e. repetitive amplification of a recorded contest, not peripheral expansion — the 20x velocity spike and 91st-percentile peer rate are volume without substance, so heat stays low despite the speedometer. Scott's explicit not-relevant down-vote is respected (the world has not genuinely moved): relevance drops high→low and the case settles into corroborated-but-cold pending an attribution-settling eval or a dev/user response isolating sampling params.
2026-10-07T05:00:27Z
The contested zone widened: multi-user negative hands-ons now cover the mid iq3–iq4 MoE tiers in agentic work (32% hallucinated site index at iq3; instruction-following failures at iq4_xs), not just the extreme IQ1 Coder build — though live confounds (Strata server sampling params, Flash-Next's own preview-model glitchiness) blur how much loss is quant-attributable. The case now reads: runability and the prune/quant route proven, dense-sibling quality confirmed at method level, while the MoE's own capability retention is actively contested in the community's most-discussed recent thread.
2026-10-07T03:33:13Z
evidence attached: reddit.post.1wzj990 — Community counter-evidence that Flash-Next degrades vs 27B dense at iq3–iq4 quants in agentic work (32% hallucinated site index at iq3), directly qualifying the extreme-compression usability claim.
2026-10-07T01:36:09Z
grounded: converges/high — ISTA-DASLab's per-tensor non-uniform GSQ-RCO quants independently arrive where Scott's own wiki already holds positions — per-tensor non-uniform quantization vs
2026-10-07T01:26:21Z
The promotion blocker — independent capability confirmation — is now satisfied at method level: two same-day unrelated users report GSQ-RCO IQ3_S on Qwen3.8-27B delivering near-full-precision quality at ~11–16GB, joining the proven deployment ecosystem (multiple rigs, Strata consensus, ik_llama.cpp fork) as a second independent evidence line. The case's contested residue has narrowed sharply to the extreme tiers (2.4 bpw MoE builds, IQ1/Q2 pruned Coder) where duyntnet's negative coding hands-on stands unanswered; the route is corroborated, the extreme-compression clause is not.
2026-10-06T23:36:36Z
evidence attached: reddit.post.1wzfv45 — Second same-day independent report of GSQ-RCO IQ3_S near-full-precision quality (16GB build with KV streaming to 262k), corroborating the quant family's retention claim.
2026-10-06T23:36:36Z
evidence attached: reddit.post.1wzha67 — Independent user reports GSQ-RCO IQ3_S runs Qwen3.8-27B near full-precision quality at ~11GB on stock llama.cpp — method-level corroboration of the quant family's capability-retention claim.
2026-10-06T04:36:31Z
IceFog72's ik_llama.cpp fork is the case's first toolchain adoption — quant-type support (Q2_0) written specifically for GSQ-RCO artifacts, with a working IQ1_M Coder run — meaning the format has crossed from things-people-run to infrastructure-people-build-on. But this deepens the already-proven runability half without touching the contested capability half (duyntnet's negative hands-on still stands unanswered, no independent evals), so the case holds at watching/low; promotion now waits specifically on independent quality confirmation.
2026-10-06T03:33:52Z
evidence attached: reddit.post.1wynomx — Fork adds Q2_0 support specifically to run GSQ-RCO quants, with a working IQ1_M run of Flash-Next-Coder in comments — adoption evidence for the extreme-compression route.
2026-10-04T21:45:34Z
A third independent deployment — a user running the directly-linked ISTA-DASLab GSQ-RCO-Coder GGUF (~27.6GB, IQ1-class) on 2x 5060 Ti + 32GB RAM — upgrades the previously title-only pruned Coder build from 'unverified' to 'ships and runs, quality unassessed', and the same thread's consensus that Strata (not llama.cpp) is the way to run these quants consolidates the deployment pattern. Capability remains contested with no evals, and at 0.17 pts/h (39th percentile, steady) the numbers agree with low heat, but the compression-envelope story now demonstrably outperforms the claimed 58–84GB floor.
2026-10-04T21:24:13Z
evidence attached: reddit.post.1wxpvys — Third-party user actually deploying a GSQ-RCO IQ1 quant of Flash-Next at ~27.6GB on consumer hardware — real-world adoption evidence beyond the claimed 58-84GB envelope.
2026-10-02T20:17:09Z
The case's meaning shifts from 'released and benchmarked' to 'released, benchmarked, and field-run': Fz1zz's deployment of the case's own GSQ-RCO IQ3_XXS at full 262K context on 31GB RAM + 48GB VRAM via Strata expert streaming (130 tok/s decode) supplies the first independent decode numbers and verifies Strata by use — but the deployment thread's commenters ask the quality question and no one answers, so usable-capability stays contested against duyntnet's negative hands-on, and the 1.89 bpw pruned Coder build remains title-only. The practical-envelope half of the hypothesis is now independently demonstrated; the capability half is not.
2026-10-02T19:29:57Z
evidence attached: reddit.post.1ww2lv9 — Field deployment runs the case's own GSQ-RCO IQ3_XXS quant at full 262K context on just 31GB RAM via Strata expert streaming, pushing the practical envelope below the claimed 58–84GB.
2026-10-02T10:59:32Z
The velocity-spike triggers are multiples of near-zero baselines (3–8 pts/h absolute on the low-traffic Victoria post, against 0.08–1.0 peer baselines) — sensor noise, not renewed attention; measured velocity is 0 pts/h at the 11th peer percentile on a ~620h-old case. No new evals, no verification of the title-only 1.89 bpw pruned Coder build, no Strata repo content arrived, so the case keeps its shape: GSQ-RCO usability still contested (card parity claims vs duyntnet's unusable-for-coding hands-on), prune-and-retrain route independently proven by Victoria, patient watching brief unchanged.
2026-10-01T00:56:43Z
The prune-and-quantize half gains independent route-level proof-of-concept on this exact MoE: rmonsurate's Victoria (REAP 44% expert cut + NVFP4 retrain, GGUF shipped, 70% TB2.1) shows expert removal with retraining retains coding/agent capability — but by a different method, so it strengthens the route without validating the still title-only 1.89 bpw 50%-pruned Coder build. With the GSQ-RCO quants' real-world usability still contested (near-parity benchmarks vs duyntnet's unusable-for-coding hands-on) and traffic cooled to ~1 pt/h from a ~41 pt/h peak, this becomes a patient watching brief pending systematic evals rather than an urgent look.
2026-10-01T00:29:45Z
evidence attached: reddit.post.1wujph3 — Fuller post of the same Victoria/Maple release with GGUF results (75.3% TB) further corroborating the prune-and-quantize route on this model.
2026-10-01T00:29:45Z
evidence attached: reddit.post.1wujtbj — REAP expert-pruning plus NVFP4 retrain of Qwen3.8-Flash-Next holding 70% TB2.1 is independent corroboration that expert-pruning retains agentic capability on this MoE.
2026-09-30T13:39:24Z
The case's meaning shifted from promising first-party claims to a contested live test: third-party tooling (Strata) already claims support with 'high tps on consumer hardware' — the first independent adoption line — while the first hands-on report (duyntnet) found the quants unusable for coding (hallucinated Delphi/C++ syntax vs working 27B quants), directly contesting the 'usable capability' half and putting the ~1.89 bpw 50%-pruned Coder build most at risk. The velocity spike has cooled to ~0/h, so this is now a watching brief pending systematic community evals rather than an urgent look.
2026-09-29T09:46:39Z
origin walked (opencode/cheap-glm, conf 0.86): anchor reddit.post.1wt4s88 -> echo.other.5778328b4d by ISTA-DASLab (Deep Algorithms and Systems Lab, Institute of Science and Technology Austria)
2026-09-29T09:33:52Z
grounded: novel/high — First-party ISTA-DASLab release lands squarely on Scott's hardware-aware local-inference territory: a 512-expert 176.9B MoE at 66.4–75.8GB with 99.4% BF16 recov
2026-09-29T09:25:17Z
case created — A first-party release proposing a distinct joint quantize-and-prune method for a very large MoE, separate from the ByteShape and Bartowski episodes, and resolvable by community benchmarking and adoption.

Decision trace

10-11 08:31sensor_dirtycomment_update
10-11 08:31sensor_dirtycomment_update
10-10 20:31sensor_dirtyvelocity_spike
10-10 12:33sensor_dirtyvelocity_spike
10-10 11:28repriceTwo new runability datapoints (IQ2_XS on RTX 3060 12GB; IQ3_S on 12GB VRAM via custom Strata fork) reinforce quantization practicality at extreme low bits, but capability uncertainty persists — no new
10-10 07:07attention_routeThe editor compared this story and chose to keep watching.
10-10 06:58attention_candidateattach
10-10 06:58attachAdditional user-reported performance for GSQ-RCO IQ3_S on 12GB VRAM via custom Strata fork; reinforces quantization practicality claims.
10-10 06:58attachUser-reported decode speeds for Qwen3.8-Flash-Next GSQ-RCO IQ2_XS on consumer hardware directly corroborates the open case's quantization claims.
10-10 06:39propose_attachAdditional user-reported performance for GSQ-RCO IQ3_S on 12GB VRAM via custom Strata fork; reinforces quantization practicality claims.
10-10 06:39propose_attachUser-reported decode speeds for Qwen3.8-Flash-Next GSQ-RCO IQ2_XS on consumer hardware directly corroborates the open case's quantization claims.
10-08 18:29repriceNew Q2_0 vs IQ2_XS Strata comparison adds another extreme-low-bit runability datapoint, but the velocity spike (86obsessed thread 135/239) is repetitive amplification of the already-priced iq3–iq4 qua
10-08 18:09attention_routeThe editor compared this story and chose to keep watching.
10-08 18:01attention_candidateattach
10-08 18:01attachUser compares Q2_0 vs IQ2_XS GSQ-RCO quantizations of Qwen3.8-Flash-Next using Strata runtime, directly engaging with the quantization variants claimed in the case.
10-08 17:59propose_attachUser compares Q2_0 vs IQ2_XS GSQ-RCO quantizations of Qwen3.8-Flash-Next using Strata runtime, directly engaging with the quantization variants claimed in the case.
10-08 17:37sensor_dirtyvelocity_spike
10-08 07:22sensor_dirtyvelocity_spike
10-08 04:18repriceNo new facts this look: the activity is growth inside the already-priced iq3–iq4 quality debate (86obsessed thread now 95/186, WishfulAgenda adds a same-tier negative, same confounds), i.e. repetitive
10-08 01:42feedback_irrelevantScott vote via UI
10-08 00:59attention_routeThe editor compared this story and chose to keep watching.
10-08 00:21sensor_dirtyvelocity_spike
10-07 19:21sensor_dirtycomment_update
10-07 17:22sensor_dirtycomment_update
10-07 17:22sensor_dirtyvelocity_spike
10-07 16:00repriceThe contested zone widened: multi-user negative hands-ons now cover the mid iq3–iq4 MoE tiers in agentic work (32% hallucinated site index at iq3; instruction-following failures at iq4_xs), not just t
10-07 14:33attachCommunity counter-evidence that Flash-Next degrades vs 27B dense at iq3–iq4 quants in agentic work (32% hallucinated site index at iq3), directly qualifying the extreme-compression usability claim.
10-07 14:26propose_attachCommunity counter-evidence that Flash-Next degrades vs 27B dense at iq3–iq4 quants in agentic work (32% hallucinated site index at iq3), directly qualifying the extreme-compression usability claim.
10-07 12:36repriceThe promotion blocker — independent capability confirmation — is now satisfied at method level: two same-day unrelated users report GSQ-RCO IQ3_S on Qwen3.8-27B delivering near-full-precision quality
10-07 12:36groundISTA-DASLab's per-tensor non-uniform GSQ-RCO quants independently arrive where Scott's own wiki already holds positions — per-tensor non-uniform quantization vs standard GGUF k-quants, and f
10-07 10:36attachSecond same-day independent report of GSQ-RCO IQ3_S near-full-precision quality (16GB build with KV streaming to 262k), corroborating the quant family's retention claim.
10-07 10:36attachIndependent user reports GSQ-RCO IQ3_S runs Qwen3.8-27B near full-precision quality at ~11GB on stock llama.cpp — method-level corroboration of the quant family's capability-retention claim.
10-07 10:26propose_attachSecond same-day independent report of GSQ-RCO IQ3_S near-full-precision quality (16GB build with KV streaming to 262k), corroborating the quant family's retention claim.
10-07 10:26propose_attachIndependent user reports GSQ-RCO IQ3_S runs Qwen3.8-27B near full-precision quality at ~11GB on stock llama.cpp — method-level corroboration of the quant family's capability-retention claim.
10-06 17:21sensor_dirtyvelocity_spike
10-06 15:36repriceIceFog72's ik_llama.cpp fork is the case's first toolchain adoption — quant-type support (Q2_0) written specifically for GSQ-RCO artifacts, with a working IQ1_M Coder run — meaning the forma
10-06 14:33attachFork adds Q2_0 support specifically to run GSQ-RCO quants, with a working IQ1_M run of Flash-Next-Coder in comments — adoption evidence for the extreme-compression route.
10-06 14:26propose_attachFork adds Q2_0 support specifically to run GSQ-RCO quants, with a working IQ1_M run of Flash-Next-Coder in comments — adoption evidence for the extreme-compression route.
10-05 08:45repriceA third independent deployment — a user running the directly-linked ISTA-DASLab GSQ-RCO-Coder GGUF (~27.6GB, IQ1-class) on 2x 5060 Ti + 32GB RAM — upgrades the previously title-only pruned Coder build
10-05 08:24attachThird-party user actually deploying a GSQ-RCO IQ1 quant of Flash-Next at ~27.6GB on consumer hardware — real-world adoption evidence beyond the claimed 58-84GB envelope.