LocalLLaMA builder Yaniss916 claims the released Kyojin ROCm engine (built on ExLlamaV3 for gfx1151) and mixed EXL3 packs run 300B-class MoE models — GLM-5.3-Flash at ~580 tok/s prefill and 26–30 tok/s decode, MiMo-V2.6-Flash up to 44 tok/s — with near-FP8 fidelity (KLD 0.151/0.071, ~90% top-1 agreement) on a single 128GB Strix Halo mini PC; independent replication and adoption would establish Strix Halo APUs as a standard tier for large-MoE local inference.

state: corroboratedheat: mediumuncertainty: mediumconvergesscott: highlocal-inference strix-halo moe-quantizationYaniss916

What is this?

Yamz Labs (Reddit u/Yaniss916) released Kyojin, an open-source ROCm inference engine built on ExLlamaV3 for AMD's Strix Halo APU (gfx1151), claiming it runs two ~300B-class MoE models — GLM-5.3-Flash and MiMo-V2.6-Flash — on a single 128GB mini PC with published throughput numbers (GLM: ~580 tok/s prefill, 26–30 tok/s decode; MiMo: up to 44 tok/s decode with speculative decoding) and near-FP8 fidelity metrics (KLD 0.151/0.071, ~90% top-1 agreement). Mixed EXL3 quantization packs and engine source are public on GitHub and Hugging Face. The hardware-tier claim — that Strix Halo APUs constitute a practical standard tier for large-MoE local inference — is supported by the builder's own measurements and subsequent independent user reports on sibling engines (Halogen, Gufo/Strixite, Radiance), but Kyojin's specific headline numbers remain builder-measured only; a git-history provenance challenge (alleged rewrite of vcruz305/exllamav3-amd) is unanswered, and no independent numeric replication of the 580/26–44 tok/s claims exists.

Why it matters to Scott

Converges with Scott's hardware-aware local inference doctrine (dev:concept.hardware-aware-local-inference): a new consumer-APU tier (Strix Halo, 128GB unified memory) now has multiple independent engines (Kyojin, Halogen, Gufo/Strixite, Radiance) that treat the accelerator's characteristics — ROCm path, mixed EXL3 quantization, MoE expert routing — as explicit runtime policy, exactly the pattern the doctrine predicts. The tier economics shift (300B-class MoE on a mini PC) bears on his gamepc substrate (dev:project.gamepc) and single-tenant appliance targets (dev:concept.single-tenant-ai-appliance, dev:project.openclaw).

dev:concept.hardware-aware-local-inferencedev:project.gamepcdev:concept.single-tenant-ai-appliancedev:project.openclawradar:atlas-local-inference-engineradar:amd-llama-cpp-prefill-speedupradar:adaptive-kv-cache-streamingradar:afm3-prompt-conditioned-pruning

queries asked of Scott's wikis

hardware-aware local inference doctrine Strix Halo AMD APU
MoE quantization EXL3 mixed-precision local inference economics
model sovereignty open weights local-first AI hardware tiers
ExLlamaV3 turboderp engine lineage ROCm gfx1151
local inference hardware tier strategy consumer APU vs GPU
RAG knowledge systems local model serving Strix Halo

Measured heat

now 31 pts/hpeak 166 pts/hcomments 9/hpeers p95momentum: cooling3 platformsage 242h

points/hour across evidence · reading as of 2026-10-12 02:59:37.977291+11:00 · deterministic, not a model opinion

How the heat travelled

10-01 14:00	⭐ origin echo-reconstructed	README: "Kyojin is the Yamz inference engine for AMD Strix Halo, built on ExLlamaV3 by turboderp. It adds a ROCm decode and prefill path for Yamz Labs (Reddit u/Yaniss916) on github (echo) · attributed from reddit.post.1wwocik	—
10-03 14:16	first on r/LocalLLaMA · published · +48.3h	Two ~300B MoE models, each on ONE 128 GB mini PC (AMD Strix Halo): GLM-5.3-Flash at ~580 tok/s prefill, MiMo-V2.6-Flash up to 44 tok/s decode. EXL3 weights + open ROCm engine Yaniss916	—
10-11 01:43	first on hacker news · published · +227.7h	GMKtec EVO-X3 Mini-PC Review Looking at a 128GB AMD Local AI Box lumpa	—
10-03 14:16	amplified on r/LocalLLaMA	reddit.post.1wwocik Yaniss916	peak 65 · 58 comments · 8% of case engagement
10-05 15:25	amplified on r/LocalLLaMA	reddit.post.1wybesy Yaniss916	peak 111 · 59 comments · 12% of case engagement
10-08 11:07	amplified on r/LocalLLaMA	reddit.post.1x0o2yy deepu105	peak 33 · 76 comments · 7% of case engagement
10-10 16:25	amplified on r/LocalLLaMA	reddit.post.1x2jf0y forevergeeks	peak 0 · 7 comments · 0% of case engagement
10-11 00:20	amplified on r/LocalLLaMA 👑	reddit.post.1x2u742 sayamss	peak 732 · 312 comments · 71% of case engagement
10-11 01:43	amplified on hacker news	hn.story.50039003 lumpa	peak 4 · 0 comments · 0% of case engagement
10-03 14:20	our radar first saw it · +48.3h	discovery anchor: reddit.post.1wwocik	—

pace: p79 vs 1188 stories at the 168h mark (now 242h old) — ahead of pennsylvania-datacenter-opposition (1.0x), behind llama-cpp-specdec-moe-fusion (1.0x)

Evidence (7) — ⭐ canonical anchor

source	object	author	score	comments
🟠 reddit	Two ~300B MoE models, each on ONE 128 GB mini PC (AMD Strix Halo): GLM-5.3-Flash at ~580 tok/s prefill, MiMo-V2.6-Flash up to 44 tok/s decode. EXL3 weights + open ROCm engine LocalLLaMA Retrieved article excerpt Open article · Retrieved 2026-10-03T14:24:42.030203+00:00 # Prove your humanity We’re committed to safety and security. But not for bots. Complete the challenge below and let us know you’re a real person. [Reddit, Inc. © "2026". All rights reserved.](https://www.redditinc.com/) [User Agreement](https://www.reddit.com/help/useragreement) [Privacy Policy](https://www.reddit.com/help/privacypolicy) [Content Policy](https://www.reddit.com/help/contentpolicy) [Help](https://support.reddithelp.com/hc/en-us)	Yaniss916	64	58
🟧 echo.github ⭐	README: "Kyojin is the Yamz inference engine for AMD Strix Halo, built on ExLlamaV3 by turboderp. It adds a ROCm decode and prefill path for	Yamz Labs (Reddit u/Yaniss916)	—	—
🟠 reddit	Qwen3.8-Flash-Next (125B) on a single Strix Halo mini PC: 44-59 tok/s with speculative decoding, ~1,400 tok/s prefill, engine is open LocalLLaMA	Yaniss916	111	59
🟠 reddit	Halogen + Qwen Flash Next keeps getting better LocalLLaMA	deepu105	33	76
🟠 reddit	qwen3.6-35b-a3b on the Strix Halo with 64GB of memory LocalLLaMA	forevergeeks	0	7
🟠 reddit	Building a 4x R9700 setup for a 10 person startup LocalLLaMA	sayamss	732	312
🟧 hn	GMKtec EVO-X3 Mini-PC Review Looking at a 128GB AMD Local AI Box	lumpa	4	0

Interpretation history

2026-10-11T02:46:03Z
grounded: converges/high — Converges with Scott's hardware-aware local inference doctrine (dev:concept.hardware-aware-local-inference): a new consumer-APU tier (Strix Halo, 128GB unified
2026-10-11T02:36:21Z
The Strix Halo hardware-tier hypothesis is now corroborated by four independent engines (Kyojin, Halogen, Gufo/Strixite family, Radiance fork), multiple user reports across memory tiers (64GB–128GB), a production 4×R9700 deployment serving 16 concurrent sessions, and a hardware review. Kyojin's specific headline numbers (580 tok/s prefill, 26–44 tok/s decode) and the git-history provenance challenge remain single-poster and unanswered. The case has crossed from 'watching' to 'corroborated' on the tier hypothesis; the engine-specific claims are still gated.
2026-10-11T02:33:19Z
evidence attached: hn.story.50039003 — Hardware review of 128GB AMD Strix Halo mini-PC corroborates the platform's emergence as a practical tier for large-MoE local inference.
2026-10-11T01:03:20Z
New 4×R9700 (Strix Halo) deployment with published throughput numbers (Radiance fork, Qwen 3.8 Next Flash/27B + DSV4 Flash) adds a third independent engine corroborating the hardware-tier hypothesis. Kyojin's specific headline numbers (580 tok/s prefill, 26–44 tok/s decode) and the git-history provenance challenge remain unreplicated/unanswered. Engagement on Kyojin threads fully cooled; measured heat shows high historical percentile (89) but low current rate (1.8 pts/h).
2026-10-11T00:36:53Z
evidence attached: reddit.post.1x2u742 — Real-world 4×R9700 (Strix Halo) deployment running Qwen 3.8 Next Flash/27B and DSV4 Flash with published throughput numbers; corroborates Strix Halo as a practical local-inference tier.
2026-10-10T20:10:33Z
Second independent user report (forevergeeks: Qwen 3.6-35B-A3B at ~80 tok/s on 64GB Strix Halo) corroborates the hardware-tier hypothesis — Strix Halo as a standard tier for large-MoE local inference — but still no independent numeric replication of Kyojin's headline numbers (580 tok/s prefill, 26–44 tok/s decode). The git-history provenance challenge on Kyojin remains unanswered; Lemonade's ROCm-vs-Vulkan finding and Gufo precedent still weigh against single-poster claims. Sibling-engine field (halogen, gufo, strixite, ciru, strix-llama) active with Halogen thread growing discussion (56→76 comments) but no comparative benchmarks. Engagement on Kyojin threads fully cooled.
2026-10-10T18:47:50Z
evidence attached: reddit.post.1x2jf0y — User corroboration: Qwen 3.6-35B-A3B runs at ~80 tok/s on 64GB Strix Halo mini PC, supporting the case's claim about Strix Halo as a tier for large-MoE local inference.
2026-10-08T14:01:13Z
New independent user report (deepu105) confirms Strix Halo 128GB runs Qwen 3.8 Flash Next (~177B MoE) at ~45 tok/s via Halogen engine — corroborating the hardware-tier hypothesis (Strix Halo as standard tier for large MoE) but not the specific Kyojin engine claims. The git-history provenance challenge on Kyojin remains unanswered; Lemonade's ROCm-vs-Vulkan finding and Gufo precedent still weigh against single-poster numbers. Sibling-engine field (halogen, gufo, strixite, ciru, strix-llama) active but no comparative benchmarks. Engagement fully cooled (1.5 pts/h, 50th percentile, steady momentum).
2026-10-08T12:41:57Z
evidence attached: reddit.post.1x0o2yy — User report corroborates Strix Halo (128GB) running Qwen 3.8 Flash Next (~177B MoE) at 45 tps via Halogen, supporting the case's hypothesis about Strix Halo as a standard tier for large-MoE local inference.
2026-10-08T02:41:37Z
Engagement has decayed to near-zero velocity (0 pts/h, 17th-percentile peer rate) six days in; the brief velocity spike on the Qwen3.8-Flash-Next thread has fully cooled. Still only one community (Reddit) plus the builder's GitHub — no independent numeric replication of the 580/26–44 tok/s headlines, the git-history provenance challenge remains unanswered, and the crowded sibling-engine field (halogen, gufo, strixite, ciru, strix-llama) has no comparative benchmarks. The sustained same-team release cadence is real but does not yet constitute corroboration.
2026-10-05T18:57:47Z
The same team's second release (Qwen3.8-Flash-Next 125B, 44-59 tok/s, ~1,400 tok/s prefill, weights out, engine open) turns the case from one bold claim into a sustained release line on the tier — structurally stronger for the Strix-Halo-as-standard-tier hypothesis, yet still entirely first-party: no independent benchmark, the git-history provenance challenge stands unanswered, and the new thread itself reveals a crowded sibling-engine field (halogen, gufo, strixite, ciru, strix-llama) whose comparative numbers will be the real test. Heat to medium on a fresh release thread moving at 90th-percentile cohort rate, not high: no second community, no cross-platform spread, no independent implementation.
2026-10-05T17:32:20Z
evidence attached: reddit.post.1wybesy — Same team extends the Kyojin/Strix Halo line to Qwen3.8-Flash-Next 125B with strong published throughput and released weights — direct evidence for the Strix-Halo-as-standard-tier hypothesis.
2026-10-04T15:50:09Z
First independent hands-on signal: a stranger (ionizing) is running the MiMo EXL3 pack on this tier, reports it works with decent tool calls and feels faster than expected — but no benchmarks, so the 580/26–44 headline numbers remain the builder's own. A thread challenge that the repo is a git-history rewrite rather than an honest fork of vcruz305/exllamav3-amd adds an open provenance question; seed→watching, and the brief 3.5x velocity spike has already decayed into a single engaged thread.
2026-10-03T14:47:49Z
origin walked (opencode/cheap-glm, conf 0.85): anchor reddit.post.1wwocik -> echo.github.61dded18c5 by Yamz Labs (Reddit u/Yaniss916)
2026-10-03T14:35:25Z
grounded: converges/medium — Converges with Scott's hardware-aware local-inference doctrine (dev:concept.hardware-aware-local-inference): a first-party gfx1151 engine that treats accelerato
2026-10-03T14:26:01Z
case created — First-party engine release with concrete KLD/throughput measurements on a Strix Halo tier no open case covers, and the builder's own post is the episode's origin.

Decision trace

10-12 00:33sensor_dirtyvelocity_spike
10-11 20:31sensor_dirtycomment_update
10-11 18:52attention_routeThe editor compared this story and chose to keep watching.
10-11 18:31sensor_dirtyvelocity_spike
10-11 18:04attention_routeMultiple independent replications and a hardware review since last briefing candidate, converging on Scott's hardware-aware local inference doctrine. Lead story for 18:00 Sydney briefing.
10-11 15:27sensor_dirtycomment_update
10-11 13:51attention_routeMaterial reprice update since last briefing candidate (rev 7) adds multiple independent replications and a hardware review, converging on Scott's hardware-aware local inference doctrine. Next bri
10-11 13:46attention_candidatematerial_reprice
10-11 13:46repriceThe Strix Halo hardware-tier hypothesis is now corroborated by four independent engines (Kyojin, Halogen, Gufo/Strixite family, Radiance fork), multiple user reports across memory tiers (64GB–128GB),
10-11 13:46groundConverges with Scott's hardware-aware local inference doctrine (dev:concept.hardware-aware-local-inference): a new consumer-APU tier (Strix Halo, 128GB unified memory) now has multiple independen
10-11 13:36attention_routeMultiple independent replications since last update; converges on Scott's hardware-aware local inference doctrine. Include in 18:00 briefing.
10-11 13:33attention_candidateattach
10-11 13:33attachHardware review of 128GB AMD Strix Halo mini-PC corroborates the platform's emergence as a practical tier for large-MoE local inference.
10-11 13:33propose_attachHardware review of 128GB AMD Strix Halo mini-PC corroborates the platform's emergence as a practical tier for large-MoE local inference.
10-11 12:31sensor_dirtyvelocity_spike
10-11 12:07attention_routeMultiple independent replications since last update; converges on Scott's hardware-aware local inference doctrine. Include in next briefing.
10-11 12:03attention_candidatematerial_reprice
10-11 12:03repriceNew 4×R9700 (Strix Halo) deployment with published throughput numbers (Radiance fork, Qwen 3.8 Next Flash/27B + DSV4 Flash) adds a third independent engine corroborating the hardware-tier hypothesis.
10-11 11:42attention_routeThe editor compared this story and chose to keep watching.
10-11 11:36attention_candidateattach
10-11 11:36attachReal-world 4×R9700 (Strix Halo) deployment running Qwen 3.8 Next Flash/27B and DSV4 Flash with published throughput numbers; corroborates Strix Halo as a practical local-inference tier.
10-11 11:36propose_attachReal-world 4×R9700 (Strix Halo) deployment running Qwen 3.8 Next Flash/27B and DSV4 Flash with published throughput numbers; corroborates Strix Halo as a practical local-inference tier.
10-11 10:03attention_routeThe editor compared this story and chose to keep watching.
10-11 07:17attention_routeConverges with Scott's hardware-aware local inference doctrine; a single 128GB mini PC running 300B-class MoE at 26-44 tok/s decode would change his local zoo serving capacity. Independent engine
10-11 07:10attention_candidatematerial_reprice
10-11 07:10repriceSecond independent user report (forevergeeks: Qwen 3.6-35B-A3B at ~80 tok/s on 64GB Strix Halo) corroborates the hardware-tier hypothesis — Strix Halo as a standard tier for large-MoE local inference
10-11 05:54attention_routeThe editor compared this story and chose to keep watching.
10-11 05:47attention_candidateattach
10-11 05:47attachUser corroboration: Qwen 3.6-35B-A3B runs at ~80 tok/s on 64GB Strix Halo mini PC, supporting the case's claim about Strix Halo as a tier for large-MoE local inference.
10-11 05:40propose_attachUser corroboration: Qwen 3.6-35B-A3B runs at ~80 tok/s on 64GB Strix Halo mini PC, supporting the case's claim about Strix Halo as a tier for large-MoE local inference.
10-09 20:31sensor_dirtycomment_update
10-09 09:34sensor_dirtycomment_update
10-09 04:35sensor_dirtyvelocity_spike
10-09 01:06attention_routeThe editor compared this story and chose to keep watching.
10-09 01:01attention_candidatematerial_reprice
10-09 01:01repriceNew independent user report (deepu105) confirms Strix Halo 128GB runs Qwen 3.8 Flash Next (~177B MoE) at ~45 tok/s via Halogen engine — corroborating the hardware-tier hypothesis (Strix Halo as standa
10-09 00:03attention_routeThe editor compared this story and chose to keep watching.
10-08 23:41attention_candidateattach
10-08 23:41attachUser report corroborates Strix Halo (128GB) running Qwen 3.8 Flash Next (~177B MoE) at 45 tps via Halogen, supporting the case's hypothesis about Strix Halo as a standard tier for large-MoE local
10-08 23:36propose_attachUser report corroborates Strix Halo (128GB) running Qwen 3.8 Flash Next (~177B MoE) at 45 tps via Halogen, supporting the case's hypothesis about Strix Halo as a standard tier for large-MoE local