2026-10-11 18:00 UTC

moe-streaming

band: coolmomentum: stable score: 0.087
temperature history

Episodes (2)

Independent benchmarks will determine whether Krasis can serve the 397B-parameter Ornith MoE interactively at Q4 on a single 96GB workstation GPU by dynamically streaming experts from system RAM while sustaining roughly 20–24 tokens per second.
expirednovelscott: none
NakliTechie claims his released MIT LocalMind — a static, no-install WebGPU page whose engines stream MoE expert weights from disk while generating — runs Gemma 4 26B-A4B and a 37GB Qwen3.6 MoE larger than a 24GB Mac's RAM entirely in a browser tab at llama.cpp-comparable output, and independent replication and adoption would establish the zero-install browser tab as a practical tier for over-RAM local inference.
seedconvergesscott: high