Magnitude (YC S25) claims its open-source engine tunes kernels on-device to run open models up to 2x faster than llama.cpp (92% faster Metal decode in its benchmarks), and cross-hardware replication plus adoption by local-agent builders would establish self-optimizing serving as a practical local-inference alternative.

state: watchingheat: mediumuncertainty: mediumconvergesscott: highlocal-inference inference-engines agent-harnessesMagnitude (YC S25)

Surfaced 2026-10-02T11:01:23Z — Open source inference engine for agents that optimizes itself for your exact hardware; compiles and tunes kernels on-device so open models r — First independent replication arrived and went against the headline: an HN user benchmarked six Qwen models on an M5 Max 128GB and reports the engine 'adds next to nothing' — one terse machine with no posted numbers, so the 2x/92% claims stay announcement-class but now carry a first negative data point. The Reddit crosspost belatedly caught a real wave (6→89 pts, 24 comments) with matching skepticism (the '2x' is Metal-best-case vs +19% CUDA; 2-devs/2-months immaturity, no multi-GPU) before cooling; the case settles into a quiet wait for MLX baselines and further replications.

What is this?

Magnitude (YC S25) is an open-source local inference engine (GitHub magnitudedev/magnitude, ~5k stars, TypeScript/bun monorepo) that profiles the user's machine, recommends open models that fit it, and compiles/tunes kernels on-device for that exact hardware — Apple Silicon, NVIDIA, AMD, or CPU-only — with one-click connection to existing coding agents (it names Pi, OpenCode, Hermes). Its Launch HN claims up to 2x faster than llama.cpp with 92% faster Metal decode in its own benchmarks; the supplied snippets do not independently corroborate those numbers, so they rest on the launch's self-reporting pending community replication. Note a wrinkle: an Aug 2026 Medium piece described Magnitude as a fully-local coding agent with its own inference engine rather than an engine alone, so the snippets are ambiguous about whether the engine is the whole product or the serving layer of a broader agent story.

Why it matters to Scott

Converges with dev:concept.hardware-aware-local-inference: a YC-backed team has independently productized Scott's 'treat compilation as explicit runtime policy' stance into a self-optimizing engine, and its 2x/92%-Metal head-to-head is aimed directly at the llama.cpp substrate under his Ollama endpoint on gamepc — so the obvious next act is replicating the claim there, with the Evidence Class Ladder dictating the 2x number stays at announcement-class until independently benchmarked. The one-click Pi/OpenCode hook also continues the radar's llama.cpp-as-agent-serving thread rather than being just another speedup claim.

dev:concept.hardware-aware-local-inferencedev:technology.ollamadev:project.gamepcip:concept.evidence-class-ladderradar:concept.local-firstradar:opengemm-b200-kernel-releaseradar:kairo-measured-cuda-graph-routingradar:rooflang-inference-architecture-searchradar:pi-native-llama-cpp-runtimeradar:aa-agentperf-local-benchmarkradar:adaptive-speculative-decoding-300-gpu

queries asked of Scott's wikis

local inference server llama.cpp integration in agent harness
prefill vs decode agent latency bottleneck measurements
kernel auto-tuning / hardware-specific compilation positions
local-first agent stack privacy and sovereignty arguments
benchmark verification of speedup claims against incumbents
OpenAI-compatible endpoint support in my harness model providers

Measured heat

now 0 pts/hpeak 64 pts/hcomments 0/hpeers p14momentum: steady3 platformsage 290h

points/hour across evidence · reading as of 2026-10-12 02:59:37.977291+11:00 · deterministic, not a model opinion

How the heat travelled

09-29 14:00	⭐ origin echo-reconstructed	Open source inference engine for agents that optimizes itself for your exact hardware; compiles and tunes kernels on-device so open models r magnitudedev (Anders and Tom) on github (echo) · attributed from hn.story.49911995	—
09-30 17:37	first on hacker news · published · +27.6h	Launch HN: Magnitude (YC S25) – Self-optimizing inference engine for agents anerli	—
09-30 22:46	first on r/LocalLLaMA · published · +32.8h	Open source inference engine (like LM Studio or Unsloth Desktop) that optimizes itself for your exact hardware. Compiles and tunes its kernels on your device, so open models run up to 2x faster than llama.cpp. Works on Apple Silicon, NVIDIA, AMD or nothing but a CPU. paranoidray	—
09-30 17:37	amplified on hacker news 👑	hn.story.49911995 anerli	peak 194 · 99 comments · 75% of case engagement
09-30 22:46	amplified on r/LocalLLaMA	reddit.post.1wuj70v paranoidray	peak 96 · 24 comments · 17% of case engagement
10-03 01:19	amplified on r/LocalLLaMA	reddit.post.1wwamjw cezarducatti	peak 6 · 55 comments · 9% of case engagement
09-30 18:21	our radar first saw it · +28.4h	discovery anchor: hn.story.49911995	—
10-02 11:01	reached heat=high · +69.0h · via ledger	—	—

pace: p83 vs 1188 stories at the 168h mark (now 290h old) — ahead of gpt6-prompt-cache-controls (1.0x), behind ftc-frontier-ai-probe (1.0x)

Evidence (4) — ⭐ canonical anchor

source	object	author	score	comments
🟧 hn	Launch HN: Magnitude (YC S25) – Self-optimizing inference engine for agents Retrieved article excerpt Open article · Retrieved 2026-09-30T18:42:56.779852+00:00 Magnitude icon # Magnitude Run open models as fast as your hardware allows [Download Magnitude](https://magnitude.dev/download) [Documentation](https://docs.magnitude.dev) [Discord](https://discord.gg/EHt48pPWdC) [Follow Magnitude on Twitter](https://x.com/usemagnitude) [GitHub Repo stars](https://github.com/magnitudedev/magnitude/stargazers) Magnitude is an open source inference engine for agents that optimizes itself for your exact hardware. It compiles and tunes its kernels on your device, so open models run up to 2x faster than llama.cpp. One click connects the agent you already use (Pi, OpenCode, Hermes, Codex, and more). Works on Apple Silicon, NVIDIA, AMD, or nothing but a CPU. [Download Magnitude for macOS, Windows, or Linux](https://magnitude.dev/download) ⭐ Help us reach more developers and grow the Magnitude community. Star this repo! demo-9-29.mp4 [ ](https://private-user-images.githubusercontent.com/32600629/661966322-983328c8-93e8-4360-bfef-11e93ff76035.mp4?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3OTA3OTQwNzYsIm5iZiI6MTc5MDc5Mzc3NiwicGF0aCI6Ii8zMjYwMDYyOS82NjE5NjYzMjItOTgzMzI4YzgtOTNlOC00MzYwLWJmZWYtMTFlOTNmZjc2MDM1Lm1wND9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNjA5MzAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjYwOTMwVDE4NDI1NlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWY1MTAyM2YzZTFkYjZmMWIyYzEzOWI3YWM0MjdlYWI5ZmY1MzM0OGJkZjI0NTM1MzMwZWY4MThlYzg3NWRkOTAmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JnJlc3BvbnNlLWNvbnRlbnQtdHlwZT12aWRlbyUyRm1wNCJ9.4QKShJeSr_gDRSrApB3X2iBMVIrmuxZQ5dsvjpa-34I) ## Get started 1. [Download Magnitude](https://magnitude.dev/download), install it, and open the app. 2. Choose a recommended model in Discover and download it. 3. Connect your agent in Connections and start using it. The desktop app includes the `magnitude` CLI. No separate installation is needed. ## Why Magnitude? - Up to 2x faster than llama.cpp: 92% faster decode on Metal, 19% on CUDA - Tuned on your device: kernels are tuned on your hardware before a model runs - Built for the best models: hand-optimized kernels for popular open-weight families - Memory that flexes: 27% less memory per agent, freed when agents stop - Fast concurrent sessions: sessions share prefix caches to prevent slowdown - Works with your agent: one click to connect Pi, OpenCode, Hermes, Codex, and more - Free, private, open source: no token costs, nothing leaves your machine, Apache 2.0 ## Up to 2x faster than llama.cpp Magnitude vs llama.cpp: 9% faster prefill and 92% faster decode on Metal, 23% faster prefill and 19% faster decode on CUDA ## FAQ ### What is Magnitude? An open source inference engine that optimizes itself for your hardware. It ships as a desktop app that runs open models and connects them to the agent you already use. ### How is it faster than llama.cpp, Ollama, or LM Studio? They ship kernels precompiled for broad classes of hardware. Magnitude compiles and tunes its kernels on your actual device before a model runs, so they fit your exact chip. [See the benchmarks against llama.cpp.](https://github.com/magnitudedev/magnitude#up-to-2x-faster-than-llamacpp) ### What hardware do I need? Any Apple Silicon, NVIDIA, or AMD GPU, or nothing but a CPU. There is no fixed minimum. Smaller machines run smaller models, and more memory lets you run larger ones. ### What operating systems does it support? macOS, Linux, and Windows. ### Which models does it support? See the full list at [magnitude.dev/models](https://magnitude.dev/models). We write optimized kernels for the most popular open-weight families, which is how we beat generalist engines. ### Which agents work with it? One click connects Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline. Anything else works through the OpenAI-compatible API. ### Is it private? Yes. Prompts, files, and models stay on your machine. No internet needed once a model is downloaded.	anerli	194	99
🟧 echo.github ⭐	Open source inference engine for agents that optimizes itself for your exact hardware; compiles and tunes kernels on-device so open models r	magnitudedev (Anders and Tom)	—	—
🟠 reddit	Open source inference engine (like LM Studio or Unsloth Desktop) that optimizes itself for your exact hardware. Compiles and tunes its kernels on your device, so open models run up to 2x faster than llama.cpp. Works on Apple Silicon, NVIDIA, AMD or nothing but a CPU. LocalLLaMA	paranoidray	96	23
🟠 reddit	Strata - RTX 3090 - 128 Ram - Qwen 3.8 Flash Next LocalLLaMA	cezarducatti	3	55

Interpretation history

2026-10-03T05:20:29Z
The attach that looked like the case's resolution signal is a mis-read: reddit.post.1wwamjw benchmarks 'Strata' — a separate, itself-unverified self-optimizing engine — not Magnitude, so cross-hardware replication of Magnitude's claims remains unmet and the case still holds exactly one weak third-party datum (bythreads' terse negative M5 Max test). With every thread static for ~2 days and velocity at floor (0.33 pts/h, 44th percentile), heat drops high→low despite valve eligibility: the three-platform spread is peak-echo, the periphery is not expanding.
2026-10-03T01:23:53Z
evidence attached: reddit.post.1wwamjw — Independent RTX 3090 replication of Strata reporting ~2-2.6x PP and ~1.7-2.6x TG over llama.cpp with real OpenCode use — exactly the cross-hardware replication and agent-builder adoption the Magnitude case's resolution criteria call for.
2026-10-02T11:01:19Z
magnitude valve eligible (multi-platform, top-decile engagement) and never alerted; deterministic escalation to deliver
2026-10-01T00:59:36Z
Launch tail, not breakout: the velocity spike fired against a floor baseline and the raw deltas (+3 pts, +2 comments on a story decaying from a 52.9/h peak; Reddit crosspost static at 6 pts, 0.76 ratio) say the launch is aging, not spreading — three evidence objects remain one announcement propagated, so no independent corroboration yet. The HN thread's substance is methodology skepticism (llama.cpp config variance, a demanded MLX comparison) plus one user flagging Magnitude's own UI speed estimates for Qwen 3.8 Q8 as poor below 128K context; the case now waits on third-party replication rather than a live confirm-or-debunk window, keeping the 2x/92% claims at announcement-class.
2026-10-01T00:29:45Z
evidence attached: reddit.post.1wuj70v — shared external link with case evidence
2026-09-30T19:25:30Z
grounded: converges/high — Converges with dev:concept.hardware-aware-local-inference: a YC-backed team has independently productized Scott's 'treat compilation as explicit runtime policy'
2026-09-30T19:18:22Z
case created — Live Launch HN with a quantified head-to-head claim against llama.cpp that the community will benchmark or debunk within hours.

Decision trace

10-08 15:34review_screenjev screen: no material development (noul=0.25)
10-05 19:38review_screenjev screen: no material development (noul=0.20)
10-04 18:50review_screenThe diff is additional same-thread chatter about Strata, which the assessment already corrected as a different engine counting at most as weak category-level signal for Magnitude. The new comments (an
10-04 18:49review_screenjev screen borderline (noul=0.57) — luna review
10-04 04:20sensor_dirtycomment_update
10-03 19:20sensor_dirtycomment_update
10-03 15:20repriceThe attach that looked like the case's resolution signal is a mis-read: reddit.post.1wwamjw benchmarks 'Strata' — a separate, itself-unverified self-optimizing engine — not Magnitude, s
10-03 12:21sensor_dirtycomment_update
10-03 11:23attachIndependent RTX 3090 replication of Strata reporting ~2-2.6x PP and ~1.7-2.6x TG over llama.cpp with real OpenCode use — exactly the cross-hardware replication and agent-builder adoption the Magnitude
10-03 11:22propose_attachIndependent RTX 3090 replication of Strata reporting ~2-2.6x PP and ~1.7-2.6x TG over llama.cpp with real OpenCode use — exactly the cross-hardware replication and agent-builder adoption the Magnitude
10-02 21:01pushOpen source inference engine for agents that optimizes itself for your exact hardware; compiles and tunes kernels on-device so open models r — First independent replication arrived and went against th
10-02 21:01repriceFirst independent replication arrived and went against the headline: an HN user benchmarked six Qwen models on an M5 Max 128GB and reports the engine 'adds next to nothing' — one terse machi
10-02 21:01alert_heldOpen source inference engine for agents that optimizes itself for your exact hardware; compiles and tunes kernels on-device so open models r — First independent replication arrived and went against th
10-02 21:01alert_routeOpen source inference engine for agents that optimizes itself for your exact hardware; compiles and tunes kernels on-device so open models r — First independent replication arrived and went against th
10-02 16:21sensor_dirtycomment_update
10-02 02:22sensor_dirtycomment_update
10-02 00:21sensor_dirtyvelocity_spike
10-01 22:21sensor_dirtycomment_update
10-01 18:20sensor_dirtycomment_update
10-01 17:21sensor_dirtyvelocity_spike
10-01 10:59repriceLaunch tail, not breakout: the velocity spike fired against a floor baseline and the raw deltas (+3 pts, +2 comments on a story decaying from a 52.9/h peak; Reddit crosspost static at 6 pts, 0.76 rati
10-01 10:29attachshared external link with case evidence
10-01 10:22sensor_dirtyvelocity_spike
10-01 10:22propose_attachshared external link with case evidence
10-01 07:22sensor_dirtycomment_update
10-01 05:25groundConverges with dev:concept.hardware-aware-local-inference: a YC-backed team has independently productized Scott's 'treat compilation as explicit runtime policy' stance into a self-optim
10-01 05:18createLive Launch HN with a quantified head-to-head claim against llama.cpp that the community will benchmark or debunk within hours.