Independent testing will determine whether llama.cpp’s merged Bonsai and ternary-model support enables correct, performant local inference for 1-bit and 1.58-bit Bonsai checkpoints across common backends.

state: expiredheat: lowuncertainty: highknownscott: mediumopen-models local-inference llama-cppPrism AIllama.cpp

What is this?

PrismML’s Bonsai family uses extremely low-bit weights—1-bit and ternary/approximately 1.58-bit—to reduce model size for local inference, with PrismML reporting competitive benchmark retention. Community snippets show promising performance, including roughly 106 tokens/second in one configuration, but also weak results on some reasoning tests. Backend compatibility remains unclear and format-specific: one llama.cpp discussion says a Bonsai Q2_0 checkpoint works only with PrismML’s fork while a g64 variant is needed elsewhere, and the supplied snippets do not firmly establish that complete support has merged into stock llama.cpp or works correctly across common backends.

Why it matters to Scott

The radar already tracks the underlying Bonsai extreme-quantization claim in `radar:bonsai-extreme-quantization`; this case extends that open story into stock llama.cpp compatibility and backend validation rather than introducing a new position. It matters to Scott because verified support could make sub-2-bit checkpoints deployable on his hardware-aware local inference substrate, while format-specific failures would reinforce the need for repeatable capability audits before adoption.

dev:concept.hardware-aware-local-inferenceip:concept.capability-auditip:concept.evaluation-driven-developmentdev:project.gamepcradar:bonsai-extreme-quantizationradar:concept.llama-cppradar:concept.quantization

queries asked of Scott's wikis

sub-2-bit local inference economics
llama.cpp backend compatibility and quantization
independent evaluation of low-bit model quality
local model format fragmentation
on-device inference performance tradeoffs
grammar-constrained decoding for small local models

Measured heat

no measured readings yet — the hourly heat pass fills this in

How the heat travelled

no chain yet — the hourly chain pass fills this in

Evidence (2) — ⭐ canonical anchor

source	object	author	score	comments
🟠 reddit ⭐	1-bit / 2-bit / Ternary / Bitnet Models - Updates & Tracking LocalLLaMA	pmttyji	63	13
🟠 reddit	Beating the vendor's official runtime on free ARM cores: a from-scratch engine for ternary 8B models (decode +14%, prefill +55%). Live demo included. LocalLLaMA	Annual_Manner_5901	4	3

Interpretation history

2026-08-24T06:22:26Z
The validation window has faded without a reproducible stock llama.cpp run, backend matrix, or controlled benchmark. Preserve the merge as historical implementation context, but reopen only if concrete independent testing appears.
2026-08-22T05:30:51Z
The refreshed comment adds methodological scrutiny rather than validation: key benchmark controls such as thread count, batch size, and prompt length remain unspecified. It reinforces that neither the ARM performance claim nor stock llama.cpp support is independently reproducible yet.
2026-08-21T21:29:47Z
The ARM benchmark shows that Bonsai ternary inference can be optimized beyond PrismML’s fork, but it neither validates stock llama.cpp’s merged support nor provides an independently reproducible artifact. The case remains a backend-compatibility and correctness question awaiting a documented stock llama.cpp run.
2026-08-21T20:23:15Z
evidence attached: reddit.post.1vuq0oa — A from-scratch runtime benchmark supplies useful independent context on the practical performance of Bonsai ternary models on low-cost ARM hardware.
2026-08-21T08:32:20Z
No new evidence arrived within the staleness window, so the case remains an unvalidated implementation claim rather than a developing adoption signal. Keep it open but cold until a reproducible stock llama.cpp run reports checkpoint format, backend, correctness, quality, and speed.
2026-08-19T08:25:31Z
Anecdotal use of Bonsai 27B suggests some practical utility, but it does not identify stock llama.cpp, a backend, checkpoint format, correctness, or measured performance. The case still lacks independent validation of the merged support and its meaning is unchanged.
2026-08-19T03:31:28Z
The refreshed discussion adds skepticism and deployment questions but no independent run confirming correctness, backend compatibility, quality, or speed. The merge remains a concrete test opportunity, while the validation hypothesis is unchanged and now cooler pending actual results.
2026-08-18T19:39:35Z
No independent compatibility, correctness, quality, or performance testing has arrived; the score increase is only amplification of the already-known merge claim. The case remains a fresh implementation event awaiting substantive validation across common backends.
2026-08-18T19:30:34Z
grounded: known/medium — The radar already tracks the underlying Bonsai extreme-quantization claim in `radar:bonsai-extreme-quantization`; this case extends that open story into stock l
2026-08-18T19:26:00Z
case created — Merged mainline support is a concrete local-inference event whose compatibility, speed, and model quality remain to be validated.

Decision trace

08-24 16:22expireThe validation window has faded without a reproducible stock llama.cpp run, backend matrix, or controlled benchmark. Preserve the merge as historical implementation context, but reopen only if concret
08-24 16:22alert_silentThe only delta is elapsed staleness; no new evidence or consequential event occurred, so there is nothing to surface before a routine briefing.
08-24 16:22alert_routeThe only delta is elapsed staleness; no new evidence or consequential event occurred, so there is nothing to surface before a routine briefing.
08-22 15:30repriceThe refreshed comment adds methodological scrutiny rather than validation: key benchmark controls such as thread count, batch size, and prompt length remain unspecified. It reinforces that neither the
08-22 15:30alert_silentThe new delta is only an unanswered request for benchmark controls; it supplies no stock llama.cpp test, artifact, or independent result and can wait for routine review.
08-22 15:30alert_routeThe new delta is only an unanswered request for benchmark controls; it supplies no stock llama.cpp test, artifact, or independent result and can wait for routine review.
08-22 14:21sensor_dirtycomment_update
08-22 07:29repriceThe ARM benchmark shows that Bonsai ternary inference can be optimized beyond PrismML’s fork, but it neither validates stock llama.cpp’s merged support nor provides an independently reproducible artif
08-22 07:29alert_silentThis is useful adjacent performance context, but the developer-authored claim lacks linked code or independent reproduction and does not test the stock llama.cpp implementation central to the case. No
08-22 07:29alert_routeThis is useful adjacent performance context, but the developer-authored claim lacks linked code or independent reproduction and does not test the stock llama.cpp implementation central to the case. No
08-22 06:24alert_silentA lone developer-authored Reddit benchmark claims a new engine beats PrismML’s llama.cpp fork on a specific low-end ARM setup, but provides no linked code, artifact, or independent reproduction and do
08-22 06:24surface_candidateA lone developer-authored Reddit benchmark claims a new engine beats PrismML’s llama.cpp fork on a specific low-end ARM setup, but provides no linked code, artifact, or independent reproduction and do
08-22 06:24alert_routeA lone developer-authored Reddit benchmark claims a new engine beats PrismML’s llama.cpp fork on a specific low-end ARM setup, but provides no linked code, artifact, or independent reproduction and do
08-22 06:23attachA from-scratch runtime benchmark supplies useful independent context on the practical performance of Bonsai ternary models on low-cost ARM hardware.
08-22 06:22propose_attachA from-scratch runtime benchmark supplies useful independent context on the practical performance of Bonsai ternary models on low-cost ARM hardware.
08-21 18:32repriceNo new evidence arrived within the staleness window, so the case remains an unvalidated implementation claim rather than a developing adoption signal. Keep it open but cold until a reproducible stock
08-21 18:32alert_silentThere is no consequential new delta to route; topic heat cannot substitute for an independent compatibility or benchmark result.
08-21 18:32alert_routeThere is no consequential new delta to route; topic heat cannot substitute for an independent compatibility or benchmark result.
08-19 22:21sensor_dirtyengagement_update
08-19 18:25repriceAnecdotal use of Bonsai 27B suggests some practical utility, but it does not identify stock llama.cpp, a backend, checkpoint format, correctness, or measured performance. The case still lacks independ
08-19 18:25alert_silentThe refreshed comments add only an underspecified user testimonial, not a reproducible compatibility or benchmark result; it can wait for an independent stock-llama.cpp run with backend, format, quali
08-19 18:25alert_routeThe refreshed comments add only an underspecified user testimonial, not a reproducible compatibility or benchmark result; it can wait for an independent stock-llama.cpp run with backend, format, quali
08-19 18:21sensor_dirtycomment_update
08-19 13:31repriceThe refreshed discussion adds skepticism and deployment questions but no independent run confirming correctness, backend compatibility, quality, or speed. The merge remains a concrete test opportunity
08-19 13:31alert_silentThe new comments are discussion rather than validation and add no consequential fact beyond the already-routed merge event; this can wait for an independent benchmark or compatibility report.
08-19 13:31alert_routeThe new comments are discussion rather than validation and add no consequential fact beyond the already-routed merge event; this can wait for an independent benchmark or compatibility report.
08-19 13:21sensor_dirtycomment_update
08-19 10:21sensor_dirtyengagement_update
08-19 08:21sensor_dirtyengagement_update
08-19 07:21sensor_dirtyengagement_update
08-19 06:21sensor_dirtyengagement_update
08-19 05:39repriceNo independent compatibility, correctness, quality, or performance testing has arrived; the score increase is only amplification of the already-known merge claim. The case remains a fresh implementati
08-19 05:39alert_silentThe only new delta is modest engagement without comments or new evidence, so it adds no actionable information beyond the previously routed merge event and can wait for testing results.
08-19 05:39alert_routeThe only new delta is modest engagement without comments or new evidence, so it adds no actionable information beyond the previously routed merge event and can wait for testing results.
08-19 05:38alert_shadowThis appears to move 1.58-bit Bonsai checkpoints from format-specific forks toward stock llama.cpp on two common GPU backends, creating a practical test opportunity for Scott’s local-inference stack n
08-19 05:38alert_routeThis appears to move 1.58-bit Bonsai checkpoints from format-specific forks toward stock llama.cpp on two common GPU backends, creating a practical test opportunity for Scott’s local-inference stack n
08-19 05:30groundThe radar already tracks the underlying Bonsai extreme-quantization claim in `radar:bonsai-extreme-quantization`; this case extends that open story into stock llama.cpp compatibility and backend valid
08-19 05:26createMerged mainline support is a concrete local-inference event whose compatibility, speed, and model quality remain to be validated.