Upstream review and independent benchmarks will determine whether correcting llama.cpp’s inflated MTP buffer reservations materially expands usable context on memory-constrained AMD systems without inference regressions.

state: resolvedheat: lowuncertainty: highknownscott: lowllama-cpp local-inference context-window memory-fittingllama.cpp AMD Qwen

What is this?

The case concerns a reported llama.cpp memory-allocation issue in which the secondary context created for multi-token prediction (MTP) reserves more VRAM than necessary, reducing the context length that can fit on memory-constrained systems. A llama.cpp GitHub issue independently documents related behavior: the fitting logic reduces the main context, but the MTP draft context then initializes using the model’s larger native context and can cause an out-of-memory failure. The supplied evidence titles report an AMD ROCm benchmark improvement from 64,256 to 149,248 tokens for Qwen 27B after a patch, but the snippets do not directly verify those figures, upstream acceptance, or the absence of inference regressions.

Why it matters to Scott

The radar already tracks the same llama.cpp MTP memory-overhead development in `radar:llama-cpp-mtp-default-memory-regression`; this case adds a specific AMD/Qwen context-capacity claim awaiting validation rather than a new position. It touches Scott’s hardware-aware local-inference and evaluation-gated change practices, but the hits establish only an NVIDIA/CUDA local stack—not active AMD ROCm or llama.cpp use—so it is adjacent rather than action-changing.

dev:concept.hardware-aware-local-inferenceip:concept.evaluation-driven-developmentradar:llama-cpp-mtp-default-memory-regressionradar:concept.rocmradar:concept.speculative-decodingradar:concept.long-context-inference

queries asked of Scott's wikis

local inference memory-fitting strategy
context length versus VRAM economics
speculative decoding memory overhead
AMD ROCm local model support
inference benchmark and regression methodology
runtime context allocation architecture

Measured heat

no measured readings yet — the hourly heat pass fills this in

How the heat travelled

no chain yet — the hourly chain pass fills this in

Evidence (2) — ⭐ canonical anchor

source	object	author	score	comments
🟠 reddit	AMD llama.cpp: reducing MTP buffer overhead gave me 64K → 149K context for Qwen 27B LocalLLaMA	ea_man	75	28
🟧 echo.other ⭐	Primary benchmark log containing the figures repeated by the Reddit post: “ROCm mainline 64,256” versus “ROCm patched 149,248” context token	eaman (session attribution: “Sol”)	—	—

Interpretation history

2026-08-15T05:28:17Z
Repeated checks have produced only minor engagement and no upstream PR, controlled benchmark, or regression evidence. This duplicate AMD/Qwen claim should be retired into the broader llama.cpp MTP memory-regression case, where any substantive upstream milestone can be tracked.
2026-08-13T05:22:35Z
The staleness check adds no upstream action, controlled reproduction, or regression evidence, so the AMD/Qwen capacity claim remains an unresolved single-system result with an untested adaptation. Given its overlap with the broader MTP memory-regression case, further review should wait for a concrete upstream or benchmarking milestone.
2026-08-11T04:28:09Z
The comment refresh adds no upstream review, benchmark methodology, regression testing, or independent controlled reproduction. It is repetitive amplification of the already-priced patch adaptation, leaving the claimed AMD/Qwen context gains plausible but uncorroborated.
2026-08-09T17:28:03Z
The refreshed comments add no upstream review, controlled reproduction, or regression results beyond the already-priced untested adaptation. The large AMD/Qwen context-capacity claim remains plausible but uncorroborated and materially overlaps the existing MTP memory-regression case.
2026-08-09T15:30:13Z
The refreshed discussion adds no evidence beyond the already-priced, untested second-user adaptation. The case still awaits an upstream PR or review, controlled independent benchmarks, and regression testing before the reported memory-fitting gains can be treated as corroborated.
2026-08-09T14:29:10Z
A second user adapted the patch to current llama.cpp and reports a smaller context-capacity gain, providing tentative implementation-level support beyond the original system. The reproduction is undocumented and explicitly untested, so upstream review, controlled benchmarks, and regression checks remain necessary.
2026-08-09T11:36:22Z
No upstream review, merge, independent reproduction, or regression testing has arrived; the small engagement change is repetitive attention and leaves the AMD/Qwen capacity claim an unvalidated single-system result.
2026-08-09T11:27:00Z
grounded: known/low — The radar already tracks the same llama.cpp MTP memory-overhead development in `radar:llama-cpp-mtp-default-memory-regression`; this case adds a specific AMD/Qw
2026-08-09T11:24:20Z
origin walked (codex/luna, conf 0.94): anchor reddit.post.1vjmay5 -> echo.other.8a3ef852d9 by eaman (session attribution: “Sol”)
2026-08-09T11:22:50Z
case created — The report supplies comparative ROCm and Vulkan measurements indicating a specific auto-fit defect with large practical context-capacity effects.

Decision trace

08-15 15:28resolveRepeated checks have produced only minor engagement and no upstream PR, controlled benchmark, or regression evidence. This duplicate AMD/Qwen claim should be retired into the broader llama.cpp MTP mem
08-15 15:28alert_silentThe new delta is engagement-only and changes neither confidence nor practical significance; any future upstream action belongs in the broader MTP memory-regression case.
08-15 15:28alert_routeThe new delta is engagement-only and changes neither confidence nor practical significance; any future upstream action belongs in the broader MTP memory-regression case.
08-13 15:22repriceThe staleness check adds no upstream action, controlled reproduction, or regression evidence, so the AMD/Qwen capacity claim remains an unresolved single-system result with an untested adaptation. Giv
08-13 15:22alert_silentNo consequential event occurred; elapsed silence does not warrant an alert, and the case can wait for an upstream PR, review, or controlled benchmark.
08-13 15:22alert_routeNo consequential event occurred; elapsed silence does not warrant an alert, and the case can wait for an upstream PR, review, or controlled benchmark.
08-11 14:28repriceThe comment refresh adds no upstream review, benchmark methodology, regression testing, or independent controlled reproduction. It is repetitive amplification of the already-priced patch adaptation, l
08-11 14:28alert_silentNo consequential new fact occurred; engagement and refreshed discussion alone do not justify interrupting Scott before the next briefing.
08-11 14:28alert_routeNo consequential new fact occurred; engagement and refreshed discussion alone do not justify interrupting Scott before the next briefing.
08-10 18:21sensor_dirtyengagement_update
08-10 10:21sensor_dirtyengagement_update
08-10 03:28repriceThe refreshed comments add no upstream review, controlled reproduction, or regression results beyond the already-priced untested adaptation. The large AMD/Qwen context-capacity claim remains plausible
08-10 03:28alert_silentThe new delta is repetitive discussion without a validated benchmark, upstream action, or implementation milestone, so it can wait for routine review.
08-10 03:28alert_routeThe new delta is repetitive discussion without a validated benchmark, upstream action, or implementation milestone, so it can wait for routine review.
08-10 03:21sensor_dirtycomment_update
08-10 02:21sensor_dirtyengagement_update
08-10 01:30repriceThe refreshed discussion adds no evidence beyond the already-priced, untested second-user adaptation. The case still awaits an upstream PR or review, controlled independent benchmarks, and regression
08-10 01:30alert_silentThis is repetitive discussion rather than a new implementation, upstream decision, or validated benchmark, so it can wait for the next briefing.
08-10 01:30alert_routeThis is repetitive discussion rather than a new implementation, upstream decision, or validated benchmark, so it can wait for the next briefing.
08-10 01:21sensor_dirtycomment_update
08-10 00:29repriceA second user adapted the patch to current llama.cpp and reports a smaller context-capacity gain, providing tentative implementation-level support beyond the original system. The reproduction is undoc
08-10 00:29alert_silentThe new comment modestly strengthens plausibility but is an informal, incomplete reproduction rather than an upstream change or validated benchmark; it can wait for the next briefing.
08-10 00:29alert_routeThe new comment modestly strengthens plausibility but is an informal, incomplete reproduction rather than an upstream change or validated benchmark; it can wait for the next briefing.
08-10 00:21sensor_dirtycomment_update
08-09 23:21sensor_dirtyengagement_update
08-09 22:21sensor_dirtyengagement_update
08-09 21:36repriceNo upstream review, merge, independent reproduction, or regression testing has arrived; the small engagement change is repetitive attention and leaves the AMD/Qwen capacity claim an unvalidated single
08-09 21:36alert_silentThe new delta is only a minor engagement reobservation with no substantive evidence, so it adds nothing that should interrupt Scott before the next briefing.
08-09 21:36alert_routeThe new delta is only a minor engagement reobservation with no substantive evidence, so it adds nothing that should interrupt Scott before the next briefing.
08-09 21:34alert_silentA user-supplied patch and logs report a large AMD/Qwen context-capacity gain, but this is an unmerged, single-system benchmark that overlaps the already tracked llama.cpp MTP auto-fit issue. Scott has
08-09 21:34surface_candidateA user-supplied patch and logs report a large AMD/Qwen context-capacity gain, but this is an unmerged, single-system benchmark that overlaps the already tracked llama.cpp MTP auto-fit issue. Scott has
08-09 21:34alert_routeA user-supplied patch and logs report a large AMD/Qwen context-capacity gain, but this is an unmerged, single-system benchmark that overlaps the already tracked llama.cpp MTP auto-fit issue. Scott has
08-09 21:27groundThe radar already tracks the same llama.cpp MTP memory-overhead development in `radar:llama-cpp-mtp-default-memory-regression`; this case adds a specific AMD/Qwen context-capacity claim awaiting valid
08-09 21:24promote_anchororigin walk conf 0.94
08-09 21:22createThe report supplies comparative ROCm and Vulkan measurements indicating a specific auto-fit defect with large practical context-capacity effects.