Independent replication will determine whether four-model orchestration in Claude Code consistently underperforms simpler single-model setups on Terminal-Bench because coordination and refusal failures outweigh specialization gains.

state: expiredheat: lowuncertainty: highconvergesscott: mediummulti-model-orchestration coding-agents agent-harnesses terminal-benchQuesma Anthropic

What is this?

The case concerns a reported Quesma experiment wiring four models into Claude Code and testing the setup on Terminal-Bench, with the claim that coordination and refusal failures outweighed gains from model specialization. The supplied snippets establish Terminal-Bench as an agent benchmark and independently describe coordination as a central multi-agent difficulty; they also show that structured, DAG-based skill orchestration can outperform flat invocation. However, none of the snippets directly documents Quesma’s experiment or an independent replication, so the claimed consistent underperformance and its four specific failure modes remain unverified here.

Why it matters to Scott

The reported backfire converges with Scott’s view that multi-agent gains depend on explicit decomposition, routing, supervision, and cheap handoffs—not merely wiring several models together—and with his model-plus-harness benchmark unit. It directly invites a trace-backed replication relevant to his multi-model systems, but the Quesma result remains unverified and adjacent radar cases already track coordination overhead and model-routing regressions.

ip:framework.micro-agents-architectureip:concept.model-plus-harness-benchmark-unitdev:concept.deterministic-agent-control-planedev:concept.trace-backed-agent-comparisondev:concept.task-aware-model-routingradar:open-ended-agent-coordination-benchmarkradar:multi-model-orchestrator-worker-agentsradar:claude-subagent-roster-overheadradar:concept.coding-agent-benchmarks

queries asked of Scott's wikis

multi-agent coding versus single-agent performance
coordination tax in coding-agent harnesses
specialist agents and model-routing failure modes
refusal propagation in agent orchestration
Terminal-Bench harness design and evaluation
DAG-based versus flat agent orchestration

Measured heat

no measured readings yet — the hourly heat pass fills this in

How the heat travelled

no chain yet — the hourly chain pass fills this in

Evidence (5) — ⭐ canonical anchor

source	object	author	score	comments
🟧 hn	I wired 4 models together in Claude Code. It backfired 4 ways on Terminal-Bench	bkotrys	7	1
🟧 echo.blog ⭐	Reports that connecting four models inside Claude Code backfired in four ways during Terminal-Bench testing.	Quesma	—	—
🟠 reddit	Opus 5 solved security tasks when I asked directly, but refused the same tasks when my orchestrator delegated them (445 Terminal-Bench trials) ClaudeAI	Bartaseth	1	2
🟧 hn	Claude Opus 4.8 Costs 57.1× More, Loses All 5 Benchmarks – Beaten by the Harness	Floatboat_ai	1	0
🟠 reddit	Claude Code Orchestrator on Terminal-Bench: Same model, same tasks - Opus refused only when the work was delegated artificial	Bartaseth	13	8

Interpretation history

2026-08-14T11:29:42Z
Repeated reobservation has produced no independent run, controlled baseline, traces, or root-cause artifact; the single-operator result remains uncorroborated and the episode has faded pending genuinely new replication.
2026-08-12T10:31:20Z
The refreshed comments reinforce that delegation changes prompts, permissions, context, and refusal surfaces, and identify artifacts a valid replication should log. They add no independent benchmark run or controlled comparison, so the core underperformance hypothesis remains uncorroborated.
2026-08-12T08:37:08Z
The newly attached Reddit post is a restatement by the same operator of the same 445-trial experiment, not an independent replication. It sharpens the delegated-context refusal observation but leaves the broader claim of consistent multi-model underperformance uncorroborated.
2026-08-12T08:22:33Z
evidence attached: reddit.post.1vm7a6t — This independent Terminal-Bench report directly bears on whether delegated multi-model orchestration causes refusals and underperformance.
2026-08-11T11:44:30Z
The new link is only an adjacent claim that harness design can outweigh model choice; without methodology, results, or evidence of an independent Terminal-Bench replication, it does not corroborate the four-model backfire hypothesis.
2026-08-11T11:23:06Z
evidence attached: hn.story.49256231 — The report adds evidence that harness design and orchestration can dominate model choice in coding-agent benchmarks.
2026-08-10T22:41:37Z
The Reddit attachment turns the bare claim into a concrete single-operator report covering 445 Terminal-Bench trials, with benchmark conditions and model roles described. It remains the same experiment rather than an independent replication, and without traces or a controlled single-model comparison it does not establish consistent orchestration underperformance.
2026-08-10T22:22:55Z
evidence attached: reddit.post.1vknpd1 — shared external link with case evidence
2026-08-10T15:37:36Z
No replication, methods, traces, or benchmark scores have emerged; the unchanged observation leaves this as a single self-reported result rather than evidence of consistent multi-model underperformance.
2026-08-10T15:31:05Z
grounded: converges/medium — The reported backfire converges with Scott’s view that multi-agent gains depend on explicit decomposition, routing, supervision, and cheap handoffs—not merely w
2026-08-10T15:28:14Z
case created — The report presents a bounded negative benchmark result on a consequential coding-agent harness tradeoff that is suitable for replication.

Decision trace

08-14 21:29expireRepeated reobservation has produced no independent run, controlled baseline, traces, or root-cause artifact; the single-operator result remains uncorroborated and the episode has faded pending genuine
08-14 21:29alert_silentThe only trigger is staleness, with no substantive evidence delta; engagement and repeated observation do not warrant interrupting Scott.
08-14 21:29alert_routeThe only trigger is staleness, with no substantive evidence delta; engagement and repeated observation do not warrant interrupting Scott.
08-14 09:21sensor_dirtyengagement_update
08-13 01:21sensor_dirtyengagement_update
08-12 20:31repriceThe refreshed comments reinforce that delegation changes prompts, permissions, context, and refusal surfaces, and identify artifacts a valid replication should log. They add no independent benchmark r
08-12 20:31alert_silentThe new discussion is methodological commentary and anecdotal agreement, not a replication, trace release, or controlled result; it can wait for substantive evidence.
08-12 20:31alert_routeThe new discussion is methodological commentary and anecdotal agreement, not a replication, trace release, or controlled result; it can wait for substantive evidence.
08-12 20:21sensor_dirtycomment_update
08-12 18:37repriceThe newly attached Reddit post is a restatement by the same operator of the same 445-trial experiment, not an independent replication. It sharpens the delegated-context refusal observation but leaves
08-12 18:37alert_silentNo new experiment, controlled single-model comparison, traces, or independent operator result has appeared; the additional post repeats an already captured finding and can wait for substantive replica
08-12 18:37alert_routeNo new experiment, controlled single-model comparison, traces, or independent operator result has appeared; the additional post repeats an already captured finding and can wait for substantive replica
08-12 18:23alert_silentThis Reddit post is another link to the same Quesma experiment already captured, with no new results, replication, root-cause artifact, or first-party platform response. The reported delegated-context
08-12 18:23alert_routeThis Reddit post is another link to the same Quesma experiment already captured, with no new results, replication, root-cause artifact, or first-party platform response. The reported delegated-context
08-12 18:22attachThis independent Terminal-Bench report directly bears on whether delegated multi-model orchestration causes refusals and underperformance.
08-12 18:22propose_attachThis independent Terminal-Bench report directly bears on whether delegated multi-model orchestration causes refusals and underperformance.
08-11 21:44repriceThe new link is only an adjacent claim that harness design can outweigh model choice; without methodology, results, or evidence of an independent Terminal-Bench replication, it does not corroborate th
08-11 21:44alert_silentThe unsupported link title adds no validated replication or actionable result, so Scott loses nothing by waiting for methods, traces, or controlled benchmark data.
08-11 21:44alert_routeThe unsupported link title adds no validated replication or actionable result, so Scott loses nothing by waiting for methods, traces, or controlled benchmark data.
08-11 21:23alert_silentA lone, low-engagement link title alleges a large cost/performance gap but supplies no visible methodology, benchmark results, or evidence that this independently replicates the four-model Terminal-Be
08-11 21:23surface_candidateA lone, low-engagement link title alleges a large cost/performance gap but supplies no visible methodology, benchmark results, or evidence that this independently replicates the four-model Terminal-Be
08-11 21:23alert_routeA lone, low-engagement link title alleges a large cost/performance gap but supplies no visible methodology, benchmark results, or evidence that this independently replicates the four-model Terminal-Be
08-11 21:23attachThe report adds evidence that harness design and orchestration can dominate model choice in coding-agent benchmarks.
08-11 21:23propose_attachThe report adds evidence that harness design and orchestration can dominate model choice in coding-agent benchmarks.
08-11 08:41repriceThe Reddit attachment turns the bare claim into a concrete single-operator report covering 445 Terminal-Bench trials, with benchmark conditions and model roles described. It remains the same experimen
08-11 08:41alert_silentThe methodological detail makes the result worth tracking, but it neither independently corroborates the failure nor creates a time-sensitive decision for Scott; it can wait for the next briefing or a
08-11 08:41surface_candidateThe methodological detail makes the result worth tracking, but it neither independently corroborates the failure nor creates a time-sensitive decision for Scott; it can wait for the next briefing or a
08-11 08:41alert_routeThe methodological detail makes the result worth tracking, but it neither independently corroborates the failure nor creates a time-sensitive decision for Scott; it can wait for the next briefing or a
08-11 08:23alert_silentA single operator reports a substantial 445-trial Terminal-Bench result showing higher cost and delegation-specific refusals in a four-model Claude Code setup, but there is no independent replication,
08-11 08:23surface_candidateA single operator reports a substantial 445-trial Terminal-Bench result showing higher cost and delegation-specific refusals in a four-model Claude Code setup, but there is no independent replication,
08-11 08:23alert_routeA single operator reports a substantial 445-trial Terminal-Bench result showing higher cost and delegation-specific refusals in a four-model Claude Code setup, but there is no independent replication,
08-11 08:22attachshared external link with case evidence
08-11 08:21propose_attachshared external link with case evidence
08-11 01:37repriceNo replication, methods, traces, or benchmark scores have emerged; the unchanged observation leaves this as a single self-reported result rather than evidence of consistent multi-model underperformanc
08-11 01:37alert_silentThe legacy dirty-state trigger adds no substantive evidence or consequential delta, so the case can wait for an independent replication or technical artifact.
08-11 01:37alert_routeThe legacy dirty-state trigger adds no substantive evidence or consequential delta, so the case can wait for an independent replication or technical artifact.
08-11 01:36alert_silentA single self-reported Terminal-Bench result suggests useful orchestration failure modes, but the visible evidence provides no methods, traces, scores, or replication showing a consequential general r
08-11 01:36surface_candidateA single self-reported Terminal-Bench result suggests useful orchestration failure modes, but the visible evidence provides no methods, traces, scores, or replication showing a consequential general r
08-11 01:36alert_routeA single self-reported Terminal-Bench result suggests useful orchestration failure modes, but the visible evidence provides no methods, traces, scores, or replication showing a consequential general r
08-11 01:31groundThe reported backfire converges with Scott’s view that multi-agent gains depend on explicit decomposition, routing, supervision, and cheap handoffs—not merely wiring several models together—and with h