2026-10-11 18:02 UTC

The paper’s authors claim persistent non-decaying state can make safety failures compound across autonomous LLM-agent loops, implying long-running harnesses need lifecycle-level rather than per-step controls.

state: expiredheat: lowuncertainty: highconvergesscott: mediumagentic-security agent-harnesses long-running-agents

What is this?

“Safety Does Not Compose” is presented as a paper arguing that persistent, non-decaying loop state can cause safety failures to accumulate across long-running autonomous LLM-agent executions, making per-step safeguards insufficient. The supplied results support the broader concern through work on compounding state drift, trajectory-level constraint drift, and harness controls such as step limits, stopping criteria, signed constraints, and audit trails. However, none of the snippets directly identifies this paper or its authors, so its specific mechanism and findings remain unverified here.

Why it matters to Scott

The claimed mechanism independently supports Scott’s architecture-level position that durable agent state requires structural forgetting, independent closure, and deterministic lifecycle controls rather than per-step behavioral guardrails alone. It offers a dated-receipts and framework-extension opportunity, but relevance is capped because the supplied evidence does not directly verify the paper, authors, experiments, or findings.
ip:framework.five-surface-loop-anatomyip:framework.long-running-agentsip:concept.structural-forgettingdev:concept.deterministic-agent-control-planeradar:agent-memory-self-state-attacksradar:dfah-bench-agent-trajectory-driftradar:concept.agent-safetyradar:concept.persistent-agents
queries asked of Scott's wikis
  • persistent agent state and compounding failure
  • lifecycle governance for long-running agents
  • per-step guardrails versus trajectory-level safety
  • agent harness termination and no-progress controls
  • memory decay, reset, and state hygiene
  • constraint drift in autonomous agent loops

Measured heat

no measured readings yet — the hourly heat pass fills this in

How the heat travelled

no chain yet — the hourly chain pass fills this in

Evidence (2) — ⭐ canonical anchor

sourceobjectauthorscorecomments
🟧 hnSafety Does Not Compose: Non-Decaying Loop State for Autonomous LLM Agentssbulaev10
🟧 echo.paper ⭐“Safety Does Not Compose” attributes a long-running autonomous-agent safety failure mode to non-decaying loop state.the paper’s authors——

Interpretation history

Decision trace