Long-Horizon Agents: State, Recovery, Planning, and Verifiable Hours-Long Execution

A long-horizon agent is not one whose timeout moved from ten minutes to ten hours, nor one that pours its entire history into a larger context. It is a recoverable state machine: it maintains goals and constraints in a changing environment, decomposes plans into verifiable local actions, resumes from the right boundary after failure, and lets an external evaluator determine whether the final result is task completion, a reward proxy, or merely a trajectory that looked busy.

1. What Counts as Long Horizon, and What Does Not

A long-running episode has at least four properties: state remains causally relevant across decisions; intermediate errors change later feasible paths; the agent trades local feedback against global constraints; and after interruption it must reconstruct from external artifacts rather than guess from chat memory. Under this definition, the difficulty is not merely “more steps,” but dependencies, recovery, and accountability.

WildClawBench is valuable because it restores the native CLI harness to the evaluation configuration: in its containerized tasks, swapping harnesses for the same model can substantially change performance. It supports attributing long-horizon performance to a model-harness configuration, not assigning a runtime-free fixed ranking to a model.

2. State Is Not a Chat Summary: It Must Support Reconstruction and Attribution

AMA-Bench observes that real agent memory arises from continuous agent-environment interactions rather than only human dialogue; similarity retrieval without causal and objective information limits performance. Agent Memory separates memory-system cost into construction, retrieval, and generation. MAGE proposes a hierarchical execution-state tree that records, compresses, maintains, and revises branches to preserve valid paths and isolate errors.

Every cross-context handoff should reconstruct at least the following from external state rather than a prior-turn summary.

Context compaction is not a checkpoint. The former may help a fresh invocation understand history; the latter must also locate environment, artifacts, and authority. If an external service cannot return to its prior state, retain it as observational evidence rather than pretending it can be rerun deterministically.

3. Planning Must Face Global Constraints and Failure, Not Only Decompose Steps

DeepPlanning places active information gathering, local restrictions, and global time/budget optimization in a long-horizon task. It reminds us that locally correct choices do not make a globally feasible plan, particularly when tools fail, information is absent, or a route must change.

The design lesson from VLAs-as-Tools also applies to software agents: a high-level agent keeps temporal plans and recovery, bounded local tools perform small tasks, and replanning is triggered by explicit progress or exception events rather than constant polling. BCER Agent binds final outputs to intermediate artifacts and measurements and emphasizes bounded local recovery. These works are not evidence of a universal winning architecture; they expose two testable variables: whether progress feedback is verifiable and whether recovery rolls back only to the affected local boundary.

4. Recovery Is a Distinct Ability, Not Infinite Retrying

Good recovery has a boundary. It must not turn every failure into “try again,” nor rerun a full workflow to improve pass rate while hiding failure sources. An auditable recovery event records its trigger, affected scope, recovery point, invalidated evidence, retry budget, human handoff, and terminal outcome.

A minimal fault-injection set can include incomplete tool results, externally modified file or service state, corrupted checkpoints, denied authority, a summary that omits a key constraint, and a local success that already violates the global budget. For each fault, report recovery_success, state_reconstruction, duplicate_side_effects, resume_latency, and cost_delta. If recovery raises a process score without improving independent terminal states or safety, it is another gameable reward proxy.

5. Long-Horizon Security Is Not a Repeated Single-Turn Guard

AgentLAB places intent hijacking, tool chaining, task injection, objective drift, and memory poisoning in multi-turn environments. Its narrow conclusion is that single-turn defenses do not automatically cover multi-turn attacks. For long-running agents, the state store, tool outputs, handoff documents, and subagent messages are all untrusted-input surfaces; a prior approval does not automatically authorize the next turn.

Security state therefore belongs in the checkpoint: time-bounded authority, denied actions, domain/tool allowlists, trust labels on external inputs, risk escalation, and human confirmations. Final reports should at minimum separate task_outcome, integrity_violations, unsafe_action_rate, and false_block_rate; one success rate cannot cover all four.

6. A Falsifiable Protocol for Hours-Long Execution

The following is a design proposal, not a reported long-horizon result. It makes “works continuously for hours” a claim that can fail:

1. Freeze run identity. Retain task manifest, initial-state digest, model/harness/tool/policy versions, budget, environment image, and evaluator hash.
2. Externalize state. Each checkpoint writes goals, constraints, artifacts, dependencies, actions, observations, failures, and resources; redact sensitive content before retention.
3. Test recovery actively. Preregister fault-injection points and success definitions; distinguish observational replay from genuine reruns.
4. Separate acceptance. A candidate cannot write hidden tasks, verifier, authority policy, or final release decision; judge final state and process integrity separately.
5. Compare at equal budget. Single-agent, sequential, multi-agent, memory, and recovery variants receive the same total tokens, worker time, wall clock, retries, and external-tool allowance.
6. Report the complete curve. Report attempts, cancellations, recovery, regression, variance, peak concurrency, total cost, and cost per verified success, not only the best run.

7. How It Returns to the Core Five

This protocol complements harness engineering: that note defines runtime engineering planes; this one turns them into long-horizon execution, fault, and acceptance experiments. See synthetic agent training for data, agent research environments for environment boundaries, Agent Memory for context, retrieval, execution state, checkpoints, and safety admission, Agent Planning for global constraints, feedback-conditioned replanning, tool failure, and correct stopping, and the hot agent directions map for adjacent memory, planning, and safety choices.