Quality Collapses Before the Window Fills
Chroma's Context Rot study reports that all 18 frontier models degrade as input length grows. Even a model with a 200K-token window has already lost 30-50% of its accuracy by the 50K mark. Response quality breaks down well before the window is physically full. Treating the remaining token count as headroom and endlessly accumulating conversation and intermediate output is a clear anti-pattern.
Goal Drift After 25-30 Tool Calls
In long-horizon agents, goal drift appears once you pass 25-30 tool calls. The agent strays from its original objective and re-executes steps it has already completed. Accumulated failure logs and retry traces fill the prompt, so past noise outweighs the latest instruction. A loop that only appends and never compacts raises the drift probability in proportion to the call count.
A Full Guide: From Planning to Operations
In planning, fix the target as numbers. Set a baseline of a 95%-or-higher task pass rate, p95 latency under 20 seconds, a per-task budget of 20 tool calls, and a 50K per-session input-token cap. This budget is a safety line that triggers forced termination and summarization before performance collapses.
Classify failures into three types. First, goal drift: when the current action no longer matches the original objective sentence, recover by compacting context and re-injecting the goal rather than retrying. Second, re-execution of completed steps: keep per-step completion flags in state to block duplicates. Third, budget overrun: at 20 calls, halt automatically and route to a human-confirmation branch.
Design the recovery branches explicitly. Retry transient errors at most twice; on goal mismatch, apply a safe compaction that keeps only core state and drops history; on undecidable cases, send to a human-review queue; on budget or safety violations, stop immediately. Each branch verifies its entry and exit conditions in code.
Operational QC starts with standard logs. Record session ID, step name, tool name, token count, elapsed time, and result code under one schema for every call, and mask PII such as email, phone, and address before the log is stored. Aggregate goal-drift rate, re-execution count, and budget-overrun rate as dashboard metrics.
Run the improvement loop weekly. Sample the call ranges where drift concentrates, tag root causes, and tune the compaction point and budget cap. The goal is not zero violations but a steady rise in recovery rate and a fall in re-execution count.
Execution Summary
Context rots before it fills. Accuracy breaks 30-50% around 50K and goals drift after 25-30 calls, so instead of unbounded accumulation, make a 20-call and 50K-input budget with compact, re-inject, and halt branches your default.