Agent detects stale context → updates → validates

Stale context detection is the capability of an agent to recognize that the documentation, runbooks, or contextual information it is working with no longer accurately describes the codebase it is working on — and to proactively update and validate that context before continuing. The agent does not passively consume inaccurate context and produce incorrect output. It detects the discrepancy, generates a correction, validates the correction against the code, and either applies it automatically or surfaces it for human review.

The detection mechanism varies by context type. For API documentation, the agent compares documented function signatures against the actual signatures in code. For configuration references, it compares documented parameter names against the actual configuration schema. For runbooks, it attempts to follow the documented procedure in a sandbox and detects where the procedure fails. For ADRs, it identifies when the decision they describe has been reversed or superseded by changes in the codebase. Each detection mechanism requires the agent to have both the documentation and the ground truth it should reflect.

The update step generates a corrected version of the stale documentation based on what the agent found in the code. For straightforward cases — a renamed function, an added configuration parameter — the update is high-confidence and can be applied with minimal human review. For complex cases — an ADR whose architectural rationale no longer applies because the system has been redesigned — the agent surfaces the discrepancy and proposes options for how to handle it, but defers the decision to a human.

The validation step closes the loop. After generating an update, the agent verifies that the updated documentation is internally consistent, does not conflict with other documentation, and accurately describes the current state of the code. For runbooks, validation means running the updated procedure in a sandbox and confirming it succeeds. For API docs, validation means parsing the generated documentation and confirming it matches the code. Validation converts the update from a generated draft to a verified correction.

• Stale context is an agent safety issue - an agent that trusts stale documentation will make changes based on incorrect assumptions; stale context that goes undetected propagates errors through the system; detection is a prerequisite for reliable agent operation in long-lived codebases
• The detection loop closes the knowledge maintenance gap - without automatic staleness detection, documentation drift is only discovered when a human notices an inconsistency or an agent produces incorrect output; with detection, discrepancies are found and corrected continuously
• Validation creates trusted documentation - documentation that has been validated by an agent running the procedure or parsing the code is more trustworthy than documentation written by a human and not subsequently verified; the validation step is what distinguishes auto-generated documentation from auto-generated noise
• Detect-update-validate runs continuously without human initiative - the most valuable property of this loop is that it runs whether or not any human thinks to check; documentation accuracy does not depend on anyone's memory, availability, or discipline
• Stale context metrics become observable - when an agent is continuously detecting and correcting stale context, the detection rate, correction accuracy, and residual staleness are all measurable; this makes documentation health a quantified, monitorable property

1. Start with high-confidence, high-value detection - API reference documentation and configuration documentation are the highest-value targets for initial stale context detection because discrepancies are easy to detect mechanically (function signature mismatch, undocumented parameter) and corrections are high-confidence. Start there before moving to more complex documentation types.

2. Build the detection agent before the update agent - the most common failure mode in this area is building an agent that detects and updates simultaneously without validating that the detection signal is accurate. Build a detection-only agent first, run it against the codebase for one month, and measure the false positive rate before adding the update step.

3. Define confidence thresholds for automated updates - not every detected discrepancy should trigger an automatic update. Define thresholds: a renamed function is a high-confidence update that can be applied automatically. A restructured authentication flow is a low-confidence update that should surface a flag for human review. Confidence thresholds prevent the update agent from generating updates that are worse than the stale original.

4. Implement sandbox validation for runbooks - runbook validation requires a safe execution environment. Set up a sandbox that mirrors the production environment well enough to execute runbook procedures and verify outcomes. This is infrastructure investment, but it is the only way to validate that a runbook procedure actually works as documented.

5. Build the escalation path for low-confidence detections - when the agent detects a discrepancy it cannot resolve automatically, it must route it to a human efficiently. Define the escalation format: a summary of the discrepancy, the agent's best-guess correction, and the information the agent lacks to make the update with confidence. This format should make it fast for a human to review and act.

6. Measure detection recall and precision - recall: what fraction of actual staleness discrepancies are detected? Precision: what fraction of detected discrepancies are genuine staleness? Both metrics are important and must be measured through periodic human audit. An agent that detects 30% of staleness with 95% precision is a different problem than one that detects 90% with 60% precision.

Trusting detection without validation. A detected discrepancy is a hypothesis, not a fact. The agent has identified a potential mismatch; validation is what confirms it is a genuine discrepancy rather than a detection error. Building a pipeline that applies updates on detection without validation will produce a system that confidently applies incorrect corrections.

Running the detect-update-validate loop without audit logging. A loop that runs silently provides no visibility into what it is doing or whether it is working correctly. Every detection, every update, and every validation result must be logged in a way that is human-readable and auditable. Without logging, debugging problems in the loop is nearly impossible.

Applying the same detection logic to all documentation types. API reference staleness detection is fundamentally different from ADR staleness detection. An API function signature change is a clear, mechanical discrepancy. An ADR whose rationale no longer applies requires semantic understanding of the architectural intent and the current system state. Build separate detection agents for different documentation types rather than a single generic detector.

Conflating stale documentation with incorrect documentation. Some documentation is intentionally prescriptive rather than descriptive — it describes how things should be, not how they are. An ADR that says "we will migrate to async messaging by Q3" is not stale if the migration has not yet happened; it is aspirational. The detection agent must distinguish between documentation that is stale (describes the past, not the present) and documentation that is aspirational (describes the future, not the present). Treating aspirational documentation as stale will generate spurious correction proposals.

Optimizing for throughput at the expense of accuracy. The detect-update-validate loop should optimize for producing accurate documentation corrections, not for maximizing the number of corrections per day. A high-throughput loop with mediocre accuracy generates review burden that exceeds the value of the corrections. Slow down and improve accuracy before scaling up throughput.