Build an Evidence Map for Coding Agent Incident Triage

Last reviewed: 2026-06-30

Direct answer

A coding agent incident evidence map is a short operating record that connects the symptom, the agent surface, repository instructions, external API contract checks, and the next operator action. Build it before asking the agent to repair code or rerun a workflow, so the next step is grounded in source-backed facts instead of guesses.

The map should answer five questions. What failed? Where did it fail? Which instruction, memory, or repository context may have shaped the agent behavior? Which public documentation page supports the contract being checked? What is the smallest next action that can be verified without hiding uncertainty?

Use this workflow:

1. Setup assumptions: the operator has read-only access to the repository, the failing command output, the relevant instruction file, and the public documentation needed for the agent surface or gateway being checked.
2. Happy-path request plan: ask the coding agent to inspect the failing command, identify the smallest likely cause, list files it would touch, and propose one verification command before editing.
3. Error-path check: ask the agent to stop if the failure depends on missing credentials, private account state, unsupported endpoint behavior, or an unavailable source.
4. Minimum assertions: record the exact command or request family checked, the source URL used for the contract area, the observed status class, and whether the proposed fix changes code, configuration, instructions, or only documentation.
5. Pass/fail logging fields: run_id, incident_label, agent_surface, source_url, observed_status, change_scope, verification_command, result, and next_action.
6. Do not assert: exact model availability, prices, quotas, rate limits, uptime, latency, billing impact, or private account state unless those facts are verified in the linked source and the operator account.

For adjacent setup practices, see AI Coding Agent Setup, Security, and Model Routing and Set Safe Permission and Secret Boundaries for Coding Agents .

Who this is for

This guide is for operators who use coding agents to investigate failed builds, broken agent sessions, gateway request errors, or confusing handoffs. It is especially useful when a repair loop touches both repository instructions and an external model gateway. The goal is not to make the record long. The goal is to make the next repair attempt bounded, reviewable, and easy to stop when evidence is missing.

It also helps teams that rotate ownership between terminal agents, IDE agents, pull request reviewers, and incident responders. Different tools expose different workflows, but the handoff problem is the same: a later reader needs to know what was observed, which source was checked, what was changed, and what was deliberately left unclaimed.

Key takeaways

• Start with the symptom and the smallest reproducible command or request family.
• Separate agent-surface evidence from API-gateway evidence.
• Treat repository memory and instruction files as operational inputs that can drift.
• Keep source links clean, dated, and tied to one contract area at a time.
• Use placeholders for credentials and account-specific values.
• Prefer a small stopped investigation over a broad repair that cannot be verified.

A useful evidence map is written for the next operator, not just for the person running the current session. If the incident involves a coding assistant that can read files and run commands, the map should name the surface being used and the local boundary around it. If the incident involves project memory or repository instructions, the map should point to the relevant instruction source and describe how it might affect the task. If the incident involves a gateway, the map should point to the current documentation page for the request family and avoid making claims about pricing, limits, or account status unless those claims are separately verified.

Sanitized log-record template:

run_id: "example-run-YYYYMMDD-001"
incident_label: "agent-triage-placeholder"
agent_surface: "terminal-or-ide-agent"
source_url: "https://example.com/docs/page"
credential_reference: "<API_KEY_PLACEHOLDER>"
observed_status: "pass-or-fail-placeholder"
change_scope: "code-config-docs-or-none"
verification_command: "placeholder command, no secrets"
result: "short outcome summary"
next_action: "retry, revise, escalate, or stop"

Keep the log narrow. A single incident map should not become a general architecture document, a model benchmark, or a pricing note. If the agent proposes a change outside the observed failure, move that idea into a separate task and keep the incident map tied to the failing signal.

Sources checked

• Official source evidence 1 - accessed 2026-06-30; purpose: verify source-backed claims for this guide.
• Claude Code memory documentation - accessed 2026-06-30; purpose: verify project memory and instruction-file context for agent workflows.
• CometAPI documentation - accessed 2026-06-30; purpose: verify current CometAPI documentation navigation.
• CometAPI chat completions reference - accessed 2026-06-30; purpose: verify chat completion contract areas.
• CometAPI help center - accessed 2026-06-30; purpose: verify support and escalation documentation areas.

Contract details to verify

This table is deliberately conservative. It gives the reader safe wording that can be used before a deeper account-specific check, but it does not turn a public docs page into proof of private account behavior. That distinction matters during incidents because a repair can appear to work locally while still depending on a permission, credential, feature flag, or provider state that the operator has not verified.

Failure modes

• Evidence gap: the agent cannot inspect the failing log, source page, pull request, or local command output. The safe action is to stop and record the missing evidence instead of guessing.
• Scope drift: the agent edits files that are not connected to the observed failure. Keep the repair tied to the failing signal and leave unrelated cleanup for a separate task.
• Environment mismatch: the local check uses different versions, credentials, feature flags, or runtime settings than the hosted path. Record the mismatch before treating the result as proof.
• Unreviewed fallback: the agent changes models, endpoints, permissions, or retry behavior to make a run pass without preserving a clear review boundary. Treat access and provider failures as operational blockers, not topic failures.
• Weak handoff: the final note says the issue is fixed but omits the command, result, changed files, and remaining uncertainty. That makes the next operator repeat the investigation.
• Overclaimed gateway behavior: the record says an endpoint, model, price, quota, or support path works for the account when the operator only checked a public documentation page. Use the public page to frame what to verify, then record the account-specific result separately.

A strong evidence map also says what not to do. Do not paste full private prompts, full responses, credentials, customer data, production secrets, account billing values, or unredacted logs into the record. Do not use a passing retry as proof that the root cause is fixed unless the retry is tied to a named change and a repeatable verification command. Do not let an agent replace a narrow incident note with a broad rewrite of repository instructions unless the incident actually shows instruction drift.

Reader next step

Before you run the next repair attempt, create a one-page evidence map with three sections: observed failure, checked sources, and proposed next action. Link the failing command or request family to one public documentation source and one repository-context source. Then ask the coding agent for a bounded plan that names the files it expects to touch and the verification command it will run after the change.

If the incident involves CI, pair this map with Triage CI Failures With a Coding Agent Without Losing the Evidence . If the incident may require a pull request, use How to Hand Off Coding Agent Pull Requests for Review to keep the final note reviewable.

A practical next action can be as small as this: copy the sanitized template above, replace the placeholders with the failing command, add the source URL you checked, and write one sentence beginning with Next action:. If that sentence requires a secret, a private account setting, or an unsupported assumption, stop there and record the missing input. That pause is part of the workflow, not a failure of the workflow.

Use Write Change Scope Notes Before an Agent Pull Request as the next comparison point. Keep Agent Memory Review Before Long-Running Tasks nearby for setup and permission checks.

FAQ

What belongs in the evidence map?

Include the symptom, the failing command or request family, the relevant repository instruction source, the external docs source, the proposed change scope, and the verification result. The map should be short enough to read during an incident but specific enough that another operator can reproduce the reasoning.

Should the agent fix the incident immediately?

Not until the evidence map shows a bounded change. If the problem depends on private account state, missing credentials, or undocumented behavior, the safer action is to stop and record the blocker. A stopped investigation with clear evidence is more useful than a repair that appears successful but cannot be explained.

Can I include real prompts or responses in the log?

Use sanitized placeholders unless the material is already approved for the team record. Avoid credentials, full private prompts, full responses, commercial values, account-specific limits, and customer data. Use <API_KEY_PLACEHOLDER> whenever a credential-shaped value is needed in an example.

How do I handle gateway request failures?

Tie the failure to the exact request family and current docs page. Verify endpoint path, authentication, request fields, and response fields in the linked source before changing agent routing or retry behavior. Keep public documentation evidence separate from account-specific checks.

What makes this different from a normal incident note?

A normal incident note often records what happened after the repair. An evidence map is written before the next repair attempt. It gives the agent and the operator a bounded starting point, a source-backed contract to check, a minimum set of assertions, and a stop condition when evidence is missing.