Prompt caching can make repeated coding-agent requests faster and less expensive, but only when the reusable context is placed behind a deliberate boundary and the selected endpoint actually supports the intended cache behavior. Treat caching as a measured model-and-endpoint contract, not as a universal switch.

Last reviewed: 2026-08-05

Direct answer

For effective CometAPI prompt caching, place stable instructions and shared repository context at the beginning of the request, put the current task and changing tool results at the end, and verify a cache write followed by a cache read in the response usage fields. Recheck that contract whenever the model, endpoint, prompt layout, or repository-instruction revision changes.

The endpoint matters. The CometAPI Anthropic Messages documentation documents cache_control on content blocks for Claude requests sent through the native Messages endpoint. It also reports cache writes and reads through cache_creation_input_tokens and cache_read_input_tokens. The same page marks prompt caching as unavailable through its OpenAI-compatible Chat Completions route. Do not copy a native Messages cache setting into another endpoint and assume it will work.

Build the request in two regions:

  1. Stable prefix: reviewed agent instructions, tool definitions, coding conventions, and repository documentation that should remain identical across a useful batch of tasks.
  2. Variable suffix: the immediate task, current file selection, changed lines, fresh test output, timestamps, and tool results.

The stable region must still be current. A cache can faithfully reuse an outdated prefix if the application keeps submitting outdated material. Generate the stable bundle from a reviewed repository revision, record that revision in operational metadata, and rebuild the bundle when its source files change. The repository context packing guide provides a useful companion workflow for deciding what belongs in that bundle.

Use this operator workflow on a non-mutating evaluation task before enabling caching in an agent that can run tools:

  1. Select one long, repeatable code-review or explanation fixture. Record the model family, exact model ID, endpoint family, prompt-builder revision, and stable-prefix revision.
  2. Run an uncached baseline. Confirm that the answer is complete and record input tokens, output tokens, latency, and the provider’s cache counters.
  3. Enable the cache mechanism supported by that exact endpoint. For the documented CometAPI Claude path, mark the intended stable content block with ephemeral cache_control.
  4. Send the first request. A cache write may occur, so do not call the first request a hit.
  5. Send a second request while keeping the stable prefix byte-for-byte equivalent and changing only the task suffix.
  6. On the happy path, require a correct response and positive cache-read evidence. Compare the second request with the baseline and first request. A lower latency number alone is not proof of a cache hit.
  7. On the error path, classify the result before retrying. If the endpoint rejects a cache field, remove only that field for a controlled uncached test and mark the model-endpoint combination unsupported until its contract is reviewed. If the request succeeds but the read counter remains zero, inspect prefix changes, eligibility thresholds, endpoint choice, and timing. Do not blindly replay a request after an agent has started executing tools.

Keep logs useful without storing prompts, source code, tool payloads, environment values, or request headers. A normalized event can look like this:

{
  "event": "prompt_cache_observation",
  "gateway_request_id": "[REDACTED]",
  "provider_family": "claude",
  "model_id": "claude-sonnet-5",
  "endpoint_family": "anthropic_messages",
  "prompt_builder_revision": "builder-v4",
  "stable_prefix_revision": "repo-policy-v3",
  "stable_prefix_tokens": 4820,
  "input_tokens": 5110,
  "output_tokens": 620,
  "cache_write_tokens": 4820,
  "cache_read_tokens": 0,
  "cache_outcome": "write",
  "latency_ms": 1860,
  "status": "success",
  "error_class": null
}

Normalize provider-specific counters into shared fields, but retain the provider family and endpoint family so operators can trace how each number was derived.

Who this is for

This guide is for developers and platform engineers running context-heavy coding agents through CometAPI, especially teams that repeat large instruction sets, tool definitions, or repository background across reviews and implementation tasks. It is also useful when one agent client can select different provider families and must avoid treating their cache controls as interchangeable.

It is not a guide to caching completed answers. Prompt or context caching reuses eligible input processing. Your application must still validate every new model response, tool call, and code change on its own merits.

Key takeaways

  • Cache the longest genuinely stable prefix, not the entire evolving agent transcript.
  • Treat model ID, endpoint family, prompt order, and cache mechanism as one versioned contract.
  • Require provider usage counters as cache evidence; latency is only a supporting signal.
  • Keep fresh tasks, file changes, test output, and tool results after the stable boundary.
  • Rebuild stable context when its repository sources change, even if the cache remains technically valid.
  • Test cache behavior with non-mutating fixtures before introducing retries into a tool-running loop.

Sources checked

  • The CometAPI Anthropic Messages documentation documents native Claude prompt caching through cache_control, a 1,024-token minimum for the cached block, and separate cache-creation and cache-read usage fields. It also distinguishes the native Messages feature set from the OpenAI-compatible endpoint.
  • The OpenAI prompt caching guide explains that cache hits depend on exact prompt-prefix matches. It recommends placing static instructions and examples first and variable material last, and it documents cache accounting fields and explicit cache-boundary behavior for current model families.
  • The Google Gemini context caching guide documents implicit caching for Gemini 2.5 and newer models, model-dependent minimum token counts, and usage.total_cached_tokens as hit evidence. It also states that the Interactions API supports implicit caching but not manually managed explicit cache objects.

Together, these sources support a provider-aware workflow. They do not establish that every upstream provider control is forwarded unchanged by every CometAPI route. That is why endpoint verification belongs in the operating contract.

Contract details to verify

Record a cache policy beside the model-routing configuration. Keep it short enough to review and specific enough to test:

cache_policy:
  provider_family: claude
  endpoint_family: anthropic_messages
  stable_prefix_revision: repo-policy-v3
  cache_mode: ephemeral
  minimum_prefix_tokens: 1024
  expected_evidence:
    - cache_write_tokens
    - cache_read_tokens
  variable_sections:
    - current_task
    - changed_files
    - test_output
    - tool_results

Verify these details before enabling the policy:

  1. Endpoint capability: CometAPI documents Claude prompt caching on the native Messages endpoint. If routing to a different endpoint or provider family, test the actual response schema instead of inferring support from a similar parameter name.
  2. Prefix boundary: Confirm that changing task data does not appear before the intended boundary. OpenAI’s documentation says hits require exact prefix matches, so a timestamp, request-specific path, or reordered tool definition can prevent reuse.
  3. Eligibility threshold: The documented CometAPI Claude cached block requires at least 1,024 tokens. Gemini publishes model-dependent minimums, including 2,048 tokens for listed Gemini 2.5 models and 4,096 for listed newer models. Read the current model documentation rather than padding a short prompt with low-value text.
  4. Accounting map: Map Claude cache creation and read fields, OpenAI cache-write and cached-token fields, or Gemini cached-token usage into normalized metrics. Missing counters should produce an unknown result, not a guessed hit.
  5. Freshness rule: Tie the stable prefix to reviewed source material. Before each run, confirm that the prompt builder used the intended repository revision. If the source changed, rebuild the prefix and expect a new cache write.
  6. Correctness gate: Compare answer quality and tool-call validity before optimizing hit rate. A high hit rate is not useful when an agent is receiving incomplete or obsolete context.

Track the resulting request and usage evidence with the CometAPI usage tracing guide . When changing the selected model, repeat the contract check described in the model change checklist .

Failure modes

Dynamic content appears too early. A build timestamp, task identifier, current branch summary, or changing tool history is inserted before the stable material ends. The requests may look similar to a person while failing exact-prefix reuse. Move dynamic fields to the suffix and compare the serialized request structure, without logging its sensitive contents.

The cached region is below the model’s threshold. The request succeeds, but the provider reports no cache read or write. For the documented CometAPI Claude flow, a block shorter than 1,024 tokens is not cached. Gemini thresholds vary by model. Treat a short stable prefix as ineligible rather than inflating it with irrelevant context.

The client uses the wrong endpoint. A cache control accepted by a provider-native API may be unavailable on an OpenAI-compatible route. This can produce a rejected request, an ignored field, or usage data without the expected counters. Key the policy by endpoint family as well as model family.

Operators confuse a write with a hit. The first eligible request can populate a cache and still incur write accounting. Measure a later request with the same stable prefix before claiming savings.

Latency is treated as definitive evidence. Network and service conditions vary. A fast response with zero cache-read tokens is not a demonstrated hit, while a real hit can still encounter unrelated latency. Usage counters are the primary evidence.

Normalized metrics erase provider meaning. Combining every counter into a single cached_tokens field without retaining the raw semantic mapping can make writes look like reads. Preserve provider and endpoint dimensions, and document how normalized fields are calculated.

The prompt builder serves old repository context. The cache is not necessarily malfunctioning; the application may be repeatedly sending an old but identical prefix. Compare the stable-prefix revision with the reviewed instruction sources, rebuild it, and run correctness checks before resuming the workload.

A diagnostic retry repeats side effects. A coding-agent request may already have emitted or executed tool calls. Run cache experiments with non-mutating fixtures. In production, consult the recorded tool state before retrying any request.

FAQ

Does a cache hit guarantee the agent received current repository context?

No. A hit means the submitted prefix matched eligible cached input. It does not prove the application assembled that prefix from the latest approved repository state. Freshness must be enforced by the prompt builder and its revision checks.

Can one cache configuration cover Claude, OpenAI, and Gemini models?

Use one normalized policy interface if it helps your application, but implement provider- and endpoint-specific adapters underneath it. The supplied documentation exposes different controls, eligibility rules, endpoint limitations, and usage fields.

What should go in the stable prefix?

Good candidates include reviewed agent instructions, stable tool definitions, coding standards, and shared architectural context. Current tasks, changed-file summaries, fresh tests, tool results, timestamps, and user-specific details belong after the boundary.

Does a zero cache-read count mean the model request failed?

No. The response may be valid but ineligible, may have written a new cache entry, or may have missed because the prefix changed. Separate response success from cache outcome in both code and logs.

Should I make a prompt longer just to cross a caching threshold?

No. Extra low-signal context consumes input capacity and can reduce clarity. Cache only when the workload already has substantial reusable context. Otherwise, keep the prompt concise.

When should the cache contract be rerun?

Rerun it after a model change, endpoint change, SDK or request-shape change, prompt-builder change, stable-context revision, or unexplained shift in cache counters. Also rerun it when provider documentation changes the relevant behavior.

Reader next step

Choose one non-mutating coding-agent task with a large shared context. Split its request into a reviewed stable prefix and a current-task suffix, capture an uncached baseline, then run a write-and-read pair while recording sanitized usage fields. Promote caching only after the second response is correct and the expected read counter is positive.

To exercise the documented native Claude route and compare its measured behavior with your baseline, Start with CometAPI . Keep the first rollout limited to one model-endpoint contract, then repeat the same evidence check before expanding it to another model family.