Last reviewed: 2026-08-03

Direct answer

A reliable CometAPI coding agent token budget starts with the exact model that will receive the request. Resolve the route and API surface first, render the complete request, count that request with a counter or tokenizer appropriate to the selected model, and then reserve capacity for output, tool calls, and counting uncertainty. Dispatch only when the resulting total fits the applicable model contract.

This order matters because a common gateway request shape does not make tokenization universal. The CometAPI quick-start documentation describes an OpenAI-compatible base URL, requires a current model ID, and notes that some coding or reasoning models may require the Responses API. Model identity and request surface therefore belong in the preflight key. A count produced for one route must not be reused automatically after the router selects another model.

Use this conservative admission test:

estimated input + reserved output + reserved tool-loop budget + safety margin <= applicable context allowance

Also enforce any separate input or output cap in the model contract. The input estimate should represent what will actually be sent: system instructions, task text, selected repository context, conversation history, tool definitions, and supported attachments. Count before dispatch, but after all payload-building steps. If anything changes afterward, the earlier decision is stale.

When the request does not fit, do not send it and hope the provider trims the right material. Apply a declared reduction policy, rebuild the request, and count again. That policy can remove generated files, vendor trees, duplicate excerpts, or older low-priority conversation turns before it removes repository instructions or task acceptance criteria. The related repository context-packing guide explains how to choose relevant context; token preflight determines whether the chosen package fits the selected route.

Who this is for

This workflow is for platform engineers, coding-agent maintainers, and engineering leads whose agents send substantial repository context through CometAPI. It is especially useful when routing can change by task, when tool schemas are large, when conversations span multiple turns, or when a fallback model may replace the primary route.

It is not merely a cost estimator. A token budget is an admission-control contract for request feasibility. Cost reporting remains valuable, but it answers a different question after usage occurs. Teams that need that layer can pair this workflow with the CometAPI usage-tracing guide .

Key takeaways

  • Key every count to the exact model ID, request surface, counter method, and contract revision.
  • Count the final structured request, not just the user’s visible prompt or a folder’s raw text.
  • Include system instructions, history, tool schemas, and supported attachments when the counter supports them.
  • Reserve output and tool-loop capacity before deciding that a request fits.
  • Recount after trimming, payload mutation, model fallback, or model migration.
  • Treat a provider count as an estimate when its documentation says so.
  • Log decision metadata and numeric totals, not source code, prompts, headers, tool arguments, or credentials.

Sources checked

The operational contract in this guide is grounded in four refetched public sources:

  • Start with CometAPI: create an API key and make your first call documents the gateway base URL, OpenAI-compatible request formats, model-ID selection, and the need to use an API surface appropriate to the model.
  • Anthropic token counting says its endpoint accepts the structured inputs used to create a message, including system prompts and tools, and returns an input-token estimate. It also advises recounting against the model that will actually be used when tokenizer behavior differs.
  • Google’s guide to understanding and counting tokens documents pre-request input counting, post-response usage fields, system-instruction and tool token accounting, and programmatic model limit lookup.
  • OpenAI’s Tiktoken counting recipe explains that different models can use different encodings. OpenAI marks this recipe as archived, so use it as a tokenizer concept and implementation reference, not as a current model or API contract.

Together, these sources support a provider-aware design. They do not support assuming that one local character ratio, one old tokenizer table, or one provider’s count is portable across every CometAPI model.

Contract details to verify

Create a versioned token-contract registry rather than scattering limits through agent code. Each approved route should record:

  • The exact model ID passed to CometAPI.
  • The API surface used by that model, such as Chat Completions or Responses.
  • The documented input, output, and combined limits that apply.
  • The source and retrieval date for those limits.
  • The provider counter, approved local tokenizer, or conservative estimator used for preflight.
  • Which request components the counter accepts and includes.
  • The output reservation, tool-loop reservation, and safety-margin policy.
  • The approved fallback routes and the requirement to recount after a route change.
  • The action to take when the counter or contract metadata is unavailable.

Do not approve a model migration merely because a sample request returns successfully. Tokenizer behavior and applicable limits are part of the migration contract. Add them to the CometAPI model-change checklist and test representative large requests before changing production routing.

Happy-path operator workflow

  1. The agent assembles its instructions, task, selected repository excerpts, conversation history, and tool definitions.
  2. The router resolves an exact CometAPI model ID and the required API surface.
  3. The preflight service renders the final structured payload without dispatching it.
  4. The service uses the counter assigned to that model contract. Where a provider offers structured token counting, send the same supported components that will appear in the generation request. Where an approved local tokenizer is used, pin its version and model mapping.
  5. The service retrieves the applicable limits and reservations from the registry, then calculates the admission total.
  6. If all input, output, and combined constraints pass, the service records a sanitized allow decision and sends the unchanged payload.
  7. After completion, it records returned usage totals when available and compares them with the preflight estimate. Material drift becomes a contract-review signal, not an automatic reason to expose request content in logs.

For a concrete policy example, suppose a route has an applicable allowance of 128,000 tokens. The final request estimates 82,000 input tokens, while policy reserves 16,000 for output, 12,000 for tool-loop growth, and 8,000 as a safety margin. The admission total is 118,000, so the request passes that combined check. These figures illustrate the calculation only; they are not specifications for any named model.

Error-path operator workflow

If the same calculation exceeds the applicable allowance, the counter fails, required limit metadata is missing, or a fallback changes the model, stop before gateway dispatch. Record a block reason, then choose one of the declared recovery actions:

  1. Remove excluded paths, duplicate excerpts, generated artifacts, and older low-priority history in that order.
  2. Rebuild the full payload and recount it against the same model.
  3. If routing changes, restart preflight with the new model’s counter and limits rather than carrying the old count forward.
  4. If the request still does not fit, split the task into bounded phases or escalate for a human decision.

Do not repeatedly send the same oversized payload. A retry changes neither its count nor its contract.

Sanitized decision logging

A useful log records enough information to reproduce the admission decision without storing proprietary prompts or credential material:

{
  "event": "agent_token_preflight",
  "run_ref": "run-42",
  "route_model": "model-a",
  "request_surface": "responses",
  "counter_kind": "provider-counter",
  "contract_revision": "rev-7",
  "estimated_input_tokens": 82000,
  "reserved_output_tokens": 16000,
  "reserved_tool_tokens": 12000,
  "safety_margin_tokens": 8000,
  "applicable_limit_tokens": 128000,
  "admission_total_tokens": 118000,
  "trimmed_items": 2,
  "decision": "allow",
  "payload_logged": false
}

For a blocked request, change the decision and add a short enumerated reason such as missing_contract, counter_unavailable, or over_budget. Do not add the request body, repository excerpts, environment contents, headers, tool arguments, or raw provider responses simply to make the log more detailed.

Failure modes

Counting before routing. A model-agnostic estimate is accepted, then the router selects a model with a different tokenizer or limit. Resolve the route first and make model identity part of the cache key.

Counting only visible prompt text. System instructions, history, and tools remain outside the estimate. Build the structured request first and use a counter that accepts the relevant components. Anthropic’s documentation explicitly includes system prompts and tools, while Google’s documentation says system instructions and tools are counted.

Mutating an approved payload. A later middleware step adds a tool, policy block, or repository excerpt after preflight. Treat any token-bearing mutation as invalidating the decision.

Reusing a count after fallback. The primary route fails and the same count follows the request to a different model. Restart preflight for the fallback’s exact contract.

Treating estimates as guarantees. Anthropic states that its token count can differ slightly from actual message input. Preserve a safety policy and compare estimates with returned usage rather than setting the threshold at the last documented token.

Leaving tool growth unbounded. The initial request fits, but repeated tool results consume the remaining context. Set a tool-iteration ceiling, reserve loop capacity, and recount the accumulated conversation before another model call.

Treating counter failure as zero. A timeout or unsupported input leads to an allow decision. Use fail-closed behavior for missing contract evidence, or apply a separately approved conservative estimator and a stricter threshold.

Using an archived table as current truth. The OpenAI recipe is explicitly archived. It demonstrates model-specific encodings, but current model mappings and limits need current contract evidence.

Logging content to diagnose budgets. Full prompts and repository excerpts appear in centralized logs. Keep numeric totals, route identity, contract revision, component counts, and decisions; investigate content locally under the repository’s access controls.

Confusing feasibility with price. A request can fit a context contract and still violate a spending policy. Run cost and rate-limit controls as separate gates.

FAQ

Is there one tokenizer that works for every model behind CometAPI?

No such portability is established by the checked sources. CometAPI provides a common integration path and model selection, while the provider documentation describes model-specific counters or encodings. Bind the counting method to the selected model contract.

Can post-response usage replace preflight counting?

No. Post-response usage is valuable for calibration, reporting, and drift detection, but it arrives after the request has been admitted. Google’s documentation distinguishes pre-request count_tokens results from the richer usage data returned with an interaction. Use both at their appropriate stages.

Should every request reserve a fixed percentage of headroom?

The checked sources do not establish a universal percentage. Derive reservations from the route’s documented limits, the maximum useful output, expected tool iterations, and observed estimate-to-actual variance. Version that policy and test it with representative workloads.

May token counts be cached?

Yes, under a strict application policy, but only when the model ID, request surface, complete token-bearing payload, counter version, and contract revision are unchanged. Any changed file excerpt, instruction, tool definition, history turn, attachment, or fallback route requires a new count.

What should happen when no exact counter is available?

Do not silently substitute another model’s result. Block the route until a contract is approved, or use an explicitly reviewed conservative estimator with additional margin and representative tests. The decision should be visible in sanitized logs.

Does token preflight eliminate provider errors?

No. Limits and interfaces can change, and documented counters may return estimates. Keep normal error handling and review evidence when behavior changes. The CometAPI error-response review guide provides a complementary workflow.

How is this different from repository context packing?

Context packing answers which files and excerpts are relevant. Token preflight answers whether the fully rendered request fits the exact model route while preserving room for useful output and tools. A production coding-agent workflow needs both decisions.

Reader next step

Start with one approved coding model and create its registry entry: exact model ID, API surface, current limits, counting method, reservations, and failure policy. Build four fixtures covering a normal request, an oversized request, an unavailable counter, and a fallback route. Confirm that only the normal request dispatches immediately, that trimming triggers a recount, and that fallback always restarts preflight.

Then review the sanitized fields with the people who operate the agent and the people responsible for repository confidentiality. Once the contract and logs are ready, Start with CometAPI and connect the preflight gate to the exact route your coding agent will use.