Last reviewed: 2026-08-28

Direct answer

Use a create, overlap, canary, drain, and revoke sequence for CometAPI API key rotation. Create a replacement key without changing the model route or any unrelated configuration. Keep the old key available while one isolated coding-agent worker proves the replacement can complete a representative request. Roll the new configuration out in small batches, stop assigning new work to old workers, and let their active jobs finish. Revoke the old key only after every consumer has moved and your sanitized telemetry shows no remaining old-generation traffic for a defined drain interval.

The CometAPI quickstart documents dashboard key creation and recommends keeping the value in environment configuration rather than source code. CometAPI’s per-key usage guide describes separate keys for clients or workflows and independent rotation or revocation. Neither source establishes a universal overlap window, revocation delay, or reversible disable control, so verify those account-level behaviors before touching production.

This staged approach follows the gradual-rollout principle in Google Cloud’s rotation guidance : introduce a new version without immediately invalidating the old one, detect breakage, retain a rollback path, and clean up the old version only after rollout. It reduces interruption risk, but it is not a guarantee of zero downtime under every provider, runtime, or compromise scenario.

Who this is for

This guide is for platform engineers, application developers, and SREs who operate coding agents through CometAPI and have active workers, scheduled jobs, or queued tasks that may outlive a deployment. It applies whether workers run in containers, CI jobs, virtual machines, or a managed scheduler.

It is especially useful when a single configuration change can affect several agent pools or when long-running workers construct an API client once and keep it for the life of the process. Teams still defining access scope should first review permission and secret boundaries for coding agents .

A suspected compromise changes the priority. In that case, limiting exposure may require immediate revocation and a controlled pause rather than a long overlap. The emergency path appears below.

Key takeaways

  • Treat key rotation as a versioned deployment with an owner, canary, stop conditions, rollback revision, and completion evidence.
  • Prove that the old and replacement keys can coexist before relying on an overlap period.
  • Resolve credentials at worker startup or execution time; do not copy them into queued task payloads.
  • Change only the key during the rotation. A simultaneous model, endpoint, SDK, or prompt change makes failures harder to isolate.
  • Segment keys by workflow or worker pool when practical. Per-key visibility makes attribution and revocation more precise.
  • Never use the old key as an automatic fallback after an authentication failure. That can hide a broken rollout and extend the old key’s lifetime.
  • Keep the old configuration recoverable until the canary and staged rollout pass, but never place the value in a ticket, log, repository, or rotation record.

Happy path: create, canary, drain, revoke

  1. Inventory every consumer. List interactive workers, queue consumers, scheduled jobs, CI runners, autoscaling templates, local operations tools, and dormant recovery jobs. For each consumer, record its owner, worker pool, configuration delivery method, refresh behavior, longest expected task duration, and rollback command. Search deployment definitions for references to the secret object or environment mapping, not for the secret value.

  2. Define the evidence gate. Capture a baseline for request success, authentication-class errors, queue depth, in-flight jobs, latency, deployment revision, and traffic by worker pool. Choose a drain interval that covers the longest expected job plus the known telemetry delay. If per-key reporting is not timely enough, add a non-sensitive configuration-generation label to application logs before rotating.

  3. Create and name the replacement. Use a short operational label that identifies the workload and rotation sequence without containing any secret material. Keep the old key active. Confirm in a non-production or isolated production canary that creating the replacement did not invalidate the existing key.

  4. Store a distinct version. Put the replacement in the team’s approved secret manager and bind the canary deployment to that specific version. The OWASP Secrets Management guidance recommends centralized lifecycle management, least privilege, auditing, rotation, and reduced human contact with secret material. Do not overwrite an unversioned shared value before the canary is ready.

  5. Deploy one isolated worker. Remove the canary from normal side-effecting work. First send a minimal model request, then run a sandboxed task that represents the real agent path. Verify response shape, approved model route, latency range, request correlation, and expected failure handling. Do not use a production repository mutation as the first probe.

  6. Record sanitized evidence. A useful event contains operational identity, not credential material:

{
  "event": "cometapi_rotation_probe",
  "rotation_id": "rot-42",
  "worker_pool": "canary",
  "deployment_revision": "rev-7",
  "config_generation": "new",
  "request_id": "req-42",
  "model_route": "approved",
  "http_status": 200,
  "latency_ms": 842,
  "attempt": 1,
  "result": "success"
}

Do not log request headers, environment dumps, secret-manager payloads, raw key prefixes or suffixes, or full request bodies that may contain repository secrets. Omit sensitive fields entirely; if a rigid log pipeline cannot omit one, write [REDACTED]. Use the practices in reviewing coding-agent telemetry logs to keep the evidence useful without expanding exposure.

  1. Roll out in bounded batches. Send new jobs only to updated workers while old workers finish work already assigned to them. Advance from canary to a small pool and then to the remaining pools. At every stage, compare success and authentication errors with the baseline. Stop automatically if the new generation crosses the team’s predefined error threshold.

  2. Cover non-obvious consumers. Force or observe one scheduled-job launch, one autoscaled worker start, and one recovery-path start while the old key is still available. A healthy continuously running pool does not prove that a cold-start template or rarely used namespace was updated.

  3. Drain the old generation. Wait until no old-config workers remain, all jobs that started on them are complete or safely checkpointed, and old-generation traffic remains at zero for the chosen drain interval. Check queues for embedded configuration snapshots. The queue should carry task identity and state, while the worker resolves current protected configuration when it starts.

  4. Retire the old key deliberately. If the current account provides a reversible disable state, disable first and observe the service for a reasonable intervention window. Otherwise, revoke only after the drain gate passes. Run a controlled request from the protected verification harness to confirm the retired key is rejected, logging only its generation label and the response class. Then repeat the positive canary with the replacement.

  5. Close the rotation record. Preserve the rotation ID, affected pools, configuration revisions, canary request IDs, rollout timestamps, drain evidence, revocation time, operator identity, and follow-up actions. Do not preserve either key value or a reconstructable fragment.

Error path: stop, preserve, roll back

If the canary receives an authentication-class failure, stop before expanding the rollout. Keep the old key active, remove the canary from scheduling, and compare the secret version reference, workload identity permissions, environment or mount mapping, process reload behavior, and unchanged gateway configuration. Correct one variable at a time and rerun the isolated probe.

If failures appear after a partial rollout, stop new batches and route new work to the known-good old revision while that path remains valid. Remove failing workers from the queue, preserve their job and request IDs, and roll back the deployment reference. Before retrying a task that can run tools or write files, confirm its checkpoint and side-effect state. The guide to keeping retried CometAPI calls from running tools twice covers that separate risk.

If the old key was revoked before all consumers moved, pause affected queues rather than repeatedly retrying authentication. Deploy the tested replacement to the missed consumers, resume a small batch, and watch for duplicate or abandoned work. Do not assume a revoked key can be restored; use the controls actually exposed by the account.

If compromise is suspected, stop the planned overlap. Revoke the exposed key, pause workers that cannot receive the replacement safely, preserve sanitized incident evidence, and follow the incident process. Availability matters, but an intentionally extended compromised credential is not a safe zero-downtime strategy.

Sources checked

  • CometAPI Quickstart — confirms dashboard key creation, environment-based storage guidance, and OpenAI-compatible client configuration.
  • CometAPI per-key usage tracking — supports separate keys by client or workflow, per-key attribution, and independent rotation or revocation.
  • Google Cloud Secret Manager rotation guidance — supports gradual rollout, explicit version binding, breakage detection, rollback, reentrant rotation, and delayed cleanup.
  • Kubernetes Secrets documentation — provides the deployment reference for checking how protected configuration is exposed to containerized workers. Runtime consumption still has to be verified for the application.
  • OWASP Secrets Management Cheat Sheet — supports centralized management, least privilege, automation, auditability, rotation, revocation, and incident planning.

These sources support the workflow pattern, but they do not prove account-specific overlap duration, revocation propagation, log freshness, or application reload behavior. Those remain preflight checks.

Contract details to verify

Before production rotation, write down the answers to these questions:

  • Can the account hold the old and replacement keys as simultaneously valid credentials, and did a controlled probe prove it?
  • Does creating a key change any existing key automatically?
  • Is retirement a reversible disable operation, an irreversible revoke operation, or both?
  • How quickly does retirement take effect for new requests, and what happens to requests already accepted?
  • Does per-key usage or request telemetry update quickly enough to support the drain decision? If not, which application-side generation label supplies the evidence?
  • Does each worker read protected configuration at process start, before every job, or through a reloadable file? Updating the secret store alone does not prove that an already constructed client has adopted the new value.
  • Which deployment revision pins the old version, which pins the new version, and how is rollback performed without exposing either value?
  • Do scheduled jobs, autoscaling templates, disaster-recovery environments, and manual runbooks reference the same protected configuration object?
  • What is the longest legitimate agent task, and how much additional observation time covers queue and telemetry delay?
  • Which representative request is safe to use as a canary, and which response fields prove it reached the intended model route?
  • Who may create, deploy, retire, and audit keys? Separate these capabilities where the platform and team size allow.

If any answer is unknown, do not compensate with a longer blind overlap. Test the behavior in an isolated worker and make the result part of the runbook.

Failure modes

  • Immediate replacement of a shared value. Every worker may switch together, eliminating the canary and making a bad value a fleet-wide outage.
  • Blind use of a floating latest version. Google Cloud’s guidance warns that an invalid latest value can spread immediately. Pin a tested version to each deployment revision so rollback is deterministic.
  • Premature revocation. A queue can look idle while an old worker still owns a long task, or a scheduled job may not have started during the observation window.
  • Stale long-lived processes. Updating a secret manager or orchestrator object does not necessarily rebuild an SDK client already held in memory. Prove adoption at the process that sends the request.
  • Missed cold-start paths. Autoscaling, recovery, or infrequent job templates may retain an old reference even when the main pool is healthy.
  • Credentials copied into queue payloads. Queued work then preserves an obsolete value and spreads sensitive material into storage, dead-letter queues, and debugging tools.
  • Automatic fallback to the old key. Fallback can make dashboards look healthy while the new rollout is broken, and it extends reliance on the key scheduled for retirement.
  • Uncontrolled retries after authentication failure. Repeated agent tasks can duplicate tool calls or writes. Pause, determine side-effect state, and retry only from a safe checkpoint.
  • Rotation mixed with another migration. Changing the key, model, gateway route, SDK, and prompt together prevents a clean diagnosis. Rotate one contract surface at a time.
  • Sensitive diagnostics. Environment dumps, copied headers, shell tracing, and raw support attachments can turn a routine rotation into a disclosure incident.
  • Permanent overlap. Keeping the old key indefinitely defeats rotation. Set a retirement deadline, owner, and alert before creating the replacement.
  • One key for unrelated workloads. A shared key obscures which consumer remains on the old generation and makes revocation affect a wider set of agents. Use sensible per-workflow segmentation rather than one key per individual process.

FAQ

Can CometAPI API key rotation be guaranteed to have zero downtime?

No. The staged workflow reduces interruption risk, but success depends on concurrent-key behavior, worker refresh semantics, queue state, and revocation propagation. Verify each of those before production retirement. A security incident may also require accepting a controlled interruption to end exposure quickly.

Is storing the key in an environment variable enough?

The CometAPI quickstart recommends environment configuration instead of source code, which addresses code leakage. It does not by itself solve rotation. You still need protected delivery, least-privilege access, a known process refresh mechanism, canary deployment, and sanitized audit evidence. A long-running process may need a controlled restart or application-supported reload.

Should every coding agent receive its own key?

Not necessarily. A practical boundary is a client, environment, workflow, or worker pool with a distinct owner and risk profile. CometAPI’s per-key guidance supports workflow-level separation and attribution. Creating a unique key for every short-lived process can add operational noise without improving the revocation decision.

When is it safe to revoke the old key?

Revoke after all known consumers use the new configuration, the canary and staged batches pass, old workers finish or checkpoint their tasks, cold-start paths have been exercised, and old-generation traffic remains at zero for the defined drain interval. If telemetry is delayed or ambiguous, wait and improve the evidence rather than guessing.

What should happen to requests already in flight?

Do not assume their behavior. Verify the provider and client contract, then design the drain so revocation occurs after old-generation requests complete. Track request IDs and job state without logging credential material. If a request outcome is unknown, reconcile its side effects before retrying.

What changes if the old key may be compromised?

Shorten or eliminate overlap. Revoke the exposed key, pause workers that cannot safely receive the replacement, deploy through the tested path, and use incident-response logging. The objective changes from a routine uninterrupted rollout to rapid containment with controlled recovery.

Reader next step

Create a one-page rotation worksheet before opening the dashboard. List every consumer, its configuration version, refresh method, maximum task duration, canary request, stop threshold, rollback revision, and retirement owner. Choose one non-critical worker pool, confirm concurrent-key and retirement behavior in an isolated probe, and establish sanitized generation-level telemetry.

Then run one complete create-to-retire rehearsal without combining it with a model or SDK change. Keep the old configuration available only until the evidence gate passes, and close the record without retaining secret material. For the observability side of that rehearsal, use the existing CometAPI telemetry field review .

When the runbook, canary, and rollback path are ready, Start with CometAPI .