How to Test Agent Client Protocol Adapters Across Editors and Coding Agents

Last reviewed: 2026-09-02

Direct answer

Test an Agent Client Protocol (ACP) adapter as a boundary between two independently changing programs, not as a single successful demo. This is the core of Agent Client Protocol adapter compatibility testing: build a matrix whose rows are real editor or client implementations and whose columns cover the agent, transport, negotiated protocol version, advertised capabilities, session lifecycle, streaming updates, permission decisions, cancellation, and failure behavior. Run the same deterministic fixtures through every meaningful cell, save a sanitized transcript, and make a release decision from assertions rather than screenshots.

The reason for this boundary-first approach is that ACP standardizes communication between code editors or IDEs and coding agents . The introduction documents local agents as editor subprocesses using JSON-RPC over stdio, while remote agents may use HTTP or WebSocket; it also notes that complete remote support is still a work in progress. A client that works over a local pipe can therefore still fail when framing, reconnect, or capability handling changes in another surface.

Start with protocol version 1, which the ACP protocol repository identifies as the current stable protocol version. Treat the value exchanged in initialize.protocolVersion as the wire contract. Do not substitute a crate, package, or generated-schema release number for that negotiated value. After initialization, use the exchanged capabilities to decide which optional methods are legal, and mark an unsupported optional feature as a skipped test rather than silently treating it as a pass.

For each adapter release, require one clean happy path and a deliberately exercised error path. The happy path should initialize, create an isolated session, send a short deterministic prompt, consume all expected updates, answer any permission request, and end with a recorded stop reason. The error path should deliberately use an unsupported version or capability, malformed framing, a denied permission, a timeout, cancellation, and an abrupt peer close. A useful result is not simply “the process exited zero”; it is a bounded transcript showing what the adapter accepted, rejected, and cleaned up.

Who this is for

This guide is for platform engineers, IDE-extension authors, and developer-tool teams that connect a coding agent to an editor, command-line client, CI job, or custom frontend. It is especially useful when your adapter is distributed to more than one client, when you wrap an existing agent, or when a model gateway can change the agent behind a stable editor integration.

It is not a prerequisite for someone running an agent interactively in one terminal. If you own the integration boundary, however, you own the compatibility evidence. An editor team can use the same plan to validate an agent it does not control, and an agent team can use it to prove that a new release has not changed the messages its clients rely on.

Key takeaways

  • Separate ACP wire compatibility from SDK, crate, and schema-artifact versions. Negotiate protocolVersion and record capabilities for every run.
  • Test transport behavior explicitly: newline-delimited JSON over stdio, plus TCP or another remote path when your product supports it. Keep protocol output separate from diagnostic logs.
  • Use both a mock fixture and real implementations. The ACP Registry is a curated way to find compatible agents; its page says listed agents support authentication and can be fetched programmatically.
  • Treat session/new, session/load, session/prompt, update notifications, permission responses, and cancellation as a lifecycle, not unrelated endpoint checks.
  • Log identifiers, method names, negotiated versions, capability summaries, outcomes, durations, and error classes. Never log prompts, source contents, environment values, or credential material by default.
  • Fail closed on unknown required fields, invalid framing, missing permission responses, and version mismatches. Preserve enough evidence to reproduce the decision.

Sources checked

The ACP introduction was checked for the purpose of ACP, local stdio transport, remote HTTP or WebSocket scenarios, JSON-RPC framing, and the Markdown-oriented text representation. The ACP Registry was checked for its curated list, authentication requirement, and programmatic distribution model.

The agent-client-protocol repository was checked for the stable protocol version, the distinction between negotiated wire compatibility and artifact versions, and the role of capabilities and versioned schemas. GitHub’s Copilot CLI announcement was checked as an independent implementation reference: it describes ACP over stdio and TCP, isolated sessions, streaming updates, permission requests, cancellation, and lifecycle management.

Cursor’s ACP documentation was checked for a concrete stdio JSON-RPC 2.0 surface, newline-delimited framing, the initialize/authenticate/session/new/session/prompt flow, permission outcomes, session loading, and extension methods that either block for a response or act as notifications. Finally, JetBrains’ ACP overview was checked as an independent IDE-side interoperability reference whose title presents ACP as a way to use any coding agent in any IDE.

These sources describe contracts and implementations; the matrix and operator controls below are a practical test design derived from those contracts. Recheck the linked documentation when an agent, client, schema artifact, or transport changes.

Contract details to verify

1. Initialization and negotiation

Make initialization the first assertion in every fixture. Send a stable client name and version, the smallest truthful capability set, and protocolVersion: 1 when the harness supports the stable protocol. Confirm that the response is valid JSON-RPC, records the negotiated version, and reports capabilities that the adapter actually implements. Add a negative fixture that asks for a version the peer does not support; the expected result is a classified incompatibility and a clean shutdown, not a retry loop that changes the request until it happens to pass.

A minimal, secret-free request fixture can look like this:

{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":1,"clientCapabilities":{"fs":{"readTextFile":false,"writeTextFile":false},"terminal":false},"clientInfo":{"name":"adapter-harness","version":"0.1.0"}}}

Keep the fixture in source control beside the expected schema assertion. The repository explains that two schema-artifact releases can describe the same wire protocol, so pin both the artifact revision used to generate the fixture and the negotiated protocol value observed at runtime.

2. Transport and framing

Give each transport its own test row. For a local adapter, verify that one newline-delimited JSON message produces one response or notification and that partial writes are buffered until a complete line arrives. Cursor documents this stdio and JSON-RPC 2.0 arrangement explicitly. For a remote-capable adapter, repeat the same assertions over the supported HTTP, WebSocket, TCP, or proxy path; the Copilot CLI announcement provides a real stdio mode and a TCP mode to use as independent cases.

Keep stdout (or the remote protocol channel) machine-readable. Route diagnostics to stderr or a separate structured sink. Inject a harmless diagnostic line in a test and assert that the client rejects or isolates it rather than parsing it as a protocol message. Test an oversized line, a truncated JSON object, an idle connection, and a peer that closes mid-update. Each should produce a bounded error classification and release the child process, socket, and session resources.

3. Sessions, prompts, and updates

Use a fresh temporary working directory and a unique synthetic session identifier for every case. Exercise session/new, a deterministic text prompt, and the complete update stream. If the peer advertises resume support, exercise session/load after a controlled restart; otherwise record “not supported” instead of inventing a resume guarantee. Assert that updates are associated with the requested session and that the final response includes the expected stop reason.

The prompt should be tiny and deterministic, such as asking the fixture agent to report a fixed word without changing files. The assertion should check message shape, sequence, and termination, not a model’s prose style. If the protocol carries diffs or other structured coding UX data, validate the type and required fields while redacting repository contents from stored evidence. ACP’s introduction notes that it reuses JSON representations from MCP where possible and adds coding-oriented types such as diffs; that is a reason to test typed payloads rather than flattening everything to text.

4. Authentication and permissions

Authentication is a separate state in the matrix. The registry says its curated agents support authentication, while individual clients expose different method identifiers and setup paths. Test an already authenticated sandbox, an unauthenticated startup, a user-declined authentication request, and an expired session without placing any credential value in fixtures or logs. A pass means the adapter reports the state clearly and offers a bounded recovery path; it does not mean the harness attempts to guess or print a secret.

Permission handling deserves a blocking test. Cursor documents session/request_permission and the outcomes allow-once, allow-always, and reject-once, and warns that an unanswered request can block tool execution. Send a request that requires approval, record the pending request ID, return an allow-once decision in the happy path, and return a rejection in the error path. Assert that the client never auto-approves a request whose ID or session does not match the active test.

5. Extension and capability behavior

Optional methods must be driven by the negotiated capability map. Cursor’s documentation distinguishes blocking extension methods, such as a question or plan-approval request, from fire-and-forget notifications, such as todo or task updates. Build one fixture for each behavior: a blocking request must receive exactly one valid response, while a notification must not be answered as if it were a request. Unknown optional notifications can be counted and surfaced for review; unknown required requests should fail closed with a useful classification.

6. A concrete operator workflow

Run the following workflow in a disposable checkout or isolated container. First, pin the adapter build, fixture-agent build, client build, protocol-artifact revision, and transport. Second, start the adapter with stdout captured as protocol bytes and stderr captured separately. Third, send initialize, verify version and capabilities, then authenticate only through the test environment’s normal mechanism. Fourth, create a session with a known working directory and send one deterministic prompt. Fifth, consume updates until the final response, answer a permission request with allow-once when it appears, and assert a clean stop. Sixth, hash the sanitized transcript and attach it to the run record.

The happy-path harness can describe its expected sequence without embedding a live endpoint:

initialize -> authenticate (if advertised) -> session/new -> session/prompt
             -> session/update* -> session/request_permission (optional)
             -> permission decision -> final response -> cleanup

Then repeat from a clean process for the error path. Send an unsupported protocol version, a malformed line, and a prompt after the session has been cancelled. Deny one permission, delay one response beyond the harness timeout, and close the peer while updates are streaming. For each case, assert an error class, elapsed-time bound, process or socket cleanup, and absence of a second side effect. A failure is actionable only when the operator can tell whether it occurred during negotiation, transport, lifecycle, permission, or cleanup.

Use a compact, sanitized event record such as:

{"run_id":"run-001","client_name":"editor-fixture","agent_name":"agent-fixture","transport":"stdio","protocol_version":1,"request_method":"session/prompt","request_id":"42","session_id":"session-001","capabilities_hash":"sha256:fixture","outcome":"ok","error_class":null,"duration_ms":184}

The allowed fields are intentionally operational: build labels, method names, opaque short IDs, a capability summary or hash, outcome, error class, and timing. Replace prompts, file paths, source text, environment values, and credential material with [REDACTED] or omit them. Do not store raw protocol payloads until a reviewer has confirmed that the redaction boundary is effective.

Failure modes

Artifact-version confusion. A generated schema changes layout while the wire protocol remains compatible, or a client assumes a package version is the protocol version. Detect this by logging both artifact revision and negotiated protocolVersion; gate behavior on the latter and regenerate fixtures deliberately.

Capability overclaiming. The adapter advertises filesystem, terminal, resume, or extension support that it cannot honor. Run a minimal-capability case and then one case per advertised feature. A missing implementation should be reported as unsupported before a session starts, not discovered after a tool has changed a checkout.

Framing contamination. A debug print, banner, or stack trace enters the JSON-RPC channel. Keep diagnostics on a separate stream, test one-message-per-line framing, and fail the run on the first non-JSON protocol line. This is particularly important for stdio adapters, where Cursor documents newline-delimited JSON.

Permission deadlock. A client receives a permission request but never answers, or answers an old request after a restart. Set a request deadline, bind the decision to session and request IDs, and test allow, reject, cancellation, and timeout. Cursor’s documented blocking behavior makes this a user-visible failure rather than a cosmetic log issue.

Session leakage. A session/load or reconnect attaches to the wrong working directory, shares state with a prior test, or leaves a child process running. Use disposable directories, unique IDs, explicit cleanup assertions, and a post-run process/socket check.

Streaming truncation or reordering. The adapter declares success after the first update, drops the final update, or mixes notifications from two sessions. Record sequence numbers where available, correlate every update to a session, wait for the terminal response, and replay a captured fixture before blaming the model.

Remote-surface optimism. A local stdio test passes while a proxy, TCP, HTTP, or WebSocket deployment fails. The ACP introduction calls full remote support a work in progress, so label remote rows accurately and do not claim coverage for a transport you have not exercised. Add reconnect and idle-timeout cases when remote support is part of your product.

Extension-type mismatch. A client replies to a notification or fails to reply to a blocking extension method. Maintain an allowlist of request versus notification methods per capability set, and make duplicate responses a hard failure.

Registry drift. A curated agent is upgraded or removed and the matrix silently changes. Record the registry entry and agent build used, keep a known-good fixture agent, and review changes before replacing a baseline. The registry is a discovery surface, not a substitute for pinned compatibility evidence.

FAQ

Is ACP the same as MCP?

No. ACP is the editor-or-client to coding-agent boundary. The ACP introduction says it reuses JSON representations from MCP where possible, while adding coding-specific types, so shared shapes do not make the protocols interchangeable. Test the ACP envelope and lifecycle separately from any MCP servers the agent invokes.

Which protocol version should a new adapter test?

Begin with stable protocol version 1, then negotiate and record the peer’s response. The protocol repository cautions that schema-artifact releases and wire compatibility are different concerns. Add a version row when a peer supports another protocol version, and keep unsupported-version behavior explicit.

Do I need a real agent or is a mock enough?

Use both. A mock makes malformed frames, timeouts, and deterministic update sequences repeatable. A real implementation catches authentication, permission UI, streaming, and extension behavior that a mock may accidentally simplify. The ACP Registry offers a curated set of compatible agents, and Copilot CLI and Cursor publish independent ACP surfaces that can serve as concrete matrix rows.

How should I test permissions without exposing secrets?

Use a sandbox account or pre-authenticated test environment and exercise the protocol’s decision states, not credential values. Store only the method identifier, request ID, outcome, and timing. If a diagnostic contains sensitive text, replace it with [REDACTED] before persistence.

What makes a test a release blocker?

Block on failed initialization, a wrong negotiated version, invalid framing, an unanswered required request, cross-session data, an unbounded process, or a side effect after cancellation. Treat an unsupported optional capability as a documented skip only when the peer truthfully reports that it is unsupported.

How often should the matrix run?

Run the fast fixture suite on every adapter change and the full real-agent matrix when the client, agent, protocol artifact, transport, authentication setup, or permission behavior changes. Re-run the error path after dependency upgrades; regressions often appear in cleanup and framing rather than in the first successful prompt.

Reader next step

Create a table with one row for each client-agent pair you actually support and columns for version negotiation, transport, capabilities, authentication, sessions, streaming, permissions, cancellation, and cleanup. Add one deterministic happy fixture and one failure fixture to your CI job. Save the sanitized event record, artifact revision, and final decision together so a reviewer can reproduce the result.

When the adapter produces a code change, pair the protocol transcript with the reviewable diff workflow . For operational approvals and follow-up ownership, use the operational decision ledger . Start with one real client and one real agent today, make the error path fail visibly, and expand the matrix only after the first row is reproducible.