Prevent SSRF in Coding-Agent URL Fetchers

Last reviewed: 2026-10-15

Direct answer

Prevent server-side request forgery in coding-agent URL fetchers by treating a URL as a request for network authority, not as ordinary text. Prefer named, preapproved destinations over arbitrary URLs. When arbitrary public retrieval is genuinely required, parse once with a standards-based URL parser; permit only the intended scheme and port; reject credentials and ambiguous host forms; resolve every address family; deny every destination that is not allowed by policy; and make the connection use an address that passed that same decision. Disable automatic redirects or repeat the complete decision for every hop.

Do not stop at a hostname string check. A hostname can resolve to several addresses, change between validation and connection, or redirect to a new host. A syntactically valid URL can still point at loopback, link-local, private-use, shared, documentation, benchmark, or another special-purpose range. A production policy should derive its classifications from maintained registries and deployment-owned rules, not from a short hand-written list copied into one patch.

MITRE’s CWE-918 definition describes the core weakness: a server receives a URL or similar request and retrieves it without sufficiently ensuring that the request goes to the expected destination. It notes that unexpected hosts, ports, and schemes can turn the server into a proxy around network access controls. The OWASP SSRF prevention cheat sheet recommends allowlisting when trusted applications are known, disabling automatic redirect following, validating both IPv4 and IPv6, and addressing DNS rebinding by tying validated resolution to the connection.

Those sources define the weakness and defensive principles. The review contract and test matrix below are operational recommendations. They do not prove a particular patch safe until its parser, resolver, HTTP client, proxy, container network, and deployment routes have all been inspected and tested together.

Who this is for

This guide is for maintainers reviewing agent-written link previewers, documentation importers, webhook testers, package inspectors, issue attachment downloaders, image fetchers, browser helpers, and source-verification tools. It also applies when an agent adds a generic fetch_url tool for another agent. Model output, repository text, issues, pull-request comments, tool results, and remote API fields are all untrusted URL sources unless a separate authority decision says otherwise.

The control here is request-level destination validation. Pair it with the site’s network egress allowlist guide , which constrains what the workload can reach even when application logic fails, and the permission and secret boundary guide , which limits what a compromised fetcher can expose. Defense in depth matters because application validation and network topology can drift independently.

Key takeaways

• Accept a destination identifier whenever the product can map it to a fixed host; do not accept a complete URL merely for convenience.
• Parse once with the runtime’s current standards-based URL implementation, then compare normalized components rather than searching the original string.
• Permit only required schemes, normally https:. Reject embedded usernames, passwords, fragments when unused, unexpected ports, and unsupported schemes.
• Resolve all candidate IPv4 and IPv6 addresses. One acceptable answer must not hide another answer that policy forbids.
• Classify numeric literals after parsing, including IPv4-mapped IPv6. Do not rely on textual prefixes such as 127. or 10..
• Bind the permitted resolution to the actual connection. A separate unchecked lookup creates a DNS-rebinding and time-of-check/time-of-use gap.
• Disable redirects by default. If the workflow needs them, cap hops and rerun parsing, authorization, resolution, and connection binding on every Location target.
• Keep cloud metadata, control planes, container bridges, service discovery, Unix-socket adapters, proxies, and non-HTTP schemes outside the fetcher’s reachable authority.
• Bound response bytes, decompressed bytes, headers, duration, content types, and concurrent requests. SSRF prevention does not prevent resource exhaustion.
• Log policy outcomes and destination classes, not full URLs, query strings, credentials, response bodies, or resolved internal addresses.

Sources checked

• OWASP Server-Side Request Forgery Prevention Cheat Sheet was reached successfully on 2026-10-15. The live page supports allowlisting known destinations, parser-based validation, IPv4-and-IPv6 checks, disabling redirect following, monitoring allowed domains, and binding a validated DNS result to the connection to resist rebinding.
• CWE-918: Server-Side Request Forgery from the MITRE CWE Program was reached successfully on 2026-10-15. The current CWE 4.20 entry supports the threat model: a service retrieves a supplied destination without ensuring it is expected, potentially reaching protected hosts, ports, local files, or alternate protocols.
• IANA IPv4 Special-Purpose Address Space was reached successfully on 2026-10-15. The page identifies the registry itself as last updated on 2025-10-09; the HTTP representation returned a 2026-10-13 Last-Modified header. Its maintained registry supplies address blocks and properties such as source, destination, forwardable, and globally reachable; it includes private-use, shared, loopback, and link-local blocks.
• IANA IPv6 Special-Purpose Address Space was reached successfully on 2026-10-15. The page identifies the registry itself as last updated on 2025-10-09; the HTTP representation returned a 2026-10-13 Last-Modified header. It supplies the corresponding IPv6 classifications, including loopback, IPv4-mapped, unique-local, and link-local entries.
• Node.js URL documentation was reached successfully on 2026-10-15 as the v26.12.0 documentation. It documents the WHATWG URL parser, component properties, parse failure behavior, embedded username/password fields, and conversion of internationalized hostnames to ASCII.
• Node.js DNS documentation was reached successfully on 2026-10-15 as the v26.12.0 documentation. It documents returning all address results with dns.lookup(..., { all: true }), the operating-system facilities behind dns.lookup(), and the different behavior of dns.resolve*().

These are current primary technical references. IANA registry content and runtime behavior can change, so implementation evidence should record the registry snapshot or library version actually used. The article deliberately does not claim that public reachability alone makes a destination trustworthy.

Contract details to verify

A reviewer should be able to reconstruct the authority decision from a small contract. If a patch cannot state the intended destinations and failure behavior, a URL-validation helper will only hide the ambiguity.

Narrow the input before parsing

The strongest design does not accept arbitrary destinations. Map a small action such as fetch_release_notes to a code-owned host and path template, or accept a document identifier that the application expands. OWASP distinguishes this allowlist case from applications that must reach arbitrary external destinations; the former is easier to secure and audit.

If a whole URL is required, impose a length limit before parsing and use one canonical parser for both policy and request construction. For a Node.js implementation, the WHATWG URL API exposes protocol, hostname, port, username, and password; Node’s documentation also states that Unicode characters in a hostname are converted to ASCII. Compare the parsed hostname, not a substring of the original input. Reject a nonempty username or password so credentials cannot obscure the host or leak into logs. Reject schemes beyond the declared set rather than trying to enumerate dangerous ones.

Avoid parser disagreement. Do not validate with a regular expression and send with a library that interprets the string differently. Do not decode or normalize the target again after authorization. If a proxy or SDK reparses the request, include its behavior in the contract and tests.

Classify every resolved address

An allowed hostname is not yet an allowed connection. Resolve all addresses the production connector may choose and reject the request if any result falls outside policy. Node’s DNS documentation shows that dns.lookup() can return all discovered address objects with { all: true }; it also warns that this API uses operating-system name-resolution facilities, which can differ from the network DNS queries made by dns.resolve*(). The validator and connector must deliberately use compatible semantics.

Do not encode the policy as a handful of string prefixes. Use a maintained IP-address library and a reviewed data source. The IANA IPv4 and IPv6 special-purpose registries contain more than familiar private and loopback space and explicitly publish whether registered blocks are globally reachable. A conservative public-fetch rule should deny addresses not approved by policy, including special-purpose space, and should normalize IPv4-mapped IPv6 before classification. Deployment-owned deny ranges must also cover organization networks, container and overlay networks, control planes, service endpoints, and metadata routes that a public registry cannot know.

Being globally reachable is necessary for a general public fetcher, not sufficient. The destination may still be disallowed by product policy, content policy, tenant boundaries, or legal restrictions. Conversely, a tightly scoped internal integration may intentionally use a private address; model that as a separate named connector with its own identity and allowlist, not an exception in the public fetcher.

Bind validation to the connection

The most subtle failure is checking an address and then letting the HTTP client resolve the hostname again. The answers can change between the two operations. OWASP’s current guidance calls out domain allowlisting alone as insufficient against DNS rebinding and says the client must connect only to validated addresses.

Use a connector that accepts the reviewed address while preserving the original hostname for TLS server-name indication and certificate verification. Keep the logical host in the HTTP Host field, and never disable certificate validation merely because the socket connects to a numeric address. If the client cannot safely bind a resolution result, route the request through a hardened destination-aware fetch service or proxy that performs authorization and connection in one control plane.

Apply policy to the final socket peer as a second check when the runtime exposes it. Abort if it differs from the approved address set. Treat resolution failure, an empty answer set, a mixed permitted-and-forbidden set, or an unsupported address family as a denial. Do not retry through the ordinary client path, because that discards the property the validation was meant to establish.

Treat every redirect as a new request

OWASP recommends disabling redirect following because a permitted first URL can redirect to a forbidden destination. Manual redirect handling is appropriate only when the product needs it. Set a small hop limit, reject relative or absolute targets that cannot be resolved unambiguously against the current URL, and rerun the entire scheme, credential, port, hostname, address, and connection decision for each hop.

Do not forward authorization headers, cookies, client certificates, or sensitive custom headers across an origin change. Avoid changing a safe method into a state-changing method. Record only the hop number, normalized destination class, and policy result; a Location value may itself contain secrets. Cycles, missing locations, malformed targets, and limit exhaustion should return stable failures rather than falling back to automatic client behavior.

Build a harmless adversarial test matrix

Most tests should use a fake resolver, a recording connector, and a local disposable HTTP fixture. That makes every answer and socket target deterministic without probing real private services. The fixture can issue redirects and compressed responses; it does not need access to a developer’s network.

Use reserved fixture names and addresses that cannot route to a real service, plus injected resolver results. Never test by requesting a cloud metadata endpoint, scanning a private subnet, or aiming at another tenant. A deny test proves the decision path when the recording connector observes zero connection attempts.

Happy-path operator workflow

1. Inventory every changed path that can initiate a request, including SDK helpers, image and package libraries, redirect handlers, proxy adapters, and browser fallbacks.
2. Trace every part of the destination to its origin. Mark model output, repository content, issues, tool results, remote fields, and redirect headers as untrusted.
3. Replace general URLs with named destinations or identifiers wherever the feature permits it.
4. Define one parser and a component policy for scheme, credentials, hostname, port, path, query, and fragment.
5. Resolve all relevant families and classify every result using a maintained library, current IANA-derived data, and deployment-specific network rules.
6. Configure the connector to use only reviewed addresses while retaining hostname-based TLS verification. Confirm any proxy follows the same policy.
7. Disable redirects or implement bounded per-hop authorization without forwarding sensitive headers across origins.
8. Add header, body, decompression, content-type, timeout, concurrency, and cancellation limits before reading response data.
9. Run the adversarial matrix with fake resolution and a recording connector, then run a narrow integration test inside the real sandbox and egress policy.
10. Review the final diff for ordinary-client fallbacks, automatic redirects, proxy inheritance, duplicate parsers, disabled TLS checks, and logs containing complete URLs.

A useful success record names the fetch-policy revision, purpose, normalized destination class, scheme, effective port, address-family set, redirect count, status class, byte buckets, elapsed bucket, and cleanup result. It should not retain the full URL, query, headers, response body, raw DNS answers, cookies, tokens, or repository-derived labels.

Error-path operator workflow

Reject invalid syntax or policy before DNS or network I/O. Use stable codes such as url_parse_failed, scheme_denied, credentials_denied, port_denied, destination_denied, resolution_mixed, redirect_denied, peer_mismatch, response_limit_exceeded, or fetch_timeout. These codes support tests and operations without echoing hostile input.

If resolution changes, returns a forbidden answer, or cannot be bound to the connection, stop. Do not retry with the hostname and the default HTTP client. If a redirect fails policy, return the redirect denial rather than the first response body. If the response violates a size or type limit, close the stream and discard staged bytes; do not publish a partial artifact.

If operators believe a legitimate target is blocked, change the destination policy through review and rerun the same request in a new attempt. Do not add a wildcard, exempt one resolver result, or switch off the sandbox network boundary during an incident. An unavailable validator, registry snapshot, resolver, or destination-aware proxy is a fail-closed state.

Failure modes

The patch blocks only obvious private IPv4 prefixes. That misses IPv6, mapped addresses, loopback, link-local, shared, special-purpose, deployment-specific, and alternative numeric forms. Canonicalize with a maintained library and decide from reviewed range data.

The validator accepts if any address is public. The connector may choose a different answer. Require all candidate answers to satisfy policy, then restrict the connection to the approved set.

The code validates DNS and calls ordinary fetch() afterward. The HTTP stack may resolve again. Bind the reviewed address to the socket or delegate both decisions to one hardened fetch service.

Automatic redirects remain enabled. A public first hop can redirect into protected space or to a disallowed port. Disable redirects or authorize every hop from scratch.

TLS verification is disabled to support address pinning. That trades one vulnerability for interception. Preserve the logical hostname for certificate and server-name checks while controlling only the peer address.

A URL regex is treated as a parser. Encoding, user information, internationalized names, IPv6 brackets, ports, and relative resolution create disagreement. Let one standards-based parser define the components used by both policy and construction.

The application trusts its firewall as the only control. Egress rules are essential defense in depth, but broad internal routes, proxies, or later rule drift can expose destinations. Keep application authorization and network enforcement independently testable.

Logs become the exfiltration path. Full URLs and errors can contain credentials, queries, fragments, internal names, and response snippets. Retain policy codes and coarse classes instead.

A generic fetch tool inherits ambient credentials. Cookies, proxy credentials, cloud identities, client certificates, and authorization headers make an otherwise simple request more powerful. Give the fetcher a minimal environment and construct headers explicitly.

The patch validates only the request and ignores the response. Large or compressed bodies, slow streams, and active content can exhaust or compromise downstream processing. Bound the stream and keep fetched bytes inert until content validation succeeds.

FAQ

Is allowing only HTTPS enough?

No. HTTPS protects transport to the selected peer; it does not decide whether that peer is authorized. An HTTPS URL can still resolve or redirect to a protected address, and disabling certificate validation to make a pinned connection work removes an important control.

Can a hostname allowlist replace address checks?

No. It narrows intent and should be used when destinations are known, but DNS rebinding and compromised DNS can map an allowed name to a forbidden address. Check every resolution result and bind an approved result to the connection.

Should the fetcher accept direct IP literals?

Usually not when the product expects named public sites. If literals are required, parse them canonically and apply the same address policy without DNS. Never compare the original string with a prefix list.

Are the IANA globally reachable flags a complete security policy?

No. They are authoritative inputs for special-purpose classification. Add organization, cloud, container, overlay, service-discovery, and product destination rules. Global reachability does not mean the destination is approved or safe.

Is a HEAD request safe for validation?

No. It still creates an outbound request and some servers treat methods unexpectedly. Authorization must occur before any method is sent. A preflight request also does not safely reserve the DNS answer for a later GET.

What evidence should a reviewer retain?

Retain policy and test versions, destination and address classes, the connector binding result, redirect count, response-limit results, stable failure codes, and the sandbox egress-policy revision. Omit URLs with queries, raw addresses when sensitive, headers, bodies, tokens, and uncontrolled error text.

Reader next step

Choose one URL-fetching path in the next coding-agent patch and write its authority contract before approving the implementation. Add the mixed-address, redirect-to-forbidden, resolution-change, proxy-inheritance, and oversized-response cases to a fake-resolver test suite. Then run one harmless allow case and one deny case inside the actual sandbox described in the network egress allowlist guide , recording the exact application-policy and network-policy revisions. If the connector cannot prove that it used a reviewed address, keep the patch blocked until that boundary is redesigned.