Signed Agents: The Coming Schism Between Anthropic, OpenAI, and Everyone Defending the Web

For about thirty years the web has run on an identity system held together with tape. A request shows up, it claims to be Googlebot in its user agent string, and you either believe it or you do a reverse DNS lookup to check. That was the whole protocol. It worked because there were a handful of crawlers that mattered, they mostly behaved, and impersonating one got you caught quickly enough that it was not worth it at scale.

That world is gone. The number of automated clients hitting a typical site has exploded, a large share of them are AI agents acting on behalf of a specific person rather than indexing for everyone, and the user agent string has become a suggestion. An agent can call itself anything. Many call themselves Chrome. So the industry is now trying to do the thing it avoided for three decades: give bots a real, cryptographic identity.

The proposals are arriving fast. Web bot auth is an IETF draft. Cloudflare, Akamai, AWS, and Google have shipped support for some version of it. There is a parallel world of agent manifests like agent.json. And underneath the technical drafts is a disagreement that is not technical at all, about whether an AI browsing on your behalf is a bot that should announce itself or a user that gets to stay anonymous. That disagreement is where the schism is. This post is a map of the proposals and an argument about which way the fault line runs.

What “signed agents” actually means

Strip away the branding and there is one core idea: an automated client attaches a cryptographic signature to its HTTP requests, and the server can verify that signature against a public key the client publishes. If it checks out, you know the request really came from whoever owns that key. No reverse DNS, no user agent guessing, no allowlist of IP ranges that goes stale every quarter.

The serious version of this is web bot auth, specified in the Internet-Draft draft-meunier-web-bot-auth-architecture. It is authored by Thibault Meunier of Cloudflare and Sandor Major of Google, which already tells you something about who is driving it. It does not invent new cryptography. It sits on top of RFC 9421, HTTP Message Signatures, which is a finished and stable spec for signing parts of an HTTP request and carrying the result in two headers.

Here is the shape of a signed request:

GET /article/some-page HTTP/1.1
Host: example.com
Signature-Input: sig1=("@authority" "@method" "@path");created=1782912000;expires=1782912600;keyid="poqkLGiymh_W0uP6PZFw-dvez3QJT5SolqXBCW38r0U";alg="ed25519";nonce="b1Fq...";tag="web-bot-auth"
Signature: sig1=:TUhJ...base64url...:
Signature-Agent: "https://agent.example-ai.com"

Three pieces do the work:

• Signature-Input lists which parts of the request were signed (method, path, authority), when the signature was created and when it expires, the key id, the algorithm, a nonce to frustrate replay, and a tag of web-bot-auth that marks the intent.
• Signature carries the actual signature bytes in base64url.
• Signature-Agent optionally points at the operator’s directory so the server can find the right public key.

The keys are asymmetric on purpose. The draft mandates Ed25519 or RSA-PSS and explicitly forbids shared secrets like HMAC, because a shared secret gives you authentication without non-repudiation. With an asymmetric key the agent cannot later deny that it signed a request, and a site that holds the public key cannot forge requests in the agent’s name. The key id is a SHA-256 thumbprint of the JSON Web Key, so it is stable and self-describing.

Where does the public key live? At a well-known directory on the agent operator’s own domain, following the companion http-message-signatures-directory draft:

GET /.well-known/http-message-signatures-directory HTTP/1.1
Host: agent.example-ai.com

200 OK
Content-Type: application/http-message-signatures-directory+json
{
  "keys": [
    { "kty": "OKP", "crv": "Ed25519", "x": "JrQLj5P...", "kid": "poqkLGiymh_W0uP6PZFw-dvez3QJT5SolqXBCW38r0U" }
  ]
}

The verification flow on your side is mechanical. Parse Signature-Input. Confirm the web-bot-auth tag and that the signature has not expired. Pull the keyid. Find the matching public key, either from a directory you already cached or by fetching the operator’s well-known endpoint. Verify the signature per RFC 9421. If it passes, you are holding cryptographic proof of who sent the request. That is genuinely new, and it is a real improvement over a user agent string you have to take on faith.

agent.json is a different animal, and the conflation matters

Search for “agent.json” and you will land in a second world that looks adjacent but is solving a different problem. The /.well-known/agent.json file comes from Google’s Agent2Agent protocol. It is a manifest where an agent advertises its name, its endpoint, the skills it offers, and the auth flows it supports, so that other agents can discover it and call it. There is also a competing ai-agent.json from other vendors, and a handful of portable-manifest specs all fighting for the same path on your server.

These are agent-to-agent discovery. Web bot auth is agent-to-origin authentication. One answers “what can this agent do and how do I call it,” the other answers “is this request really from who it claims.” They get lumped together because both bolt an identity onto agents and both squat in /.well-known/, but they are not interchangeable, and a defender who confuses them will build the wrong control. A manifest tells you what an agent says about itself. A signature tells you something true. If you are deciding whether to serve a request, you want the signature.

The reason this distinction is about to matter commercially: signed identity is the rail that pay-per-crawl runs on. You cannot charge an AI company for access if you cannot reliably tell which company is at the door. Cloudflare’s marketplace ambitions, and everyone else’s, depend on the signature being trustworthy. Identity is the meter. That is most of why the infrastructure companies are the ones writing the drafts.

Why this becomes a schism instead of a standard

A clean standards story would end here: everyone adopts web bot auth, crawlers sign, sites verify, the user agent string retires. That is not going to happen cleanly, because the parties do not actually agree on the question the standard is trying to answer. The question is: when an AI acts on behalf of a specific human, is it a bot or is it a user?

The answer used to be obvious because the two populations did not overlap. Crawlers indexed the web for everyone and ran from datacenters. Humans browsed for themselves and ran from Chrome. Now there is a fast-growing middle: an agent that opens a real browser, on a real residential connection, and fetches a page because one specific person asked it to. Is that a bot you should gate, or a user you should serve? The honest answer is that it is both, and which word you choose is a policy decision dressed up as a technical one.

Two camps are forming around that decision.

The declared-identity camp. Anthropic publishes its crawlers and what each one does. ClaudeBot fetches for training, Claude-User fetches in response to a live user request, Claude-SearchBot supports search quality, and the documentation tells you the consequence of blocking each through robots.txt. Anthropic states that its crawlers respect robots.txt and will not attempt to bypass anti-circumvention measures like CAPTCHAs. Google authored half of the web bot auth draft and runs an experimental program for authenticating its fetchers with it. Cloudflare wrote the other half and built the verification into its network. The throughline is a belief that automated traffic should be legible: it should say what it is, honor the site’s stated wishes, and increasingly prove its identity with a key. In this view, signing is the natural endpoint of being a good citizen.

The indistinguishable-user camp. The other posture is that an agent driving a browser for a user is the user, and should look like one. A user-driven agentic browser that presents as ordinary Chrome, with no special token in the user agent and no signed identity on its traffic, is making a claim: this is a person browsing, and a person browsing does not have to identify themselves as a machine. There is a real principle underneath it. People use automation to access the open web all the time, from screen readers to ad blockers to scripts, and forcing every automated access to wear a cryptographic badge is a meaningful change to how anonymous the web is allowed to be. But the same posture is also extremely convenient if your business is reach, because an agent that is indistinguishable from a human cannot be metered, rate limited as a bot, or charged for access. The incentive and the principle point the same way, which is what makes the position durable.

The schism is the gap between those two postures. The first camp is building a world where good agents carry keys and the unsigned tail is suspect by default. The second camp has a structural reason never to sign, because signing is the thing that makes its traffic gateable. A standard only ends the argument if both sides want to be identified. Here, one side’s entire value proposition is being mistaken for a person.

I want to be careful about where fact ends and forecast begins. The factual part is the drafts, the authorship, the published crawler policies, and which infrastructure vendors have shipped verification. The forecast is that these harden into two camps and that signing becomes a competitive dividing line rather than a universal handshake. That is a prediction, and predictions about standards adoption are often wrong. But the incentive structure is real and it is not subtle, and incentives like that usually win.

What this means if you run a website

The temptation, reading all this, is to wait for the standard to settle and then flip a switch. That is the wrong move, because the standard settling is exactly the part that may not happen, and because the practical posture is the same whether it settles or not.

Verify signatures when they are present, and treat a valid one as a strong positive signal, not as a gate. If a request carries a web-bot-auth signature that checks out against a published key, you now know who it is with high confidence. That is worth a lot. It lets you apply a real policy to that specific operator: allow this one, rate limit that one, charge a third, block a fourth. It does not mean you should require a signature from everyone, because almost no legitimate traffic signs today, and a hard requirement would block far more good than bad.

Do not let signing become a monoculture you depend on. A signature answers “who,” never “should I serve this.” The hard population was never the agents willing to identify themselves. It is the agents that will not, and a verification-only strategy is blind to exactly that group. Plenty of automated traffic has a structural reason to stay unsigned, and that traffic does not go away because a standard exists. It is the traffic you most need to catch.

Keep scoring unsigned traffic on behavior and fingerprint. This is the part that does not change. For everything that arrives without a signature, which will be the overwhelming majority for years, you are back to the signals that have always done the work: TLS and JA4 fingerprints that survive a faked user agent, behavioral telemetry that separates a driven browser from a human one, honeypots that catch clients that read pages no human would. We have written about JA4 fingerprinting against AI scrapers and about the browser-as-a-service population that industrializes real-Chrome, real-IP agents specifically to defeat these defenses. Signed agents do not retire any of it. They add a high-confidence lane on top.

The right mental model is layering, the same one that applies to why CAPTCHAs stopped working. A valid web bot auth signature is the cleanest signal you will ever get, but it is one signal, available from a minority of clients, and it is most available from exactly the clients that were never your problem. It belongs in the stack. It does not replace the stack. The private access token work points at the same future from the user side: cryptographic attestation of “is this a real, allowed client” gradually replacing inference, but unevenly and partially, with inference carrying the load everywhere attestation has not arrived.

Where this goes

The optimistic reading is that web bot auth becomes the reverse-DNS-check of the next decade: a quiet, boring, widely-supported way to verify the crawlers that want to be verified, with a marketplace for access built on top. That is a good outcome and it is plausible for the declared-identity camp. Cloudflare, Google, and Anthropic are aligned enough on it that the infrastructure will exist regardless of who else shows up.

The realistic reading is that it covers the cooperative half of automated traffic and leaves the other half exactly where it is. The agents with a reason to be seen will sign. The agents with a reason to blend in will keep presenting as Chrome, and the work of telling them apart from a human stays where it has always been, in the messy world of fingerprints and behavior and traps. The schism is not really between two protocols. It is between traffic that wants to be identified and traffic that does not, and no header makes the second kind disappear.

So the question for a defender is not “which standard wins.” It is “what do I do about the traffic that will never sign,” and the answer to that has not changed. Verify the signatures you get, and keep a real detection stack for everything else, because everything else is still where the bots are.

If you want that stack without building it from scratch, FCaptcha is the open-source path: behavioral telemetry, proof of work, fingerprinting, and honeypots in a self-hostable library with the algorithm out in the open. The same engine powers WebDecoy’s hosted detection if you would rather it be managed, and you can start a free trial and have it scoring your traffic in an afternoon. When signed agents do show up at your door, you will be ready to verify them. When the unsigned ones show up, you will be ready for those too.

Further Reading

• Next.js Bot Detection: Block AI Crawlers at the Edge
• JA4 Fingerprinting Against AI Scrapers
• Browser-as-a-Service Detection
• Private Access Tokens: Apple’s CAPTCHA Killer
• Why CAPTCHAs Are Dead (And What Replaces Them in 2026)
• Detecting Vision-Based AI Agents
• Protect Content From AI Training Scrapers
• The RAG Bot Problem: Real-Time AI Content Fetching