Web Bot Auth at the Edge: Public Support Across Five AIWebSignals Connector Providers
Google-Agent changes the observability question. It is a user-triggered agent identity used by agents hosted on Google infrastructure to navigate and act on the web upon user request. Google says these fetchers generally ignore robots.txt and is experimentally signing a subset of Google-Agent requests with Web Bot Auth. We therefore studied the authentication layer that can distinguish a claimed agent identity from cryptographically verified traffic.
Aggregate result
- 4/5 AIWebSignals managed edge connector providers have explicit first-party public documentation stating Web Bot Auth support.
- 1/5 remains unknown from the bounded first-party documentation review.
The four documented providers are Cloudflare, Vercel, AWS CloudFront/AWS WAF, and Akamai. Fastly is classified as unknown, not unsupported. This study measures public documentation support; it does not certify a live customer configuration or imply that every request arriving through a documented provider is cryptographically verified.
Why Google-Agent makes this different from crawler policy
Google added Google-Agent in March 2026 for agents hosted on Google infrastructure that navigate the web and perform actions upon user request. Google classifies it among user-triggered fetchers and says this class generally ignores robots.txt rules. Google publishes dedicated user-triggered-agent IP ranges and is also experimenting with Web Bot Auth using the identity https://agent.bot.goog.
Google explicitly warns that not every Google-Agent request is signed. During the experimental rollout, verification therefore has multiple evidence levels: a matching User-Agent is a claim; network/reverse-DNS evidence can strengthen that claim; a successfully verified Web Bot Auth signature can provide cryptographic identity evidence. These layers should not be collapsed into one boolean.
Connector-provider observations
Observed at . The cohort is fixed by the managed edge/CDN providers already represented in AIWebSignals' connector registry rather than selected after seeing results.
| Provider / connector | Public documentation status | Observed first-party evidence |
|---|---|---|
Cloudflarecloudflare | Documented public support | Cloudflare documents Web Bot Auth as a cryptographic verification method for verified bots and agents and specifies Signature-Agent, Signature-Input, and Signature request handling. First-party source |
Vercelvercel | Documented public support | Vercel states that its bot verification system supports Web Bot Auth and that Bot Protection can use HTTP Message Signatures to verify automated traffic. First-party source |
AWS CloudFront / AWS WAFcloudfront | Documented public support | AWS documents Web Bot Auth support in AWS WAF for AI agents and automated tools, including CloudFront-protected applications relevant to the AIWebSignals CloudFront connector path. First-party source |
Akamaiakamai | Documented public support | Akamai states that Web Bot Auth is supported by its bot and abuse solutions, with edge signature verification and validated-bot classification. First-party source |
Fastlyfastly | Unknown from bounded public documentation review | Fastly publicly documents that client.bot.name is inferred from User-Agent and is not itself verified, with network ownership suggested as additional evidence. The bounded first-party documentation review did not locate an explicit Web Bot Auth support statement. This is unknown, not unsupported. First-party source |
What each documented provider establishes
Cloudflare documents Web Bot Auth as a verification method for verified bots and agents using HTTP Message Signatures and a public key directory. Vercel states its bot verification system and Bot Protection support Web Bot Auth. AWS documents Web Bot Auth support in AWS WAF, including CloudFront-protected applications, with verification-state labels for managed Bot Control. Akamai states that Web Bot Auth is supported by its bot and abuse solutions and describes edge signature verification followed by validated-bot classification.
Fastly publicly documents that client.bot.name is inferred from User-Agent and that Fastly makes no attempt to verify that claim through that variable; it suggests network ownership as additional evidence. We did not locate an explicit first-party Web Bot Auth support statement during this bounded review. Absence from this review is not evidence that Fastly lacks an implementation.
Product implication: claimed identity and authenticated identity are separate fields
A User-Agent containing Google-Agent is useful observational evidence but is spoofable. AIWebSignals should recognize it as a Google user-requested agent while keeping User-Agent-only identity confidence at claimed. Provider-native verification evidence can raise identity confidence to verified. Raw signatures, IP addresses, cookies, authorization headers, and raw User-Agent strings do not need to be persisted to preserve that distinction.
This matches AIWebSignals' existing evidence model: first-party request records are observed events, while identity strength is separately labeled. The accompanying product regression adds Google-Agent recognition without promoting a User-Agent match to verified identity.
Methodology and limitations
The five managed edge/CDN providers represented by AIWebSignals hosted connector paths: Cloudflare, Vercel, AWS CloudFront, Akamai, and Fastly.
documented_public_support requires a first-party provider source that explicitly states Web Bot Auth support. unknown_public_documentation means no explicit first-party Web Bot Auth support statement was located in the bounded documentation review; it does not mean unsupported.
This is a documentation-support cohort, not a live functional certification. Public documentation does not establish that Web Bot Auth is enabled for every customer property, that every agent signs every request, or that AIWebSignals has observed a signed request through that provider.
Google's Web Bot Auth rollout is experimental and Google says it does not sign every participating agent request. Provider support can also be product-, plan-, rule-version-, or configuration-dependent. This study did not send forged Google-Agent traffic, did not attempt to impersonate a signed agent, did not access authenticated customer infrastructure, and did not test private provider consoles.
Primary references
- Google: Google-Agent user-triggered fetcher
- Google: Authenticate requests with Web Bot Auth (experimental)
- Cloudflare: Web Bot Auth
- Vercel: bot verification supports Web Bot Auth
- AWS: AWS WAF Web Bot Auth support
- Akamai: Web Bot Authentication
- Fastly: client.bot.name identity limitations
Last reviewed September 8, 2026.
Related AIWebSignals evidence
Related technical evidence
Verified AI agent evidence across edge providers · AI Bot Index · Connect first-party AI traffic