Start with the evidence hierarchy
A web server can observe the request time, method, hostname, path, status code, selected headers, and network-level metadata available at the edge. Those facts are materially different from an analyst's interpretation of why the request occurred. For AI traffic, the distinction matters because a bot token can communicate a declared purpose while still not exposing the originating prompt, model reasoning, user objective, or downstream use of the retrieved content.
AIWebSignals therefore treats identity and intent as bounded claims. A recognized crawler token can support classification such as search, training-oriented crawling, or user-requested retrieval when the provider itself documents that purpose. It cannot prove that a specific page was used for training, quoted in an answer, converted into an embedding, or monetized. The same rule applies to sequence analysis: a series of requests can support a probable journey, but not a private mental state.
Keep four traffic classes separate
The first class is automated collection: crawler or agent requests that directly hit the site. The second is user-requested retrieval, where an automated fetch occurs because a person asked a system to inspect a URL or find current information. The third is human referral traffic: a person clicks from an AI answer, search experience, browser, or agent surface into the site. The fourth is economic activity: a subscription purchase, licensed request, or other transaction with explicit settlement evidence.
Mixing those classes creates bad business conclusions. Ten thousand crawler requests do not equal ten thousand prospective customers. A spike in ChatGPT referral sessions is not proof that GPTBot caused the referrals. A payment challenge is not revenue until payment actually settles. Good machine-web analytics preserves these distinctions so product and policy decisions can be made from the right denominator.
Build measurements that survive scrutiny
For automated requests, retain the minimum normalized evidence needed to understand patterns: query-free path, timestamp, status, source classification, and privacy-minimized linkage. For human referrals, use analytics data that distinguishes referrer or campaign source from ordinary search and direct traffic. For policy experiments, record the exact policy change and compare request outcomes before and after the change. For payments, keep transaction identifiers and settlement state rather than inventing an estimated dollar value from crawl volume.
This approach produces fewer dramatic claims, but far more useful ones. It lets a publisher say that a named crawler requested a path 312 times, encountered a 14% error rate, and returned repeatedly after a policy change. It also lets the publisher say when there is no evidence that those requests created human traffic or revenue. That honesty is not a limitation; it is the foundation for credible AI audience intelligence.
What to measure next
A mature machine-web scorecard should combine request evidence, discovery controls, human referral analytics, conversion events, policy outcomes, and payment settlement. Those data sets do not need to be merged into a giant surveillance warehouse. They need stable identifiers, clear time windows, and documented definitions so that changes can be compared without moving the goalposts.
The practical objective is to move from observation to decisions: which agents deserve unrestricted access, where requests are failing, which content attracts repeat retrieval, whether AI referral visitors convert differently, and whether a subset of machine access has enough economic value to license or price. The evidence hierarchy keeps every one of those decisions grounded.