Technical guide · original methodology
How to benchmark regulated websites without inventing certainty
A study design that preserves sample boundaries, confidentiality, unknowns, and reproducible observations.
Define the population first
A convenience sample of sites found during implementation is not a representative industry sample. Specify the inclusion criteria, sampling frame, exclusions, observation period, and unit of analysis before publishing a percentage. A hostname, company, location, and page are not interchangeable units.
Separate pilots from benchmarks
The initial research focus is cannabis-sector technical interfaces. Confidential operator-managed pilots must remain separate from any external cohort and must not be presented as independent validation. If a future approved example is informed by operator-managed work, disclose that methodological relationship without identifying the underlying business. At launch, no industry prevalence result is asserted.
Preserve unknown states
A challenge, timeout, required login, or missing authorization can leave a question unanswered. Preserve that uncertainty rather than assigning failure. A score based on only observable fields can still mislead when observation coverage differs sharply between sites. Report coverage alongside findings, and do not force a composite ranking where comparability has not been established.
Make methods reproducible without exposing pilots
Authorized working records should preserve the tested URL, timestamp, observer, methodology version, result state, and sanitized evidence. Public releases use only approved records or clearly labeled synthetic fixtures. Confidential URLs, screenshots, stable identifiers, and distinctive timestamps stay private. A private pilot cannot substantiate a publicly reproducible industry claim merely by omitting its name.
Use a publication gate
The planned study requires a frozen external cohort, collected observations, contradiction review, re-identification review, and a documented correction channel. A study page may publish its protocol before results exist, but it must not fill the empty result table with illustrative percentages. Unknown is useful when the alternative is a fabricated industry conclusion.
Sources and scope
IETF: Robots Exclusion Protocol, RFC 9309 · AIWebSignals: current public capability contract
Provider documents support the referenced technical facts. The interpretation, examples and proposed checks are AIWebSignals methodology, not provider endorsement or measured industry results.
Review the assessment method and its limits · Request a correction privately · Read the JSON version