Emerging discovery surface

llms.txt Explained

llms.txt is an emerging convention for presenting a concise, machine-readable map of useful documentation. Adoption and interpretation vary, so its presence should be reported without treating it as a ranking or citation factor.

What a useful file contains

Keep the file short, public, and stable. Explain the site or product, link to authoritative pages, and prefer durable canonical URLs over temporary campaign links.

Use a clear H1 title and short summary.
Group links under meaningful headings.
Link to canonical public documentation.
Return text/plain or readable Markdown.

How to evaluate it

Verify that the file is reachable at /llms.txt, contains working HTTPS links, and does not expose private routes or tokens. Judge it as a discovery surface, not as evidence of model usage.

Check HTTP status and content type.
Remove sensitive or authenticated URLs.
Keep linked content consistent with canonical metadata.
Refresh the file when documentation changes.

Keep the claim proportional

The convention is a proposal, not a universal web standard. A valid file can make curated resources easier to locate, but it cannot demonstrate that a model fetched, used, or cited them.

Describe presence and validity as observable facts.
Do not call the file an AI ranking factor.
Measure actual server requests separately if you need usage evidence.
Keep normal navigation, sitemaps, and metadata healthy too.

A minimal llms.txt outline

The proposal uses Markdown: a required H1, a concise summary, and grouped links to authoritative resources. The Optional section can hold useful secondary material.

# Example Company

> A concise, factual description of the product and audience.

## Documentation

- [Product guide](https://example.com/docs): How to use the product.
- [API reference](https://example.com/api): Stable public endpoints.

## Optional

- [Changelog](https://example.com/changelog): Recent product changes.

Verify the change

Run the AIWebSignals scanner against the exact public page, review the observation confidence, and compare the saved result after your update. A technical improvement should be visible in the evidence—not assumed from a deployment.

Scan and verify

Authoritative references

Use current primary documentation when a crawler token, platform policy, or web standard changes.

Last reviewed August 18, 2026.