Agent readiness bot

Heading measures how AI assistants read and cite websites. Part of that is agent readiness: a set of deterministic HTTP probes that report whether a site's public pages can be read by the machines producing AI answers.

A Heading customer points the probes at a site. We fetch a bounded set of its public pages, record what each request returned, and report the results to that customer. Every finding is a fact you can reproduce with curl: a status code, a header, a document.

A run fetches a small, capped number of pages, once. It is not a continuous crawl. Each request carries a timeout, a response size cap and a redirect limit, and the run stops at its page cap whatever it finds.

For a site a customer measures on an ongoing plan, a run repeats weekly. A one-off check runs once, and manual re-runs are rate limited. We do not fetch daily.

Some runs target a site the customer does not own, for example a site an agency is pitching to. Those runs carry a lower page cap and a longer minimum interval between runs, because we have no relationship with that origin.

We read robots.txt and we do not obey it. One of the probes exists to report what that file says to AI crawlers, and obeying a blanket disallow would silence the report for exactly the sites that most need it. So we treat the file as data: we record what it says, pass that on to the customer, and keep fetching the pages the run needs. The caps and intervals above are what bound us instead.

If you would rather we did not fetch your site, email support@useheading.com with the domain and we will sort it out with the customer who asked for the measurement.

We store the probe results: the URL fetched, the status code, and the header and document facts a probe reads, for example whether a sitemap exists or whether a page serves content without JavaScript.

We do not store personal data from your pages. We do not train models on them. A run makes no model call at all: the probes are plain HTTP requests and comparisons.