Use your server-log evidence to inspect AI crawler requests, the pages accessed and the responses returned. Understand access before drawing conclusions about visibility.
Retrieval activity / Example log evidence
A retrieval request records access to a page. It does not prove that the page was cited or that a person visited your site.
Illustrative data and interface. Not live customer results.
AI crawler analytics
Crawler requests reveal activity on your website. Keep that evidence separate from brand mentions, citations and visits by potential customers.
The crawler view separates retrieval, search and training activity. Inspect the crawler, operator, verification evidence and request counts within the selected range.
These functions are reported separately. A training request is not a search referral or evidence that an answer cited your page.
Explore requested pages and response statuses to identify blocked or failing access. Use the log evidence to choose a specific page or response to investigate.
Connect your server logs to see which pages crawlers request and how your site responds.
A crawler requests an activewear product page and receives a 403 response. That is a concrete access issue to investigate using the server logs and your site configuration.
A successful request tells you the page was accessed. It does not establish that the information appeared in an answer, generated a citation or brought a buyer to your website.
Evidence into action
Keep your research connected to the questions, markets and business objectives that matter to you.
Connect the supported log stream and confirm which site and request data it supplies.
Review a crawler’s function, accessed paths and returned statuses. Separate blocked requests from successful access.
After addressing an access issue, return to the logs. Check access separately from answer visibility and human traffic.
Connect a supported log stream to your property in Heading, then confirm the site and request data it supplies. The crawler view uses those logs to show requests, accessed pages and response statuses.
It is read from your own server logs through the supported log-stream integration. Coverage depends on the requests captured and delivered by that source.
No. A request records crawler access. Citation evidence comes from the captured AI answer, and a crawl does not prove that a citation followed.
They describe different crawler functions. Combining them can hide what actually happened and lead to misleading conclusions about AI search discovery.
The view includes response statuses and blocked or failing requests. Inspect the source logs and configuration to diagnose the reason rather than assuming every failed request has the same cause.
See how AI crawlers access your site and turn the log evidence into a focused technical investigation.
Apply for access