See what a crawler finds on your page.
Check one public domain or URL. You get four clear answers and concrete next steps.
Fetching and checking the page safely …
The check could not be completed.
Crawlers can access it
Indexing allowed
Main content detectable
Observed crawler rules
Search, training and a user-triggered fetch are separate jobs. This table shows only what the observed robots.txt says for each product token.
| Crawler | Purpose | Access |
|---|
An allowance does not prove a visit. Blocking GPTBot does not automatically block OAI-SearchBot for ChatGPT Search. Review roles and sources →
What to do next
Technical details
Take the result with you or compare it later
Up to five results stay only in this browser and are discarded after 24 hours.
What changed since the previous check?
Compare the response for another crawler token
AnalyseSpider sends one second bounded request with a documented User-Agent and shows the exact string in the comparison. This simulates a header, not the identity or full behaviour of the real crawler.
Observed differences
This check examines only the server response for this one URL. No logged-in content, no JavaScript rendering, and no guarantee of Google indexing or AI citations.
Check the whole website
Use a full technical crawl when you need to find and prioritise problems across many pages.
Open Crawl Foundry Site AuditCheck questions and sources in AI search
See which follow-up questions and sources appear in real OpenAI and Gemini queries.
Open AI FanoutAnalyseSpider, Crawl Foundry and AI Fanout are operated by Matthias Ramahi and are not independent recommendations.
How the check works
- Check a URLEnter one public domain or URL and start the check.
- Read the crawler responseWe fetch the server response like a simple HTTP crawler and explain it.
- Act on the resultYou get a few concrete pointers instead of an invented overall score.