Can AI read your website?
A dedicated look at AI crawler access, content and discovery.
Go beyond the website health basics.
- Crawler access
- Readable content
- Clear next steps
Allow up to 40 seconds. We will show the findings together when the check finishes.
- The page and its robots.txt rules
- Requests using AI crawler names
- Sitemap, llms.txt and Markdown
- Agent discovery documents
AI readiness report
What is done and what is left
FoundationsScored
Beyond the basicsNot scored
A snapshot of this page, not a prediction of AI mentions. Recheck after site changes.
Build on the basics
Give AI more to work with.
Pick a capability to see what it adds and how to get started.
Not every site needs every integration. These are opportunities, not faults or a prediction of AI mentions.
No capability results were recorded in this report. The available checks are in All evidence.
Repair the foundations
What needs fixing?
Start with the most important changes.
Use with your AI tool, or send to your web developer.
No technical repairs found
The completed core checks passed. This does not measure how useful your content is for a particular question.
Make your expertise easier to find
Content & discovery opportunities
Suggestions based on this page and its links, not a prediction of search rankings.
No content opportunities were recorded. This is not a full assessment of your content strategy.
The recorded evidence
What did we actually check?
Crawler access & training policiesService-by-service results for ChatGPT, Claude and other crawlers
Crawler access by service
Your declared rules and test requests from Scavo using crawler names. These are not visits from the AI providers themselves.
AI training choices
Whether model-training crawlers may use your content. This is your call and does not affect the score: blocking training does not stop AI search from citing you.
Recorded check results
Use with your AI tool, or send to your web developer.
Read the brief
Use with your AI tool, or send to your web developer.
Read the brief
How the score works
Technical readiness, not an AI ranking
The score covers the basics: crawler access, readable page content and links that help crawlers find your other pages. It does not measure mentions or citations in AI answers.
Blockers matter most. A noindex instruction or a page that is empty without JavaScript limits the score, even if other checks pass.
Optional means optional. Missing llms.txt, Markdown, agent interfaces or WebMCP does not cost points. Neither does blocking AI training. Broken links to files you already advertise are still worth fixing.
- AStrong technical basics 85-100
Most checks passed in this test.
- BA few gaps to review 70-84
Several technical checks could be improved.
- CRoom to improve 55-69
Review the access and content findings.
- DImportant issues to review 40-54
Several technical checks need attention.
- FMajor technical gaps 0-39
Start with the recorded blockers and evidence.
Questions
Questions about AI readiness
What the score means, how we test crawler access, and what to do with the results.
Browse AI visibility guidesWhat does the AI readiness score measure?
How ready one page is, technically, for AI crawlers and agents. Access counts for 40 points, readable content for 35 and discovery for 15, with up to 10 bonus points for agent-ready extras. It does not measure how often AI answers mention your business; for that, look at referrals in your analytics or the AI performance report in Bing Webmaster Tools.
How do you test whether AI crawlers are blocked?
We read robots.txt, then make bounded requests using selected crawler user-agent names. These requests come from Scavo, not the AI providers. A different response is something to investigate, not proof that the real crawler is blocked. Your server or CDN logs are needed to confirm actual provider visits.
Should I block AI training crawlers?
That is your decision, and the score ignores it. Search crawlers such as OAI-SearchBot, Claude-SearchBot and PerplexityBot are what let AI answers find and cite you. Training crawlers such as GPTBot, ClaudeBot, Google-Extended and CCBot collect content to train future models. You can block training and still be found.
Do I need an llms.txt file?
No. It is an optional publishing convention, not a requirement for AI search or a guarantee of citations. This checklist records whether a readable guide is available. Add one only if it helps your intended readers or tools, rather than just to increase a score.
What do I do with the AI prompts?
Paste a prompt into Claude Code, Cursor, GitHub Copilot or a similar tool, or send it to your developer. Each one includes what we found, the evidence and the change to make, and asks the tool to keep restrictions you set on purpose. Review the changes before they go live, then run the check again.
Can I check my whole website?
This free check looks at one page at a time, so try your homepage and your most important service or product pages. Scavo's monitoring plans add recurring website health checks and uptime alerts. They do not audit every page of a large site.
Keep an eye on your website
Let Scavo do the regular checks.
Get uptime alerts and weekly health checks, with findings you can send straight to your web team.