AI Robots.txt Validator & Generator

Check which AI bots your robots.txt allows or blocks, generate new policies per-bot, and get an instant policy grade — free, no sign-up.

Robots.txt editor

Live validator and generator

Paste or fetch a live file

Paste a domain or any page URL and we'll fetch the site's /robots.txt file.

—

Policy grade

Paste or build a robots.txt to see results.

Share

Policy checks

Checks update as you edit.

Sitemaps

Checks update as you edit.

No sitemap directives parsed.

Other crawler directives

No additional user-agent directives detected yet.

No additional bot directives.

Parsed evidence (JSON)

Normalized parser output for debugging.

Best next tool

The obvious next question, answered.

Use the next tool that helps you size the risk, explain the business impact, or validate what to fix next without cluttering the main workspace.

Free AI Visibility Tool Open tool

AI Visibility Score

Once crawler access is clear, measure the bigger picture: how likely your pages are to be discovered, understood, and cited.

AI Visibility

Next step

Fix it once. Scavo makes sure it stays fixed.

Automated weekly scans, uptime checks as often as every minute, and clear fix priorities — so nothing slips through the cracks.

Run free scan Start free trial

Raccoon Cookie Checkpoint

Can we use optional cookies?

Essentials always stay on. Optional cookies help with preferences, analytics, and smarter prompts.

Current setting: Decision needed