Free Robots.txt Checker

robots.txt tells search engine crawlers which parts of your site they're allowed to visit — but it doesn't remove a page that's already indexed, and a blocked page can sometimes still appear in search results without its content. Paste a URL below to see exactly what your site's robots.txt allows or disallows for that specific path.

Learn more: Robots.txt vs Noindex: What's the Difference (and When to Use Each)