What is a robots.txt file?

A robots.txt file tells search engine crawlers which pages or directories they can or cannot access on your website.

How do I use the generated robots.txt?

Copy the generated text and place it in the root directory of your website (e.g., https://example.com/robots.txt).

No. All processing happens in your browser. Your data never leaves your device.

User-Agent

Crawl Rules

Sitemap URL

Crawl Delay (seconds)

Generated robots.txt will appear here

Validate and analyze your robots.txt file for errors.

Generate XML sitemaps for your website's pages.

Create SEO meta tags for your webpages.

Preview how your page appears in Google search results.

All processing happens entirely in your browser. Your data is never uploaded to any server. Once you close this page, all data is gone.