Robots.txt Generator

Generate robots.txt rules with crawl directives and sitemap references.

Robots Rules

Configure crawler directives and optional sitemap metadata.

User agent

Allow paths (one per line)

Disallow paths (one per line)

Crawl delay (optional)

Host (optional)

Sitemap URL

Click generate to preview directives before publishing.

Output

Default behavior is allow-all when both allow/disallow lists are empty.

Generate robots.txt rules with crawl directives and sitemap references.

This tool runs entirely in your browser. Your files and input data are not uploaded by Toolhive.