Robots.txt Generator
Build a robots.txt for your site with a guided UI.
Preset
Block 1
robots.txt
User-agent: * Disallow:
🔒 Generated entirely in your browser.
About robots.txt
A robots.txt file tells well-behaved crawlers which parts of your site to skip. It is advisory — malicious bots ignore it. Use it for crawl management, not security.
- Put the file at the site root:
https://example.com/robots.txt User-agent: *applies to all crawlersDisallow:alone (empty value) means allow everything- Include a
Sitemap:line so crawlers can find your sitemap
You might also like
- Sitemap ValidatorValidate sitemap.xml against the protocol and inspect every URL inside.
- Hreflang Tag GeneratorGenerate <link rel="alternate" hreflang="…"> tags for international SEO from a simple table.
- Keyword Density CheckerAnalyse text for word, phrase, and n-gram frequency — see what your content is actually about.
- Slug GeneratorTurn any string into a clean, URL-safe slug with configurable separators and length.