Robots.txt Rules

Paste robots.txt, enter the crawler’s product token (e.g. Googlebot, not its whole HTTP header), and a path including any query string. Ordinary inputs update directly. For large inputs or potentially costly wildcard matches, choose Test rules or press Enter; use Ctrl/⌘+Enter in the editor.

Case-insensitive exact bot groups are merged. Only if none exists is * used. Paths are case-sensitive; * matches any sequence and a final $ anchors the end. The longest normalized pattern wins, including * and $ in its octet length; Allow wins ties. This priority follows Google’s public matcher. Unicode and percent escapes are normalized, but an escaped reserved separator such as %2F stays distinct from /. Fragments are excluded; query strings remain.

Empty Allow/Disallow directives have no effect but separate bot groups. Invalid lines are reported; unrelated records such as Sitemap are ignored. No sitemap analysis, fetching, caching, HTTP-status or search-indexing simulation; robots.txt is not access control.

Your three exact inputs stay locally in this browser, including unfinished or invalid text. No decisions, matching rules or open detail panels are saved; nothing is uploaded. Limits: 512 KiB UTF-8 text, 16 KiB path and 8 million matching steps; exceeding them produces an error, not partial results. Browser storage limits still apply.