Computers, Internet, Web Design and Development, Authoring, Online Tools, robots.txt

robots.txt

4 Sites in robots.txt

Robots.txt Checker

Analyzes a robots.txt file searching for syntax and "logical" errors, and shows a summary of what effect it will have.

http://tool.motoricerca.info/robots-checker.phtml
Robots.txt Generator

Information on the use of robots.txt as well as a robots.txt generator and a list of search engine robots.

http://www.robotstxt.ca/
Robots.txt Syntax Checker

Online tool for syntax verification to robots.txt files, provided by Simon Wilkinson.

http://www.sxw.org.uk/computing/robots/check.html
Robots.txt Validation

Creator and validator of robots.txt files.

http://www.clockwatchers.com/robots_tool.html

Search Resources