Cool Robots.txt Tool
Hey, you're new. We like you already! You obviously have great taste! If you like what you read here you'll probably want to subscribe to our RSS feed (or the audio RSS feed). Stick around and be sure to speak up and post a comment or two!
I recently found an awesome tool to help you create a valid robots.txt. Many sites lack a valid robots.txt file which is putting your site at many different risks. The Robots.txt Builder allows webmasters to create unique and authentic robots.txt files in a very easy to use interface. The tool allows users to to scan their site using Yahoo Site Explorer to help find directories that you may not want being cached. Typically, these might include administrative areas or sensitive customer information databases.
Image no longer available
Overall, I found the tool to be quite useful. It allowed me to quickly generate a robots.txt file and give me the relief that I made it correctly.
Like this post? Subscribe to the RSS feed and get lots more!
Leave a comment or trackback from your own site.
Tags: Directories, Search & Marketing, Yahoo
Posted in Search & Marketing
