Current Robots and Crawlers List

Advanced Robots.txt Generator Professional can be kept up to date with the most recent list of robots which is updated on a weekly basis.

In order to use the most recent list, download the XML document from the link below, and place it in the installation directory of Advanced Robots.txt Generator.

Current Robots and Crawlers list [03/17/2008] (XML document)

Notice that all robots and crawlers which you have added yourself will be erased once the new list is entered. It is to your advantage to nitify us in bots@basisoft.com on any new bots that you are adding on this list so that they will be added in the next available robots list.

Advanced Robots.txt Generator Standard offers complete control of more than 400 robots and crawlers within minutes.

Advanced Robots.txt Generator Professional is an advanced tool supporting close to 2000 robots and crawlers and allows for use in dynamic (PHP, ASP etc.) websites.

The Professional edition is available for $35, offers weekly updates of the most current robots and crawlers and naturally comes with free documentation and customer support. We also honor a 30 day no-questions asked refund policy.

See complete features list

Creating robots.txt files with Advanced Robots.txt Generator couldn't be simpler. It is just a matter of three steps:

1. Select robots/crawlers from the list of 300 robots.
2. Select the files / directories of your website.
3. Compile robots.txt - and upload automatically to your webserver.

Download Advanced Robots.txt Generator and try it now!

As a special offer this week we offer a free copy of the book 'Pay-per-Click. The Complete Guide'. The first edition was a best seller that has sold 25,000 copies and is now available for free download for all new purchases of Advanced Robots.txt Generator.

Download Advanced Robots.txt Generator and get your free copy now!