| Advanced Robots.txt Generator Help |
 | About Robots.txt Generator |
 | What is New |
| About BasiSoft Software |
| Online Help |
| Manual |
| General |
| Terminology |
| File Types |
| User Interface |
| Startup Screen |
| Properties Screen |
| Commands |
| File |
| New |
| Open |
| Close |
| Save |
| Save As |
| Exit |
| Actions |
| Add Robot Control |
| Edit Robot Control |
| Delete Robot Control |
| Compile Robots.txt |
| Upload robots.txt |
| Add Robot |
| Sync Local Directory with FTP Site |
| Properties |
| Help |
| Help |
| Online Help |
| Contact Support |
| About |
| About |
| Visit Our Website |
| Robots.txt in Six Easy Steps |
| Step 1 - Enter Your Project Details |
| Step 2 - Select Robots |
| Step 3 - Select Files and Directories |
| Step 4 - Repeat steps 2 and 3 |
| Step 5 - Create Robots.txt |
| Step 6 - Upload to your Web Server |
| Advanced Topics |
| Changed Files/Directories |
| Site Not on Local PC |
| FTP Listing |
| Add Robot |
| Download the most Current Robots List |
| Directory Prefix |
| Some non-local files |
| Robots.txt Syntax |
| The Robots META tag |
| Supported Robots |
| Robots List |
| Googlebot |
| Inktomi Slurp |
| MSNbot |
| AskJeeves |
| Alta Vista |
| ABCdatos |
| Acme Spider |
| Ahoy! |
| Alkaline |
| Ananzi |
| Anthill |
| Arachnophilia |
| Arale |
| Araneo |
| AraybOt |
| Architext |
| Aretha |
| ARIADNE |
| Arks |
| ASpider |
| ATN Worldwide |
| Atomz |
| Auresys |
| Backrub |
| BBot |
| Big Brother |
| Bjaaland |
| Black Widow |
| Bloodhound |
| Borg-Bot |
| BoxSeaBot |
| Bright.net caching robot |
| BSpider |
| CACTVS Chemistry Spider |
| Calif |
| Cassandra |
| Checkbot |
| ChristCrawler |
| Churl |
| cienciaficcion |
| CMC/0.01 |
| Collective |
| Combine System |
| ComputingSite Robi |
| ConfuzzledBot |
| CoolBot |
| Cusco |
| CyberSpyder Link Test |
| CydralSpider |
| Desert Realm Spider |
| DeWeb(c) Katalog/Index |
| Die Blinde Kuh |
| DienstSpider |
| Digger |
| Digimarc Marcspider/CGI |
| Digital Integrity Robot |
| Direct Hit Grabber |
| DNAbot |
| DownLoad Express |
| DragonBot |
| DWCP (Dridus' Web Cataloging Project) |
| EbiNess |
| e-collector |
| EIT Link Verifier Robot |
| Elfinbot |
| Emacs-w3 Search Engine |
| Esculapio |
| Esther |
| Evliya Celebi |
| FastCrawler |
| Felix IDE |
| FetchRover |
| Fido |
| Fish Search |
| Fluid Dynamics Search Engine robot |
| Fouineur |
| Freecrawl |
| FunnelWeb |
| gammaSpider, FocusedCrawler |
| gazz |
| GCreep |
| GetBot |
| GetterroboPlus Puu |
| GetURL |
| Golem |
| Grapnel/0.01 Experiment |
| Griffon |
| Gromit |
| Gulper Bot |
| Hamahakki |
| HamBot |
| Harvest |
| havIndex |
| HI (HTML Index) Search |
| HKU WWW Octopus |
| Hometown Spider Pro |
| ht://Dig |
| html_analyzer |
| HTMLgobble |
| Hyper-Decontextualizer |
| iajaBot |
| IBM_Planetwide |
| Imagelock |
| IncyWincy |
| Informant |
| InfoSeek Robot |
| InfoSeek Sidewinder |
| InfoSpiders |
| Inspector Web |
| IntelliAgent |
| Internet Cruiser Robot |
| Internet Shinchakubin |
| I, Robot |
| Iron33 |
| Israeli-search |
| JavaBee |
| JBot Java Web Robot |
| JCrawler |
| JoBo Java Web Robot |
| JoBot |
| JoeBot |
| Jubii Indexing Robot |
| JumpStation |
| image.kapsi.net |
| Katipo |
| KDD-Explorer |
| Kilroy |
| KIT-Fireball |
| KO_Yappo_Robot |
| LabelGrabber |
| larbin |
| legs |
| LinkScan |
| Link Validator |
| LinkWalker |
| Lockon |
| logo.gif Crawler |
| Lycos |
| Mac WWWWorm |
| Magpie |
| marvin/infoseek |
| Mattie |
| MediaFox |
| MerzScope |
| MindCrawler |
| mnoGoSearch |
| moget |
| MOMspider |
| Monster |
| Motor |
| Muncher |
| Muninn |
| Muscat Ferret |
| Muncher |
| NDSpider |
| NEC-MeshExplorer |
| Nederland.zoek |
| NetCarta WebMap Engine |
| NetMechanic |
| NetScoop |
| newscan-online |
| NHSE Web Forager |
| Nomad |
| Northern Light Gulliver |
| nzexplorer |
| ObjectsSearch |
| Occam |
| OntoSpider |
| Openfind Data Gatherer |
| Orb Search |
| Pack Rat |
| PageBoy |
| ParaSite |
| Patric |
| Pegasus |
| PerlCrawler |
| Phantom |
| PhpDig |
| PiltdownMan |
| Pimptrain.com's robot |
| Pioneer |
| Poppi |
| Popular Iconoclast |
| PortalB Spider |
| Portal Juice Spider |
| psbot |
| Raven Search |
| RBSE Spider |
| Resume Robot |
| RixBot |
| RoadHouse Crawling System |
| Road Runner: The ImageScape Robot |
| Robbie the Robot |
| RoboCrawl Spider |
| RoboFox |
| Robot Francoroute |
| Robozilla |
| Roverbot |
| RuLeS |
| SafetyNet Robot |
| Search.Aus-AU.COM |
| SearchProcess |
| Senrigan |
| SG-Scout |
| ShagSeeker |
| Shai'Hulud |
| Sift |
| Simmany Robot |
| Site Searcher |
| SiteTech-Rover |
| Site Valet |
| Skymob.com |
| SLCrawler |
| Sleek |
| Smart Spider |
| Snooper |
| Solbot |
| Speedy Spider |
| SpiderBot |
| Spiderline Crawler |
| SpiderMan |
| spider_monkey |
| SpiderView(tm) |
| Spry |
| Suke |
| Suntek |
| Sven |
| Sygol |
| TACH Black Widow |
| Tarantula |
| Tarspider |
| Tcl W3 Robot |
| TechBOT |
| Templeton |
| AskJeeves / Teoma |
| The NorthStar Robot |
| The NWI Robot |
| The Peregrinator |
| The Python Robot |
| The TkWWW Robot |
| The Web Moose |
| TITAN |
| TitIn |
| TLSpider |
| UCSD Crawl |
| UdmSearch |
| UptimeBot |
| URL Check |
| URL Spider Pro |
| Valkyrie |
| Verticrawl |
| Victoria |
| vision-search |
| void-bot |
| Voyager |
| VWbot |
| W3M2 |
| w3mir |
| Walhello appie |
| WallPaper / Crawlpaper |
| w@pSpider by wap4.com |
| WebBandit Web Spider |
| WebCatcher |
| WebCopy |
| Web Core / Roots |
| Webfetcher |
| Webfoot Robot |
| Webinator |
| weblayers |
| WebLinker |
| Weblog Monitor |
| WebMirror |
| WebQuest |
| WebReaper |
| webs |
| Websnarf |
| WebSpider |
| WebStolperer |
| WebVac |
| webwalk |
| WebWalker |
| WebWatch |
| Web Wombat |
| Wget |
| whatUseek Winona |
| WhoWhere Robot |
| Wild Ferret Web Hopper |
| WWWC |
| World Wide Web Wanderer |
| World Wide Web Worm |
| XGET |
| XYLEME Robot |
 | F.A.Q. |
 | Register Advanced Robots.txt Generator |
| Support |
 | Online Help |
© 2008 BasiSoft Solutions