Webfoot Robot
Name: webfetcher
Purpose: Indexing
Comments: Generic Web Crawler
Email: L.McLoughlin@doc.ic.ac.uk
Updated:Jan. 2006-2007
Advanced Robots.txt Generator Website.