Terminology

Robots. Any software that runs through web pages navigating through the hypertext links and retrieving the content for indexing, monitoring, link validation or other purposes.

Robots.txt. The file that controls the behaviour of robots for specific directories and files of your website.

Robot Control. In Advanced Robots.txt Generator any set of instructions that link specific robots to directories of your website.

Compile. Create automatically the robots.txt file based on the specified Robot Controls in Advanced Robots.txt Generator.

Upload. The process of sending automatically the robots.txt file to your web server.

Advanced Robots.txt Generator Website.