The first step for creating your Robots.txt file is to enter your project details:
These details include:
Project Name: Choose a name for your project (mandatory field)
Project URL: Enter the URL for your project (mandatory field)
Local Project Directory: Select the directory in your local PC where your web project / site is located. (mandatory field)
FTP Host: Enter the ftp host for uploading your files to your server.
FTP Username: Enter the FTP username.
FTP Password: Enter the FTP password.
Host Directory: Enter the root directory path of your site in your web server.
FTP Listing: For websites with no local copy a local copy is created that mirrors the remote directory structure so that the robots can be selected on this list.
Site not on local PC: By checking this box you specify that you do not have local access to your files.
Directory prefix: Enter a directory prefix to appear in the paths that are created in the Robots.txt file (advanced option).
Sitemap: Enter the location of the sitemap of your website (supported by some robots).
Crawl Delay: Time delay which the robots must hold before returning to the website (supported by some robots).
Advanced Robots.txt Generator Website.