Step 2 - Select the Robots

Top  Previous  Next

 

The next step is to click on the 'Edit Robot Control' button and start selecting the Robots from area A.

 

selectrobots

 

 

As you select the Robots and crawlers in area A their details will appear in area B as part of the Robots.txt exclusion tree.

 

 


Advanced Robots.txt Generator Website.