A robots.txt file enables you to command all or certain search engines’ spiders to steer clear of all or particular pages or files on a given Web site. A robots.txt files is a plain-text file that must be uploaded to your Web site’s root directory. Once uploaded to your site, visiting search engine spiders will read the robots.txt file and implement its directions before accessing the affiliated Web site. All search engine spiders comply with robots.txt standard.

Be Sociable, Share!

• 

No related posts found.