Robots.txt

A file used to keep Web pages from being indexed or to tell which pages you want a search engine to index.