Ranking & Robots txt file
Posted: Sun Jan 31, 2010 4:43 pm
Robots txt file is very usefull to prevent any bot to index your web site, infact each web site should have because i do not think that many crawlers check and index your web site with having robots.txt