#The following file "/robots.txt" file specifies that no robots should visit any URL starting with "/stats/": #See also example on http://info.webcrawler.com/mak/projects/robots/norobots.html or #Search for robots.txt in the MSDN library July 2001 or #http://www.robotstxt.org/wc/norobots.html #-------------------------------------------------------------------------------- # robots.txt for http://www.lpdleusden.nl/ User-agent: * Disallow: /stats/ Disallow: /beheer/ Disallow: /docs/