# /robots.txt file for http://webcrawler.com/ # mail webmaster@webcrawler.com for constructive criticism User-agent: * Disallow: /attachements Disallow: /cgi-bin Disallow: /images Disallow: /temp Disallow: /tp-downloads Disallow: /FirearmsManuals Disallow: /FirearmManuals Disallow: /Firearm\ Manuals