Business / Search Engine Optimization (SEO) / Robots.txt: A file which sits in the root of a site and tells search engines which files not to crawl. Some search engines will still list your URLs as URL only listings even if you block them using a robots.txt file. Do not put files on a public server if you do not want search engines to index them!
Search Google for Robots.txt:
Technology / Home Audio / Spider: The rear suspension element on the cone of a loudspeaker. All the moving parts of the speaker are suspended by the spider in the rear and the Surround in the front. The spider is so named because of i MORE
Business / Internet Marketing / Yahoo Slurp: Yahoo web crawler. Instructions to Yahoo Slurp can be directed in the robots.txt file. Googlebot statistics can be viewed with web analytics software, on webserver logs, and in Google webmaster tools. MORE