Preventing PDFs from being spidered.
On an email list I lurk on, someone asked if there was any way to prevent PDFs on their website from being indexed by crawlers, such as Googlebot, MSNBot, and their cousins. A helpful reply pointed tehm to Jakob Nielsen sidebar on Preventing Search Engines from Spidering PDF Files. Nielsen suggesst using your robots.txt file […]