Yahoo! Slurp 3.0
Over the past few weeks, we've [Yahoo!] been preparing for the latest version of the Yahoo! Search crawler with some infrastructure updates, which recently caused a variance in our crawl behavior.
As the new software undergoes a phased rollout to our production crawlers over the next several weeks, you'll see the following changes:
a) The crawlers will start crawling from a different and much smaller set of IP addresses, but it'll still be from the crawl.yahoo.net domain.
b) The crawlers will also publish a new user-agent, 'Yahoo! Slurp/3.0.'
Link:
http://www.ysearchblog.com/archives/000531.html





0 Comments:
Post a Comment
Links to this post:
Create a Link
<< Home