搜索引擎爬虫蜘蛛的USERAGENT大全

百度爬虫
Baiduspider+(+http://.baidu./search/spider.htm”)

google爬虫
Mozilla/5.0 (patible; Googlebot/2.1; +http://.google./bot.html)
Googlebot/2.1 (+http://.googlebot./bot.html)
Googlebot/2.1 (+http://.google./bot.html)

雅虎爬虫（分别是雅虎中国和美国总部的爬虫）
Mozilla/5.0 (patible; Yahoo! Slurp China; http://misc.yahoo../help.html”)
Mozilla/5.0 (patible; Yahoo! Slurp; http://help.yahoo./help/us/ysearch/slurp”)

新浪爱问爬虫
iaskspider/2.0(+http://iask./help/help_index.html”)
Mozilla/5.0 (patible; iaskspider/1.0; MSIE 6.0)

搜狗爬虫
Sogou web spider/3.0(+http://.sogou./docs/help/webmasters.htm#07″)
Sogou Push Spider/3.0(+http://.sogou./docs/help/webmasters.htm#07″)

网易爬虫
Mozilla/5.0 (patible; YodaoBot/1.0; http://.yodao./help/webmaster/spider/”; )

MSN爬虫
msnbot/1.0 (+http://search.msn./msnbot.htm”)