It's the Bingbot Now!

Starting October 1st, the Microsoft-owned web crawler (the user agent that helps index websites) is getting a new name: Bingbot. Previously known as the msnbot, the name change reflects the new Microsoft brand for search, Bing.

The Live Search team tells us that robots.txt files written for msnbot won’t be affected, so you don’t need to edit them if you’re a website owner. However, if you create new directives for the Bingbot and don’t remove the older msnbot directives, the Bingbot directives will take precedence. To demonstrate this point, the team showed this example which allows Bing to crawl all whole hosts except on the folder /folder1/:

User-agent: bingbot
Disallow: /folder1/

User-agent: msnbot
isallow: /folder1/
Disallow: /folder2/

User-agent: *
Disallow: /

If you have questions about the new bot, you can reach out to the team at bingbot@microsoft.com.