Bing to update BingBot’s user agent
Bing announced[1] today that it “will start the transition to a new bingbot user-agent” as it continues to roll out the evergreen BingBot[2]. Evergreen BingBot, the version of BingBot that is able to crawl the web like a modern browser, is currently in use but will be used to “cover all the sites” over the “coming months.” With that, Bing will begin using a new user agent to convey which version of BingBot is crawling your web site.
New user agent. Bing will use a user agent that identifies the specific version of Microsoft Edge is crawling your site. Here is the format for both desktop and mobile:
Desktop – Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/W.X.Y.Z Safari/537.36 Edg/W.X.Y.Z
Mobile – Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 Edg/W.X.Y.Z (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
“W.X.Y.Z” will be substituted with the latest Microsoft Edge version Bing is using, for eg. “80.0.345.0″.
Old user agent. Here are the current and old user agents for BingBot:
- Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
- Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
- Mozilla/5.0 (Windows Phone 8.1; ARM; Trident/7.0; Touch; rv:11.0; IEMobile/11.0; NOKIA; Lumia 530) like Gecko (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
Do we need to worry? Most sites do not need to worry. Microsoft said “For most web sites, there is nothing to worry as we will carefully test the sites to dynamically render fine before switching them to Microsoft Edge and our new user-agent.” But if you have hardcoded any user agents into your scripts, you will need to revise those scripts to ensure BingBot can continue to crawl your site.
How to test. Bing said you can test it by installing the new Microsoft Edge[3] browser “to check if your site looks fine with it.” Bing said “if it does then you will not be affected by the change.” “You can also register your site on Bing Webmaster Tools[4] to get insights about your site, to be notified if we detect issues and to investigate your site using our upcoming tools based on our new rendering engine,” Bing added.
Google also. Google is also migrating to the new GoogleBot[5] this month. Google is currently testing the new user agents, so you may be able to see them in your log files. I do not believe it is fully rolled out yet for Google.
Why we care. Both BingBot and GoogleBot are getting a new user agents, and that is exciting. But it can impact your site if you had any user agent detection methods for BingBot and/or GoogleBot. Make sure to test your site to see if it supports the new user agent. Most sites probably do not need to worry about this but you have done any advanced bot detection, you may need to take steps to update those scripts.
About The Author
Barry Schwartz is Search Engine Land’s News Editor and owns RustyBrick[6], a NY based web consulting firm. He also runs Search Engine Roundtable[7], a popular search blog on SEM topics.
References
- ^ announced (blogs.bing.com)
- ^ evergreen BingBot (searchengineland.com)
- ^ Microsoft Edge (aka.ms)
- ^ Bing Webmaster Tools (www.bing.com)
- ^ migrating to the new GoogleBot (searchengineland.com)
- ^ RustyBrick (www.rustybrick.com)
- ^ Search Engine Roundtable (www.seroundtable.com)
Powered by WPeMatico
