Bing Webmaster Tools gets a new robots.txt tester
Bing has added a new feature to the new Bing Webmaster Tools[1]: a robots.txt tester. This is a feature Bing had in the tool prior to 2009 but dropped supporting it for the past ten-years or so. Now it is back and better.
Where can I access it? You can access the new robots.txt tester within the new Bing Webmaster tools over here[2]. From there you can enter a URL to see how BingBot will fetch the URL.
What does it look like? Below is a screenshot of the tool. Enter a URL, test it with BingBot and Bing’s Ad bot and also see how the robots.txt file differs from HTTP to HTTPS:
More details. The Bing help document[3] says that this “Robots.txt Tester helps Webmasters to analyze their robots.txt file and highlight the issues that would prevent them from getting optimally crawled by Bing and other robots.”
Fabrice Canal from the Bing team told us[4], “We resurrected and enhanced our old robots.txt tester.” Canal explained that this was a feature they had many years ago that the company brought back to life and made better.
Google also has a similar tool, you can learn more about it over here[5].
Why we care. Now you can see how BingBot sees your content. If you need to troubleshoot an indexing or ranking issue with Bing, this is one of the tools you can add to your toolkit to see how Bing can access your pages.
About The Author
Barry Schwartz a Contributing Editor to Search Engine Land and a member of the programming team for SMX events. He owns RustyBrick[6], a NY based web consulting firm. He also runs Search Engine Roundtable[7], a popular search blog on very advanced SEM topics. Barry’s personal blog is named Cartoon Barry[8] and he can be followed on Twitter here.
References
- ^ new Bing Webmaster Tools (searchengineland.com)
- ^ over here (www.bing.com)
- ^ document (www.bing.com)
- ^ told us (twitter.com)
- ^ over here (support.google.com)
- ^ RustyBrick (www.rustybrick.com)
- ^ Search Engine Roundtable (www.seroundtable.com)
- ^ Cartoon Barry (www.barryschwartz.org)
Powered by WPeMatico
