Google’s Gary Illyes recently discussed the importance of allowing Googlebot access to websites and the consequences of blocking the GoogleOther crawler. GoogleOther is a generic crawler used by Google for various purposes outside of search, ads, video, images, news, desktop, and mobile. It is utilized for internal research and development by Google’s product teams.
It’s interesting to note that there are three types of GoogleOther crawlers: GoogleOther for public URLs, GoogleOther-Image optimized for image URLs, and GoogleOther-Video optimized for video URLs. These crawlers are used for research and development purposes to enhance Google’s products.
When asked about the non-search features supported by GoogleOther, Gary Illyes mentioned that it is primarily used for research and development crawling, rather than supporting specific features. Blocking GoogleOther may not affect Google Search rankings, but it could impact other Google products and services.
Pros of blocking GoogleOther include saving server resources and preventing Google from using site content for AI. However, blocking GoogleOther could also mean missing out on opportunities to participate in Google’s research for new products or features related to search.
On the other hand, allowing GoogleOther access to a site may raise concerns about privacy and potential misuse of site data. It’s essential for site owners to weigh the pros and cons of blocking GoogleOther based on their specific needs and concerns.
In conclusion, the decision to block GoogleOther is not straightforward and requires careful consideration. Ultimately, site owners should evaluate the potential benefits and drawbacks before making a decision. It’s important to stay informed about the implications of allowing or blocking GoogleOther to ensure the best outcome for your website.
For more insights on this topic, you can listen to the Google SEO Office Hours podcast for further information.