Stealth Googlebot or Google Operating System?

  • 0
  • December 7, 2008
Patrick Altoft

Patrick Altoft

Director of Strategy

Website monitoring company Net Applications has discovered that around 30% of the traffic originating from Google HQ has identifying information stripped from the user agent string.

This traffic, which people are assuming is Google employees, has the operating system identifier stripped from the user agent string.

Speculation is that Google is testing a Google OS amongst employees:

The new data showed that a percentage of Internet users in Google’s offices (principally based in the company’s Mountain View headquarters) are using an operating system that essentially shields itself from detection by stripping traffic of identifying information. Vizzacarro describes this data, known as a user agent, as a string of information that a computer uses to identify itself. Removing it (possibly via a proxy server) means that outsiders like Net Applications can’t tell which operating system a particular Web user is using. (Net Applications uses other methods, like a Web site’s JavaScript to detect other information about a user and determine that the traffic is coming from Google.) About 11% of Google’s Web traffic currently shows up like this. The level fluctuates daily, Vizzacarro says. A few days ago it was around 30%.

Whilst this might be true there are other explanations for the data. My first thought was that Google was using a stealth spider pretending to be a normal user to build up a more accurate picture of the web. Googlebot can’t see images, scripts, Flash and accept cookies. A stealth Googlebot could do all of these things and help build a better index in the future.

Free of charge. Unsubscribe anytime.