How to handle Googlebot during site downtime

This week one of my clients was carrying out some web design work that required part of the site to be taken offline for several hours.
Google normally spiders the site quite a lot so the client didn’t want to mess up any rankings during the downtime.
The best way to manage this according to Google is to give spiders a 503 (Service unavailable) header code when they visit. This tells search engines that the problem is temporary and they will come back another time.

Serving the error message is easy if its just a static php page and you can even choose the number of seconds Google should wait before coming back.

<?php
header("HTTP/1.1 503 Service Temporarily Unavailable");
header("Status: 503 Service Temporarily Unavailable");
header("Retry-After: 3600");
php?>

Add a nice error message here

However sometimes you need to apply the code to all pages in the site, in this case you need to add this code to your .htaccess file

Options +FollowSymLinks
RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_USER_AGENT} ^.*google.* [NC]
RewriteRule .* /errorpage.php

You should make sure your error page is giving the 503 header as shown in the php snippet above and most importantly test your code as soon as it goes live to check for problems.

BY Patrick Altoft AT 5:28pm ON Friday, 26 October 2007

Patrick Altoft is Director of Search at Branded3 and has worked in the SEO industry for over 10 years. With experience across some of the worlds largest brands as well as startup businesses Patrick is well known in the industry and speaks regularly at the major SEO conferences and events. Follow Patrick on Twitter or Google+