Google PlusFacebook iconTwitter icon+44 113 260 4010 contact@branded3.com

Blogstorm »

WordPress Crawl Rate Tracker Plugin

With the advent of Google Analytics and other JavaScript based solutions a lot of webmasters have lost the ability to easily analyse log files to see how search engines are interacting with their site.

Update: A new version of this plugin compatible with WordPress 3+ is now available here.

Google Webmaster Central offers a nice graphing feature so you can see the crawl trends but there is no way to access meaningful data. Studying crawl rate is hugely important because it helps you find out which pages are trusted most by the search engines (especially Google). The more Google trusts a page the greater the frequency it will re-visit that page. If you build a few deep links and see your crawl rate for the target page suddenly increase you can be confident that the links are highly trusted and that you should try to get some more.

Today we are releasing a plugin for WordPress that allows you to track crawl activity for MSN, Yahoo & Google from your dashboard.

  • Access crawl data for 3 main search engines
  • See the exact times a spider accessed any page on your site
  • Monitor your crawl rate over time
  • Discover how crawl rate is related to PageRank

Download the plugin here.

Screenshots

Crawl Rate

Crawl Rate

Crawl Rate

A big thanks to Open Flash Charts for the graph function.

BY Patrick Altoft AT 10:10pm ON Monday, 25 February 2008

Patrick Altoft is Director of Search at Branded3 and has worked in the SEO industry for over 10 years. With experience across some of the worlds largest brands as well as startup businesses Patrick is well known in the industry and speaks regularly at the major SEO conferences and events. Follow Patrick on Twitter or Google+

Comments

← Older Comments

  • Pingback: Ошибки верстки приводят к бану страниц в Google » Оптимизация блога » WPbloging - профессиональный блоггинг на Wordpress

  • http://www.medyum.tc Medyumlar

    Thanks for a great plugin!

  • http://cybro.info cy Roberto

    It is a nice information and now have this plugin installed in my wpress blog.

  • http://www.sheilamarshallmft.com Couples Therapy

    Thanks for the plugin, awesome.

  • http://www.geschenkeparadies.eu Geschenkideen

    Thats a fine plugin, really funny and useful, thanks!

  • http://zashime.0fees.net/blog zashime

    This is nice…. thanks for this

  • http://www.gagner-de-argent.info Luc namessi

    Great post. This plugin is awesome. Thank for the share.

  • http://www.gmstop.com/ Jocuri online

    Thanks for this useful plugin!!!

  • http://chovy.com chovy

    Any partiulcar reason you don’t submit this plugin to wordpress official site?

  • http://www.extremecreative.com.au web design brisbane

    I have been waiting for this for a while!! Thanks. By the way, I had problems too, but Omry Yadan tip helped me a lot
    Thanks Omryyyyyyyy!!!!

  • http://www.bydaughters.com design babae brisbane

    greata rticle..just what i needed..thank you very much for sharing

  • http://www.bydaughter.com Graphic Chick Brisbane

    Very cool. Definitely something I need!!

  • http://www.bydaughters.com Graphic Chick Brisbane

    Very cool – thanks for sharing this. I am going to add it to my blog asap.

  • Pingback: Blogs – The wave of the future? « Damiano's AEG5018 Blog

  • http://fritzimages.com Ed

    Plugin problems in my theme
    Fatal error:
    Fatal error: Cannot redeclare curpageurl() (previously declared in /home/content/21/5301021/html/blog/wp-content/themes/magazine-premium/functions.php:1901) in /home/content/21/5301021/html/blog/wp-content/plugins/crawlrate/sbtracking-log.php on line 69

  • http://deseneonline.org/ Desene online

    Great post. This plugin is awesome. Thank for the share.

  • Spirou

    Really nice plugin, however I’m having some issues, graph is not showing, all I get is a blank square with an url inside it “…wp-content/plugins/CrawlRate/sbtracking-chart-data.php?chart_data=1″ (removed the domain name) Any idea on how to fix this? Btw I’m on wp 2.9.2

  • http://www.dovizbilgi.com döviz

    Thanks for a great plugin!

  • http://www.enduroturf.com.au Grass hopper

    yes, this definately looks like something worth trying. Thank you.

  • Rob Davies

    Great script. I’m looking at using this for my site (non WP). Also, bing has changed their bot name recently (they use msn and bing in the agent).

    Rob

  • http://proactoltestimonials.com Mack

    Gonna try it also.

  • http://www.pixxelpassion.de Chris Donner

    Great tool, using it since a few days now and really glad with it.

  • http://www.bindasrocks.com ramji

    thank you very much this plugin is very useful for me.

  • http://www.zcmhi.com ZCMHI

    It would be better to support Chinese search engines such as baidu,sogou,youdao and soso.

  • http://internetmarketingbizblog.com http://internetmarketingbizblog.com

    Awesome plugins and many thanks for sharing…This is definitely something worth installing for all blogs.

  • http://pamanojseo.wordpress.com seo in chennai

    What do you think of pop-under exchanges? I’m having trouble getting visitors to my site,
    so I’m thinking about trying one. Are they effective????????

  • Pingback: 15 Wordpress Plugins I'm Using and Why You Should Too - MarketingProfessor.com

  • https://sites.google.com/site/billighandykaufen/ Harrison

    This is a good selection of sites you made. My favorite of your selection is Ben Hulse Design.

  • http://www.joomla-web-developer.com Joomla Developer

    Hi there.
    THX for your nice Plugin, This is definitely something worth installing for all blogs.

  • http://makingonlinemoney.site88.net Dion

    Hi,

    I’ve just copied the plugin but it does not come up. I activate it but can’t find any settings

    Any suggestions how I can make it work?

  • http://www.ebusiness-club.info Tom

    Awesome plugin, but after I updated to WP 3.1 the graph function is not working anymore.

  • http://asseenontvpromoters.com Dale

    My issue is the same as Tom, i am using WordPress 3.1 and not getting the graph, any solution to this? would like to see how it works thanks today is March 16, 2011

  • http://www.bloggz.net Atif

    I am also using this wordpress plugin – Thanks

  • http://www.aptus.net.au Aptus Brisbane IT Support

    love this plugin! Howver need to work on my interpretation of the information, somthing i struggle with on google analytics. Be kind i have just started on my web design word press jopurney!

  • http://www.cheap-kcups.info Rob

    Hi, I found this plugin to be awsome. Great work mate!!

  • http://fun-games-to-play-when-bored.info/ Fun Games To Play When Bored

    Very handy! I was using AWstats on my wordpress page and it looked like i was getting alot of traffic…. turns out it was spiders! :( ohwell least the search engines are finding me

    Thanks again

  • http://www.buildamagneticnetwork.com MLM Training

    Great plugin. Thanks. Now I don’t have to go into Awstats all the time!

  • http://www.labeltemplates.info Nick Laton

    Great plugin. I’ve downloaded and can see the stats already.

    Well done.

  • http://mysarasotafloridahomes.com Sarasota Real Estate

    I am using it on 3.1.2 and it seems to be broken. I see a couple other comments to the same effect. Any solution??

  • http://jeremypenrod.com/ Utah Freelance Web Design

    This is an incredibly cool plugin. Thanks so much for doing this and releasing it free! I especially loved that it went back through my logs and filled in data from the last month. Neat to see data instantly. I’m using it on my freelance web design portfolio site and am fascinated to watch how Google crawls my site.

    I’m working on a roundup of useful SEO/Internet Marketing WordPress plugins for a blog post, and this one is certainly in the top 3. Excellent work.

  • http://livingontheinternet.com Garry

    Hi,

    I downloaded the plugin, but the chart does not show up. It just shows a blank square. I am using Firefox. Is there any other thing I need to do?

    Regards,
    Garry

  • http://www.ford.com Kylee Houston

    ;-) a link to this posting was provided by Themelis Cuiper – search engine result advert visionary – so you must be doing a beautiful job?

  • http://vinylflooringonline.co.uk Vinyl Flooring

    In the event you keep on submitting helpful articles or reviews on your web site, there is always a larger opportunity that a number of individuals will go to your web page.

  • Pingback: Eleven must have Wordpress plugins for your blog

  • http://www.mattfraser.name Matt Fraser

    I really like this plugin as I’ve used it before but it doesn’t seem to be working with the latest version of WordPress, any chance an update is going to be coming soon?

  • Pingback: How to log search engine spider access to your site

  • http://www.groomweddingspeech101.net Marc @ groom wedding speech

    This tool is really great. No more guessing but raw backed up statistics!
    Thanks for sharing this with us!

  • http://odkuwki.blogspot.com/ blog

    Tool Crawl Rate Tracker Plugin is really, really great.
    Thanks very much

  • http://selectedmultimedia.com Tvrtko

    Great plugin!

  • http://www.jocuridefacutmancare.org jocuri de facut mancare

    i love this plugin because using it since a couple of days and seeing really good results and crawl stats. Thanks…

  • http://bestskiholidays.blogspot.com/ Best Ski Holidays

    Great post. This plugin is awesome. Thank for the share.

  • http://www.aloeverapflanze.net Aloe vera Pflanze

    Great post. This plugin is awesome. Thank for the share.

  • http://www.orchidbox.com frank orchid box

    I guess now in 2012 WP has different plug ins. It would be great if you could update this post.

  • http://twrix.com Nayan

    It give more details then what my Hosting Provider & Google Webmaster gives.

    nice plugin !

  • http://www.what-is-the-illuminati.com What is the illuminati

    Very cool – thanks for sharing this. I am going to add it to my blog asap.

  • http://www.what-is-the-illuminati.com What is the illuminati

    i love this plugin because using it since a couple of days and seeing really good results and crawl stats. Thanks…

  • http://getbestvacuumforpethairremoval.com Garry

    Hi,

    I installed the plugin , on my site Best Vacuum For Pet Hair Removal but it gives me an error as below. I am using wordpress 3.3.1
    ===
    Open Flash Chart

    JSON Parse Error [Syntax Error]
    Error at character 0, line 1:

    0:
    ===

  • http://www.kopfschmerzen-hilfe.com Kopfschmerzen Hilfe

    Nice plugin and I have been using it on my blog for about two years now. I am wondering if it has been updated since it’s release?

    I am using it on Wp 3.3.1 with no issues, but am about to install it on a friend’s new blog and wanted to see if the download on this page is the most current.

  • http://www.islandcottagemaldives.com Akram

    This is so cool but will this have any effect on my server load

  • http://www.marcorealestatestars.com/ Marco Island Real Estate

    Getting an error message. What did I do wrong?

  • http://www.erocket.co.uk Dave Fowler

    Patrick, thanks for the updated link for WordPress 3. Nice to see this useful post (and plugin) is remaining useful; not all that common in the digital graveyard of old blog archives :-)

  • http://www.patrickwagner.com/ Patrick Wagner

    This look like a very useful plugin – I can’t wait to test it out. I guess a user could better understand if their efforts are providing more crawls or less.

  • http://www.facebook.com/grant.corban Grant Corban

    This plugin was last updated 494 days ago, and is Compatible up to:
    3.1.4. We are now at wordpress 3.5. It works fine on my website, and
    helped me track down some really old links I thought I had cleared off
    years ago, however I have never been able to clear my logs. The error
    states,

    “WordPress database error: [You have an error in your
    SQL syntax; check the manual that corresponds to your MySQL server
    version for the right syntax to use near '' at line 1] DELETE FROM

    Thisplugin shows my site gets crawled close to 1,000 times a day, and the
    log keeps growing. Due to this, and the concerns of this plug in not
    being updated – there are always concerns with security with older
    plugins, i guess I will have to disable it. A shame as it is a fantastic
    plug in for knowing you are being indexed.

  • http://twitter.com/ibilarma PHSEO

    Hi,

    We used this plugin to all of our clients and it really works. It helps our seo team to monitor the crawl rate of bots however, can you add a reporting feature that will export the data so we can present it to the clients. For now, we just used the screenshot I think it would be better if the plugin has reporting tab. Looking forward for you reply. Thanks

← Older Comments
Feedburner Users

Tags

.Net AJAX Android apps Bing Branded3 C# CMS competwition conversion CSS development e-commerce email F# Facebook features Gmail Google HTML HTML5 IE9 interactive internet Internet World JavaScript Leeds Digital Festival Microsoft mobile web PHP rankings SEO smartphones social media Twitition Twitter URL user experience web design web usability Windows Windows 8 WordPress YouTube Zend Framework

© 2013 Branded3 Search Limited Company reg. no. 06479012, All rights reserved.
Part of

St Ives Group Logo