As we are all aware and for some time now, the search industry has comprised of more than just the volume of links you can get to point to your domain. Google‘s algorithm is now ever more complex, with an incredibly dense number of elements affecting it.
User experience is becoming ever more a ranking factor. According to Moz’s survey on ranking factors, (a survey given every two years to prominent members of the search community), a page’s perceived value to users is becoming ever more important.
This graph shows what the industry experts had suggested might change, in terms of affecting Google’s ranking algorithm.
Furthermore, even if bounce rate, time on site and other UX based stats weren’t ranking factors, you would still want to get the most out of your traffic. If you did everything else perfectly but didn’t take your user experience into account, it doesn’t matter how much traffic you drive to your website; it will have all been futile if users cannot convert.
This is where most internet marketers or developers might have suggested A/B Split testing of some kind. This works by a simple piece of Google code being added to the page you want to test, that can then either keep traffic on the original page or divert it to test page, in a random even split. Eventually a winner will be declared by the software. The level of accuracy and the time for which the test is run can both be determined by the webmaster.
The issue here, is that if page A has 4 issues that hinder conversions and page B has 2 issues that hinder conversions, page B will come out as the winner without highlighting the two issues that remain on the page. So your page has improved, but it hasn’t reached its full potential. In this scenario, split testing was not the best method of testing.
A popular alternative that would have provided a better fix in this example, could have been lab testing. This is when unbiased individuals under optimum conditions undergo the challenge of converting on a website; with minimal direction, narrating to the tester as and when he finds a hindrance. This is repeated with multiple candidates under these conditions until obvious bottle necks are found within the conversion system.
These are then fixed and subsequent iterations of the same test are repeated until the whole system flows smoothly. However one of the major downfalls with this is that it’s too expensive for the majority of small clients.
He was surprised to find that consultants in his industry had found guerrilla testing to be the least popular in a survey run by econsultancy.com.
Guerrilla Testing: Lab testing on a budget.
Chris’ talk looked into guerrilla testing a company’s internal software, as well as completing a task on a website. The company will often have to provide testers who know the process of what the software is for, however this is not the case when completing something like buying a holiday on a travel site or booking a test drive on a car dealer’s page. This is a lot more handy for us search marketers as the candidate pool for the test is rapidly increasing.
Guerrilla testing is essentially lab testing done on site by available unbiased individuals being monitored by an experienced member of your UX team. The tester is made as comfortable as possible and away from distractions, so that the task at hand can be focused on. He is giving a basic and general instruction; e.g. ‘Go on the website in question and buy a holiday to Florida’, and he is then left to his own devices.
Website specific jargon should be omitted from such instructions, as they might help the user overcome an obstacle that future users might have trouble with. The person monitoring the experiment needs to sit back and give the tester a chance to make mistakes, giving hints only to ensure there isn’t a standstill in progress.
Getting the tester to narrate his process can be helpful, as problems can be highlighted on the go. An alternative is to interview the tester directly after the process is completed. Software is even available to screen capture the process, whilst also recording the tester’s facial expressions with a webcam.
This process is repeated with multiple candidates who obviously have never seen the website before, and as few as 5 candidates can have astonishing results. The results are then collated to show areas that most candidates struggled with.
Issues can then be classified by severity (i.e. how much they hamper the conversion process) and complexity (i.e. how hard they are to fix). These fixes can then be implemented.
When should I consider using guerrilla testing?
Guerrilla testing should be used in the same set of circumstances that you would use lab testing. In terms of SEO however I strongly believe that split testing is still a valuable asset when trying to improve on a simple conversion factor such as the placement, size or text of a button on a page. However if you are trying to improve on a more complicated sequence of events and look into how the user interacts with a website from the moment he lands on a page, Guerrilla testing might be the way to go.
- Considerably cheaper than lab testing. (internal resourcing)
- Results of the test can be obtained very quickly (Split testing can take over two weeks).
- Can offer solutions that return quick wins
- Very few tools required. (Microsoft Expressions has screen capture capability)
- Can’t be analytically and numerically proved such as with split testing.
- User might have a biased advantage to a test if it’s not set up correctly.
- UX monitor might miss something or help candidate too much.