Perfecting User Experience Design with A/B Testing

November 2009

As a web designer it's hard to get the user experience design right first time, you never know how users are going to use the site or how big a difference small iterations will make, so it's important to test the user experience and try out different solutions. That's where A/B testing comes in.

A/B testing

A/B testing involves showing different users different versions of a site and measuring how each version performs.

  • Design two (or more) versions
  • Randomly show users different versions
  • Track which version performs/converts better
  • Evaluate
  • Use the best version
Sounds complicated doesn't it? Thanks to Google it's not as hard as you think. Google has a Website Optimizer service that enables you to easily setup A/B tests and then analyze the results for you.

A/B testing in action

I recently setup a very simple test with Sharethrough changing the call to action text of two buttons and measuring conversion to the products page.
Some users seen these buttons… and others seen these (50/50 split)
ab2.jpg
ab1.jpg
A B 5% improvement
Version B showed a 5% improvement on conversions to version A, the original.

I must stress this is a very simple test I put together but it just goes to show that even the smallest text change can make a difference. I can now go onto setup more tests and experiment with button placement, language used etc. while measuring conversions to enquiries and conversions to user signups.

A/B testing resources

Before you get started with A/B testing, check out the following:
Carsonified: How to do A/B testing in Wordpress
picture-281
37 Signals: Writing Decisions: Headline tests on the Highrise signup page
picture-29
UX Booth: How to increase site performance through A/B split testing
picture-301
ABtests.com
picture-31
Google: Website Optimizer
picture-32

Do you use A/B testing?

Please share any experience or tips while doing A/B tests.

« 5 Essential Document Templates for Freelance Designers New Years Resolutions For 2010 »