Perfecting User Experience Design with A/B Testing

November 2009

As a web designer it’s hard to get the user experience design right first time, you never know how users are going to use the site or how big a difference small iterations will make, so it’s important to test the user experience and try out different solutions. That’s where A/B testing comes in.

A/B testing

A/B testing involves showing different users different versions of a site and measuring how each version performs.

  • Design two (or more) versions
  • Randomly show users different versions
  • Track which version performs/converts better
  • Evaluate
  • Use the best version

Sounds complicated doesn’t it? Thanks to Google it’s not as hard as you think. Google has a Website Optimizer service that enables you to easily setup A/B tests and then analyze the results for you.

A/B testing in action

I recently setup a very simple test with Sharethrough changing the call to action text of two buttons and measuring conversion to the products page.

Some users seen these buttons… and others seen these (50/50 split)
ab2.jpg
ab1.jpg
A B 5% improvement

Version B showed a 5% improvement on conversions to version A, the original.

I must stress this is a very simple test I put together but it just goes to show that even the smallest text change can make a difference. I can now go onto setup more tests and experiment with button placement, language used etc. while measuring conversions to enquiries and conversions to user signups.

A/B testing resources

Before you get started with A/B testing, check out the following:

Carsonified: How to do A/B testing in Wordpress

picture-281

37 Signals: Writing Decisions: Headline tests on the Highrise signup page

picture-29

UX Booth: How to increase site performance through A/B split testing

picture-301

ABtests.com

picture-31

Google: Website Optimizer

picture-32

Do you use A/B testing?

Please share any experience or tips while doing A/B tests.

Receive more design content like this to your inbox

I promise not to spam you. No more than one email per week.

I blog and share links about web design, user experience, product development, HTML/CSS, email design and anything else I find interesting.

No thanks