What is A/B and Multivariable Testing?

In our previous post, we gave a step-by-step tutorial on tracking your website statistics using Google Analytics.  If you do not have any form of reliable website analytics, I strongly urge you to check out our post here.

Now that you’re receiving reliable traffic statistics on your site, the first question you probably have is:

“Why are my goal conversion ratio so low?”

This is the main question that drives many of our previous and future clients to us.

And the answer is quite simple:  Your website is not doing a good job persuading its users.

So how do you create a site that better convinces your user to act?

Simple – you test, test, and test.

There are literally an infinite amount of variations you can try testing.  Not only that, but a graphic or ad copy that worked for one site does not mean it will work for you.  Since every single site is a little bit different in its topic, traffic, demographic, etc. the only way to know what will increase you website conversions is to test it yourself.

There are two popular methods of website optimization test.

  1. A/B Testing
  2. Multivariable Testing

A/B Testing

The first and easiest method of testing is an A/B test.  An A/B test simply means you are testing two or more versions of your website against each other to see which one converts better.

This type of testing will simply split the traffic to your different landing pages evenly and see which page converts better.  The statistical comparison is really simple.  After enough traffic goes to both our variants you can compare the conversion rates and choose a winner.

The math is easy but it is a bit more involved as you have to use standard deviation but that will be discussed in a different post.

The benefits of an A/B test is you can get results relatively quickly with little traffic.  If you don’t have much traffic, I highly recommend creating an A/B testing plan.

Multivariable Testing

The more complicated but more efficient method of testing is a multivariable test.  While an A/B testing method tests two or more different pages as a whole, a multivariable testing method tests one more more different areas of a single page.

To start a Multivariable test of the landing page the first thing I need to do in my testing plan is to decide which areas to test. Let’s say I decide to test the main background image and the contact-us button on the home page of Conversion Voodoo.

Let’s say I want to test two different variants for each of the 2 sections outlined above. First, for the background section, let’s say I want to test the control (this is the original version) and a new variant. Similarly, I wanted to test the contact button text “Contact Us Today” against something like “Apply Now”.

Now when I start the multivariable test, each user will see a different permutation of all the variants.  For this example test there are a total of 4 different landing pages (ie. Original Background + Original Button Text, Original Background + New Button Text, New Background + Original Button Text and finally New Background + New Button Text).

After we get enough traffic to run through our test, we can use statistical analysis theory to calculate how each different variant increased or decreased conversions.  The math is a lot more complex and is outside of the scope of this post but most multivariable testing engines automatically calculate this for you.

So with multivariable testing we were able to test 4 different variations of this page and extrapolate exactly which variant made an impact.  For example, we can find out that the new picture increased conversions by 12% or that the new contact button decreased conversions by 4%.

With this information at hand we can create the best landing page by using all of the winning variants.

With an A/B test the only information you can reliably infer is which page as a whole performed better and you are less likely to learn which specific items convert better for your traffic.

I hope this helped you understand the differences in A/B and Multivariable testing and in an up-coming post we will walk you through step-by-step on how to create these tests using Google’s free tool, Google Optimizer.

Talk to a conversion rate expert now

Contact Us

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.