A/B Tests: Why a higher conversion rate is only the beginning

Posted by 9 Comments

A/B Tests: Why a higher conversion rate is only the beginning

Become a Marketing Genius

A/B testing gives you more than a lift in conversions. If you test one page each month, you’ll notice a shift. Your marketing team will write different copy. Your design team will create different layouts. Executives will make your coffee (ok, maybe not that last one, but still).

In many ways, A/B tests are only the beginning — and that point is completely overlooked in a recent article from Wired magazine: The A/B Test: Inside the Technology That’s Changing the Rules of Business.

This article is really, really good, so check it out (the print version even has a case study). Brian Christian lays out how A/B testing has taught many companies that the opinion of the highest paid person (HiPPO) isn’t worth a hill of beans compared to data.

The article is loaded with awesome quotes:

  • “Most websites suck because HiPPOs create them.” – Avinash Kaushik, Google
  • “Assumptions tend to be wrong.” – Dan Siroker, Optimizely
  • “In God we trust. All others, bring data.” – Bill Clerico, WePay

The article doesn’t fly off the tracks, but the crazy train starts rolling a little fast when it starts talking about results. Why should editors waste time debating headlines, it implies, when they can test them and let the results decide? And here’s another quote:

“[On a page], why do people like the ottoman better if it appears to the left of the throw rug than if it appears to the right? There’s no time to ask the question, and no reason to answer it. After all, what does it matter if you can get the right result? Keep testing, keep reacting, and save your philosophizing for the off-hours.”

A/B Testing Will Make You a Marketing Genius

Lowering your head and plowing through tests might seem efficient, but it’s really wasteful. Sure, maybe the landing page is earning more conversions, but what else have you got? The answer is nothing unless you take the time to “philosophize” about what happened.

Each test you run is a little lesson, teaching you something about the audience. You just have to look for it. Every test you create, every time you meet to discuss the best way to share your awesomeness with a prospect, it changes YOU. Simply paying attention will grow your team’s customer empathy IQ to stratospheric heights. Making you marketing geniuses in your own market!

  • Example: You test an online form that requests five fields of information vs. one that requests six. The form with five fields wins.
    What you learned: Visitors want to work as little as possible.
  • Example: You test a single call-to-action vs. four calls-to-action. The page with one call wins.
    What you learned: Pages need to focus on one goal.
  • Example: You describe the benefits your product provides vs. a list of its features. The page with the benefits wins.
    What you learned: That your audience wants to know “what’s in it for me?” more than “what does it do?”

If you stick the winners of these tests onto your site and crazily scramble to the next test, then you’re wasting a ton of insight. Sure, you can’t make iron-clad rules about the audience based on a single test, but you can start. You can review your tests and pick up trends. And those trends can save you a ton of headaches.

You will learn about your audience rather than just testing your audience. You won’t waste time testing four calls-to-action, because you’ll know that one is better. You’ll be much better off than the guy who is so busy running to the next test that he doesn’t know where to run for the best ROI.

9 Responses to A/B Tests: Why a higher conversion rate is only the beginning

  1. Jeranine says:

    Enjoyed the reminder that results need interpretation. Numbers need knowledge.

  2. Christopher says:

    Killer article! I loved it.

  3. George Smith says:

    Like the messages, love the presentation style!

  4. Shhh… Don’t tell anyone I said this, but real genius comes from process. :-)

  5. Sandy says:

    I love shoemoney!

    Ok, joking, but found this blog today and got to say, useful stuff and humanitarian approach to the data. Exactly, patterns, principles and rules, not just plain testing. I’m fortunate to have multiple sites so we can test and reapply winners so we move 4x faster than if we would have only 1 site.

    Looking forward for more useful stuff and who knows, maybe we need to hire your team!

Leave a Reply

What is 13 + 11 ?
Please leave these two fields as-is:
IMPORTANT! To be able to proceed, you need to solve the following simple math (so we know that you are a human) :-)