A/B Tests: Why a higher conversion rate is only the beginning

A/B testing gives you more than a lift in conversions. If you test one page each month, you’ll notice a shift. Your marketing team will write different copy. Your design team will create different layouts. Executives will make your coffee (ok, maybe not that last one, but still).

In many ways, A/B tests are only the beginning — and that point is completely overlooked in a recent article from Wired magazine: The A/B Test: Inside the Technology That’s Changing the Rules of Business.

This article is really, really good, so check it out (the print version even has a case study). Brian Christian lays out how A/B testing has taught many companies that the opinion of the highest paid person (HiPPO) isn’t worth a hill of beans compared to data.

The article is loaded with awesome quotes:

• “Most websites suck because HiPPOs create them.” – Avinash Kaushik, Google

• “Assumptions tend to be wrong.” – Dan Siroker, Optimizely

• “In God we trust. All others, bring data.” – Bill Clerico, WePay

The article doesn’t fly off the tracks, but the crazy train starts rolling a little fast when it starts talking about results. Why should editors waste time debating headlines, it implies, when they can test them and let the results decide? And here’s another quote:

“[On a page], why do people like the ottoman better if it appears to the left of the throw rug than if it appears to the right? There’s no time to ask the question, and no reason to answer it. After all, what does it matter if you can get the right result? Keep testing, keep reacting, and save your philosophizing for the off-hours.”

A/B Testing Will Make You a Marketing Genius

Lowering your head and plowing through tests might seem efficient, but it’s really wasteful. Sure, maybe the landing page is earning more conversions, but what else have you got? The answer is nothing unless you take the time to “philosophize” about what happened.

Each test you run is a little lesson, teaching you something about the audience. You just have to look for it. Every test you create, every time you meet to discuss the best way to share your awesomeness with a prospect, it changes YOU. Simply paying attention will grow your team’s customer empathy IQ to stratospheric heights. Making you marketing geniuses in your own market!

• Example: You test an online form that requests five fields of information vs. one that requests six. The form with five fields wins.
  What you learned: Visitors want to work as little as possible.

• Example: You test a single call-to-action vs. four calls-to-action. The page with one call wins.
  What you learned: Pages need to focus on one goal.

• Example: You describe the benefits your product provides vs. a list of its features. The page with the benefits wins.
  What you learned: That your audience wants to know “what’s in it for me?” more than “what does it do?”

If you stick the winners of these tests onto your site and crazily scramble to the next test, then you’re wasting a ton of insight. Sure, you can’t make iron-clad rules about the audience based on a single test, but you can start. You can review your tests and pick up trends. And those trends can save you a ton of headaches.

You will learn about your audience rather than just testing your audience. You won’t waste time testing four calls-to-action, because you’ll know that one is better. You’ll be much better off than the guy who is so busy running to the next test that he doesn’t know where to run for the best ROI.