Should You Split Test Your Pricing Strategy?

To give you an idea of how powerfully pricing can influence consumer behavior, consider the results of a recent study done by three management professors at Yale. The study gave subjects $1 with which they could either buy one of two packs of gum or keep their money and buy neither. When the researchers priced both packs of gum equally at 63 cents, only 46 percent of subjects made a purchase. By contrast, when they priced the packs of gum at 62 cents and 64 cents respectively, 77 percent of subjects made a purchase.

Clearly, even the slightest variation in price can have a profound impact on sales, which is why more and more companies are trying to take the guesswork out of pricing strategy by split or A/B testing price. Split testing can give businesses insight into what to charge and how to display price by revealing how actual customers respond to different pricing scenarios.

However, the marketing industry remains vehemently divided on the wisdom of split testing price. Some say value-based pricing is impossible without it, while others believe it does more harm than good. Below, we explore the arguments of both sides on the issue.

Split Testing Price is a Terrible Idea

Marketers who oppose A/B testing price believe that it’s penny wise and pound foolish, yielding short-term gains but causing long-term damage. Here are this camp’s three main contentions:

1. Statistically significant results are almost impossible to achieve

For a split test to be valid, you need sufficient test subjects in each pricing scenario to ensure that chance is not responsible for the results. In other words, you need many customers willing to purchase your product or service before you can split test and an even larger number if you want to test multiple customer personas.

As a result, split testing is not feasible for fledgling companies or those with small customer bases. Additionally, if you change the format of your pricing page at all over the course of the test, you will have to redo it to counter any effects the change had, requiring an even greater volume of customers.

2. You anchor some customers and enrage others

Image via Flickr by MarkMoz12

Even assuming you have the large customer base required for valid split test results, the practice is still ill-advised because disparate pricing breeds disgruntled customers. For instance, say you initially offer a product for $20, but split testing reveals $60 is a better price point. The problem is customers who saw the initial price are now anchored at $20, so good luck justifying a 300 percent price increase because you want to maximize revenue.

Moreover, during a split test, you’re offering different prices to different customers arbitrarily. While not illegal, the practice is ethically questionable, and if customers find out about  pricing (which in our social-media-driven world is not unlikely), they might abandon the purchase altogether, hurting sales.

3. It’s all relative

The fundamental flaw of A/B testing price is that the results are relative and thus reveal no absolute, generalizable pricing information. An A/B test simply tells you how one pricing permutation fares relative to another; it does not reveal how customers value your product. In other words, just because B performed better than A does not mean B is the ideal pricing strategy for your business. This would not be a major shortcoming if you could continually A/B test prices, but doing so is untenable for the reasons explained above.

Split Testing Price is a Great Idea

On the other side of the controversy is the pro-split-testing camp, contending that the potentially explosive positive impact of pricing strategy on sales, as evidenced by the Yale study and other research, outweighs the risks and drawbacks involved in split testing price. Their arguments are as follows:

1. It’s the best way to measure the elasticity of demand

Split testing price, especially over time, allows companies to accurately measure demand elasticity, or the percentage change in demand per percentage change in price. Elasticity of demand is a crucial piece of information to determine the price point that will maximize profit. The formula P=MC/(1+(1/Ed)) — where MC is marginal cost, P is price, and Ed is the elasticity of demand — reveals the ideal price point, even if that price was not explicitly tested.

2. You can split test different prices at different times to avoid alienating customers

Customer backlash most commonly occurs when companies split test different prices at the same time, alienating customers who learn that they were given the higher price. Alternatively, companies can change prices over time to see which price point customers prefer. For example, you might offer a product for $50 one month, $60 the next month, and $40 the next. You can then use multivariate regression to control for extraneous factors (e.g., time of year).

3. Strategic pricing levels can sell products with higher profit margins

Among other benefits, split testing price is the best method to determine how to price different tiers of products to sell the most product with the highest profit margin. Consider one of the tests psychologist William Poundstone used to study the effect of price on consumers as an example of pricing’s potential.

Subjects were offered several varieties of beer: cheap, regular, premium, and ultra-premium. When participants could buy the cheap beer for $1.60, regular beer for $1.80, or premium beer for $2.50, only 20 percent opted for the premium beer. However, when participants could buy the regular beer for $1.80, premium beer for $2.50, or ultra-premium beer for $3.40, 85 percent opted for the $2.50 premium beer, 65 percent more than when it was the highest price point. Companies can discover such massive sales differences most efficiently when they split test various price levels.

As you can see, marketers have made compelling cases both for and against split testing price. Whether it’s the right move for your company will depend on the size of your customer base, how you conduct the tests, and what you do with the information.

By Jon Correll