The Wall Street journal suggests that “Merry Christmas” still reigns in popularity as a season’s greeting over “Happy Holidays”
So how does emailing customers with one or the other impact conversion rates?
We selected a 100,000 customer list for a client who tested these subject lines:
A few additional notes on the test:
- The client is a large retailer, their customers mirror a general sample of the USA.
- The email mirrored the subject line’s message; all other elements were the same.
- We sent the email on the 21st of December.
The results show a HUGE difference . . .
As you can see the “Merry Christmas” subject line drove nearly double the click through rate of the other subject lines.
While we haven’t included the “buy rate” in the chart above due to client confidentiality, the results were similarly impressive.
Nearly doubling the number of transactions by changing an email subject line fairly well illustrates why if you’re not rigorously testing every facet of your online business (or letting us do it for you!), you’re throwing money away.
More fun with email testing:
- Personalizing your subject line can drop conversion rates . . .
- Optimize your email conversion rate with three quick tips . . .
- Unsubscribe links at the top of your emails will improve conversion rates . . .
P.S. – Contact us today to increase your conversion rate . . .
We work on a performance basis and charge nothing up front.
If you want to drastically improve your conversion rate contact us today.
I guess Happy Holidays loses out due to it not being so date specific/critical.
Did the use of & in the subject line impact delivery rates? That column is not there.
Good question – the ampersand was actually the tag we use internally to initiate the proper formatting of the first-name variable.
I think you’re confusing ampersand & used in the combo subject line and percent % used for your first name vars.
I should read my own blog posts more thoroughly.
That’s an excellent question posed – definitely something that a follow-up test could clear up.
On this campaign there were actually a couple of other derivatives once we established the general messaging that I’ll put in another blog post in the future.
Also worth revising the CTR formula. It depends on how you analyze it, but I strongly prefer to calculate CTR as “Clicks” / “OPENS”, instead of “Clicks” / “Total Base” like you did above.
It makes little sense to calculate CTR as a measure of clicks compared to total base, as the absolute number of clicks will be impacted by delivery rates and also number of opens, so you’ll be adding up the effect of several variables – delivery, opens, appeal – into one.
If you adjust this on your calculation above you’ll notice that the CTR didn’t change much on the 3 test (as you would expect, in fact). “Merry Xmas” was still better than the other 2, but the delta was much smaller.
Good test nonetheless! Thanks for sharing.
Good point – left out of the article was my suspicion that we saw a higher than normal block on happy holidays from Yahoo but without supporting data I left it out.
Also missing is the sales data which reinforces the Christmas line enough so that I felt comfortable with the Christmas messaging.
Christmas has a tradition of giving and receiving. It seems possible that recipients may associate gift giving with “Merry Christmas” more than “Happy Holidays” and subsequently will anticipate receiving some sort of gift in the message with the “Merry Christmas” subject line.
Clicks/opens ratio is comparable for both: 33% for “Merry Christmas” vs 26% for “Happy Holidays.” So, this indicates that the subject has a much greater impact on opens vs clicks. Certainly, getting the user to open the message is crucial for overall CTR.
A possible conclusion I draw is not that “Merry Christmas” improves conversion because it is more popular, but rather it improves conversion because it gives the recipient an impression of reward and encourages opening the message. Connotation of subject wording rather than popularity of subject wording could be the leading factor here.
In either case, when it comes to CTR, I completely agree it is important to test different facets of one’s campaign. Two words make a big difference.
That’s an astute observation and my internal copywriter agrees in full – whetting the appetite.
I just proves that christmas lovers are suckers!
Great example of the tests, also to go with what Carl said, adding the word ‘free’ or ‘give away’ also helps increase the open rates.
Either from the two is okay! Happy Holidays and Merry Christmas!
This is a good example of how it’s too easy to get tangled up in the numbers (such that more important and more long-term considerations are overlooked).
What if Christians (actively religious and just culturally) feel more welcomed by the “Christmas” reference, but a significant chunk of atheists such as myself, and other minorities who celebrate other holidays than Christmas are actively displeased by it?
When I get “Merry Christmas” communications from small companies in particular, I tend to assume the company management is Christian, and/or actively courting Christian customers, and/or really likes Bill O’Reilly — any of which leave a bad taste in my mouth.
This isn’t data — I’m one person, and if I funnel your messages to my junk folder from then on you won’t even know (or care, probably).
But choices like this are undeniably part of how you build the *total* image of your company — that image won’t be very coherent if it’s patched together from dozens of A/B tests with no other consideration.
…particularly when an A/B test is such a limited thing — of course, it’s not a binary choice between “Happy Holidays” (which has the standard bland corporate politically correct line for many years now) vs. “Merry Christmas”.
What about something more evocative? I don’t know, references to Nat King Cole, one of those old movies always on TV during the holidays, family gatherings, crackling fires, snow, eggnog, mulled wine, etc.? Even if you reference (well-known) western cultural traditions that not everyone will have in their own homes, it’s safer ground than diving into religion.
Great test, like it.
Very good comment from Carl about Merry Christmas suggesting a gift.
Even so, Merry Christmas is quite vague as to what the email is about. Has Merry Christmas been tested against a clearer benefit lead subject line?
Or combined with a clearer subject line? Would be interesting.
And to follow up on Rob W, ideally one would know the religion of the customer and speak to them appropriately. Getting the customer to provide religion information is another matter, explaining the benefit of providing this could be challenging and probably not worth the effort compared with more valuable information that could be requested.
Hi Zack, good tip right before the holidays. MERRY CHRISTMAS, Zack
Dennis