If you’re like the other gazillion websites trying to get people to sign up for an email newsletter, then you probably have an opt-in email popup form. You know, it’s that annoying lay-over popup box that asks new visitors to opt-in with their name and emails.
Wouldn’t it be nice if there were landing page optimization best practices for designing these? Then your opt-in popups could grab many more subscribers right out of the gate. At the very least, it would be good to know which elements are worth focusing on in testing.
Since we thought it would help a LOT of people to get some straight answers on popup opt-in forms, we got Jeremy “Shoemoney” Schoemaker on the phone (Conversion Voodoo is very lucky to work with the best of the best in online marketing). We had an idea for him:
“Whaddya think about us running some tests on the opt-in popup for your newsletter?” Jeremy’s reply was about as easy as they come, “Have at it!” So we did.
Not only that, but Jeremy requested that we post the results for his audience. All we could say was “absolutely!” We rarely get to publish results because 99% of our clients want to remain anonymous — so this was sweet!
Our goal was to test the “default” settings on the popup to see if there were the better ways to convert visitors into subscribers. We hoped to find some best practices to share with you for your website that may not have a horde of visitors every month like the Shoemoney blog. Although we counted more than 30 different elements in opt-in popups, we whittled that number down to the most relevant (i.e. the ones you should care about), which you’ll see in the tests below.
Frankly, even we were surprised by some of the test results. Wait till you see how your opt-in popup form should be designed to get the most conversions — and you better believe these aren’t the default settings on many of the off-the-shelf packages out there. Good luck and happy testing!
Headline:
If you’ve been keeping up with our landing page tear-down inspiration posts you know that one of the first things we test is the headline copy. For ShoeMoney’s popup we tested 6 varying headlines and found that there were two clear winners over the baseline.
Hover over the headlines below to see our findings:
The two headline winners both say exactly what the user is going to “get” when he/she signs up. These headlines also cater to the user’s want to make money online. You’l also notice they use specificity: “7 simple tricks”, and also urgency: “Limited offer”. Find a way to entice your website visitors with something of value worthy of their time.
Inline Field Labels:
A problem that we found with ShoeMoney’s newsletter popup was that once the user clicked into the field to type their email address, the field label disappeared! Simply by dimming the label within the field we were able to get a good sized bump in conversions. Take a look at your own forms and make sure that the labels are always visible.
Hover over the image below to see our changes and findings:
Bottom Copy:
This was another simple test. We compared the baseline bottom copy, which asked users to follow the RSS feed, to having no bottom copy at all. We believed that all friction that can keep a user from the targeted action should be removed. Guess what? It worked… big time.
As a note: we decided to leave the bottom copy visible for the final popup. Though if you want to remove something similar from your own site, we whole-heartily recommend it! And don’t be afraid of losing RSS feed subscribers because once you have then on your email list and are building a solid relationship with your users, you can always ask them to join your feed later!
Hover over the image below to see our changes and findings:
Auto-Center:
While running tests we realized that the newsletter popup on ShoeMoney’s site was in a fixed position. If the user scrolled down the page, the popup stayed at the top and was no longer visible to the user. We tested this baseline against a popup that scrolled the page with the user.
Interestingly enough, the popup that stayed at the top of the page was the winner! We believe this is due to the user still be able to see, but not reach, content behind the dimmed background of the page caused by the newsletter popup. If the user sees something they want to reach, they would be more inclined to enter their information into the popup and opt in.
As a note: on another client’s newsletter popup there was an adverse affect, so this is something you’d want to test on your own site.
Close Button:
When doing market research we find interesting items that competitors to our clients are using on their own site and we tend to test them into our own tests to see how well they do.
One particular item was the close button on the popup. ShoeMoney’s baseline popup had an “X” button at the top right corner which we tested against a “no thanks” link below the signup button at the bottom of the popup.
Turns out that ShoeMoney had it right from the beginning! The “X” button was a clear winner over the “no thanks” link.
Hover over the image below to see our changes and findings:
Clicking Dimmed Area (outside):
A notable problem we found with the baseline ShoeMoney popup was that if a user clicked on the dimmer area (outside the popup) then the popup would be closed! By tweaking the code so that the popup stayed even when clicked outside we were able to increase opt-ins by 19%.
Re-Popup if Ignored:
The final, and most code-intensive, test we performed was that if the popup was ignored by a user and that same user came to the ShoeMoney page again at a later time, the popup would reappear until the close button was clicked. The baseline has the popup disappear forever if ignored.
Even with this seemingly simple change, we increased opt-ins by a whopping 31%!
Conclusion
Never stop testing! When we went through this newsletter popup we had the intentions of testing every little detail that most people wouldn’t think would matter. Things such as different close buttons or fixed vs. scroll popups. As you have seen some of the smallest changes will lead to big conversion increases.
We hope this post has inspired all of you to jump onto your own websites right now and start making a list of items to test because there are big gains to be had, we guarantee it!
Note: All tests had confidence level of at least 95%, (so these are highly relevant data points) and well over 500,000 unique visitors during testing period (Shoemoney is a pretty big blog). Remember, always test new ideas into your site as your mileage my vary. 😉 You also should take into consideration if you should even use a popup form. We don’t advocate using popups, but we also understand how critical they can be to some sites to gain a lasting connection to visitors. You get to decide… Viva America! By Jon Correll.
wow this post blew my mind!
My favorite was the Inline Field Labels. Totally makes sense once you think about. Let the customer know exactly what to do at all times.
Can you guys be more specific how the inline label works? What happens if person starts typing – does the label hides or it stays there?
General practice: Have inline label 100% of chosen color, when not in focus. Then if focus if give to it, it should go grey (Shoemoney’s is given focus on activation). Then as soon as typing stops, it disappears. Does that help?
I use RoboForm to fill out forms found online. If Inline field labels make RoboForm think that the Name and Email fields are not ‘blank’, then it will not auto fill these fields. This really peeves me and I will think twice before I manually provide my contact data. IF you don’t respect my right to use the tools I choose, then that lack of respect will swing both ways.
What tool was used for the PopUp functionality?
Can you let us know what popup plugin is being used?
We used a customized version of: jQuery Pop Up Plugin for WP. But there are a lot of popup versions out there that would accomplish the same task. Thanks!!
We use Bounce Exchange to grab emails with the pop, you can capture on page exits to (we get about 2%)
Could you also give us the number of visitors you had run for the tests to see if there is a conclusive positive effect?
95% confidence isnt that useful if the number of visitors who were used for this test is too low.
Good question! We edited the post to add that info (over 500,000 unique visitors).
Thanks!
Very useful findings, thanks to you and ShoeMoney for posting them. I was also wondering whether the size or the design of the popup have any influence at all.
Your welcome. Thanks for reading. Happy Testing!!
I have seen (and been slightly annoyed by) the popups on the site, and would like to add a suggestion – could you maybe track where people are coming from? For example, as an existing subscriber to Shoemoney’s newsletter, it was most frustrating to follow a link FROM the newsletter to a page asking me if I wanted to sign up to the newsletter. You may have increased new subscribers but surely it’s worth making the effort not to alienate existing ones?
That’s a great point. It can be annoying, and I think adding a simple “no popup” link parameter would be a great idea. I don’t think jQuery popup plugin that was used has that feature. We’ll look into an add-on customization and post a “how to” (or find out it already has it). Thanks!!
That bugs me on a lot of sites. Hubspot in particular asks for signup and demographic info for every download, even clicking from their emails multiple times on the same day. Sure the info may change, but not hourly.
Yup. Like I said, it is annoying. A some value that data quite a bit, and therefore are willing to annoy a bit more than other sites.
Great share, is there a report that showcases the full results? Also if popups work so well, why aren’t you using them on this blog? I am genuinely curious.
We have not created a report with the full results. Actually, we had a popup a couple of years ago, but over time decided that we preferred the “contact us” model for prospects. (since we focus on a very VERY narrow market of high traffic websites.) Thanks for coming!
Valuable knowledge to share – thanks for sharing Jon.
This plugin you are using – works – but its not perfect. Take a look on your iPhone.
The timing is interesting, my first task this morning was to test the fancybox jquery plugin on safari mobile – and it doesn’t work right on the iPhone. I was thinking of testing signup with a popup box rather than redirecting to a new page.
You’re right Merrick. That is a bug in the popup code. Actually, there is a second bug as well, mobile phones were supposed to be ignored (the was a ShoeMoney customization). We notified the site admin with the fixes. Hopefully that should be updated soon.
Thanks!!
That is some pretty good testing and good results. With half a mil of visitors this is accurate data for the webmaster / make money online niche!
Going to put the changes on my own sites too. Thanks a ton!
Wow outstanding results!
Did you notice any change in site exits when you changed the pop-up to not disappear when the user clicked the greyed-out area?
Not any statistically relevant data on this, unfortunately. Thanks!
A better question is how many of the news letters end up being ignored in someone’s inbox because they really didn’t care about your content that much anyway?
nice stats bro. I’m also a freak for stats. I split test everything.
Where can I get that popup plugin at?
Thanks,
Joey
We put the link in one of the comments below. Rock and roll and conversions! 🙂
Jon – have you tested the amount of time elapsed before the popup appears? Like… instant pop vs. 3 second delay vs 10 second delay, etc? In any test, not just this one. That would be interesting.
Yes. We did try that and other tests. Short answer, the longer the delay, the lower the conversion. We interpreted that it was increasing the annoyance factor. It’s already annoying, but when someone is starting to read an article, then the popup interrupts… Arghhhh!
Anyway, that was our interpretation.
Thanks!
Having pop-ups on your site is a really efficient way to irritate your users. And Re-popup if ignored, seriously? Great way to alienate your users.
The only pop-up optimization is to remove them completely and have something less obstructive instead (like a bar at the top).
That is definitely the deeper and mots important question: “to be or not be to be a popup site? That IS the question.”
We decided “not to be” for our site, as we felt it might distract from our message. But, if you are going to potentially annoy visitors using a popup optin strategy, at least do a better job at getting a connection from those who are interested in connecting, right?
This is a interesting comment and one people make.
While people do complain, looking at the statistics it has no impact at all. In fact if anything it brings more visitors because of the newsletter content sequence.
Yes, I’d agree with that based on my own experience. I was recently reading a great article about that topic, and other related to pop ups, at ProBlogger, here https://www.problogger.net/archives/2008/10/23/how-to-drastically-increase-subscriber-numbers-to-your-email-newsletter Not that my visitor numbers are anywhere close to yours, Jeremy. 🙂 Thanks for this wonderful post, too. Cheers, Thea
Great article. Hopefully this one adds a little to it for you, and helps you on your site. Thanks for coming!
Is there a way to trigger the popup after the visitor has been online for say 30 seconds or longer with the approach of “hey you seem to like our content” or something to that effect, I think if there was some additional filtering based on ip addresses, registered users etc, this would be much more widely embraced from a user perspective.
We actually tested with a delay, that we didn’t include in the post, and a delayed popup lowered conversions. If I were to take a guess, I assume it multiplies the annoyance factor but interrupting while they are actually engaged in your content. I think only using the popup with new visitors would help minimize annoyance, but there isn’t a 100% sureh way of doing this, except for turning it off. Thanks!
Thanks for sharing the case study, I love detailed results like these! One thing that I’ve always wondered and plan on testing with my sites is having an opt-in box popup when the user leaves the viewable browser window. Less of an annoyance, and I feel the opt-in box should be served only after someone finishes reading content.
Great idea. Thanks for the comment!
It would be interesting to compare regular labels to in-field labels.
Have any tests shown in-field labels outperforming regular labels?
We have plenty of other sites we’ve tested this one, and have gotten mixed results. But it is a worthy thing to test. We tested other elements not listed in this post, but I don’t think that was one of them. Will check. Thanks!
Amaizing stuff. I got one question. What was the % of non-existing emails e.g. oijdwdwe @ gmail.com etc.?
Because its javascript bots were not posting to it. We do get that a lot on our HTML forms though but not the popup.
I use a pop up after a visitor has been on my page for 45 seconds that says “Hey Reader Do “Like” or “Hate” this website?
The “Like” is a FB “Like” button connected to my fan page and gets me tons of new fans per day. The “Hate” button closes the site.
Great idea Justin. Thanks for the comment!
This information is super valuable, too bad its so rare to see it published. I can only imagine the knowledge you guys have on best practices from doing so much conversion optimization for clients.
One of the most annoying things about pop-ups is that they don’t display properly and allow closure when viewing the page from a mobile phone. Often the popup is too large for the phone screen and the x is missing or not viewable. Sites that plan to use pop-ups need to detect the screen characteristics before serving the popup to ensure the popup is viewable and closeable from all types of devices.
Agreed. I think this goes to the core of any interaction with a site visitor: MAKE SURE IT WORKS FOR THEM! 🙂
Thanks for the comment.
Very valuable post!! Thank you very much. I always struggle with whether to do the popup or not. I didn’t want to be “one of those sites”. Do you find taht the pop up doesn’t hurt your site? How would you measure that?
I’ve always wanted to try this but I am afraid it would bump up my bounce rates, because in my head the idea is practically the same as interstitial ads which most often times would drive the visitor away. Sites like Shoemoney already has a big core audience and I think the results would be drastically different on sites that are not so big like that of Jeremy’s.
I’m actually a Shoemoney newsletter subscriber but I don’t even remember when I signed up!
From a reader perspective, usually I unsubscribe when there are too many adverts but Shoemoney actually doesn’t email that often. His blog digest once a week is actually quite helpful as I don’t visit his site every day so I can just scan his digest newsletter and pick out any blog posts of interest.
I have added a popup plugin and running some tests. Hopefully I can also find ways to get more subscribers. Your post was really useful in that direction. I can’t wait to have them subscribe! 🙂
I am using the generate theme and i immediately have an offer on the top fold of my front page. Then in my blog, i have a quick pop-up video (an autoplay) with me reminding you to opt-in because my free report is available for a very limited time….Since I started doing that, I have tripled my opt-in rate.
I was nervous about that at first but this is an internet marketing website and I changed made a couple tweaks, thanks to your suggestions and my opt-in rate has gone up even more.
Thanks!
Well! It is really nice of you to post the results of such a great test.
However I am curious about one thing… What impact did the popup had on the bounce rate of the blog? I had been using a popup too but I found out that it was increasing the bounce rate on my blog by around 40%.
I had no impact at all on my bounce rate – I guess it depends on the message in the pop-up. Look at wise-pops . com – great tool to create pop-ups with smart features.
I would like to have a popup opt-in newsletter signup for my website when page loads with Facebook twitter like button. It is not a wordpress website. I do have account with constant contact. I just need the popup design. I am not sure how to do it. Is there any Lightbox, jQuery which I can add to my Home page HTML? Is there any website to just design the popup and get the code? I can incorporate that plugin in my home page on load. Excuse me for such a sloppy question, I am new to this. Thank you!