A/B Testing – Meta Baeldung http://meta.baeldung.com Marketing experiences growing Baeldung Fri, 15 Mar 2019 16:08:29 +0000 en-US hourly 1 https://wordpress.org/?v=5.1.1 How a Contextual CTA will lead to a higher Conversion Rate http://meta.baeldung.com/how-a-contextual-cta-will-lead-to-a-higher-conversion-rate http://meta.baeldung.com/how-a-contextual-cta-will-lead-to-a-higher-conversion-rate#respond Sat, 15 Mar 2014 20:27:01 +0000 http://meta.baeldung.com/?p=172 1. Single CTA for All Pages

Like so many blogs out there, my blog – baeldung – had a single sidebar optin form for a long time. It is a form that I experimented with quite a lot, but alas a single form can only do so much.

Makes sense if you think about it – there’s no such thing as a one size fits all Call to Action – yet, for the last 6 months, mine basically said:

Free eBook REST Services with Spring

My Single Call To Action Optin Form

My Single Call To Action Optin Form

I can do better!

Now – my blog has a simple structure with a handful of main categories – and it would make perfect sense to have a different CTA for each of these. What I’d like to see is – for example – for my Persistence category, a CTA such as:

Free eBook Persistence with Spring

True – that means having a new eBook – nobody said it was going to be easy – but it would make a lot more sense that the generic CTA I have now.

And so – with this goal in mind (and with the help of an editor) – I launched my second eBook, out of an already existing blog series I published on the blog. This eBook became the lead magnet for my second, now contextual Call To Action.

2. Contextual CTA depending on the page

OK, so let’s look at this – my new Optin Form uses the second eBook similarly:

Free eBook Persistence with Spring

My Second Call To Action Optin Form

My Second Call To Action Optin Form

As you can clearly see – the Optins are pretty much the same, except they use different lead magnets.

This second Optin is shown on the pages of my blog that have something to do with Persistence – shocking, I know. I’m showing the widget with the Persistence Optin on pages that have the Category Persistence and the other widget on everything else.

Technically, I’m using the Widget Logic wordpress plugin, but any plugin that will allow you to conditionally show widgets based on the category of the article will do just fine.

So there you have it – a simple execution of this concept, but one that a heap of sense.

3. The Results

It’s the numbers – specifically the conversion rate of each of these Optin Forms – that will make or break the experiment. Luckily the numbers are good:

  • the Original Optin (Rest Services with Spring eBook) => 0.75% CTA
  • the New Optin (Persistence with Spring eBook) => 0.93% CTA

That is a 24% increase in my conversion rate – which is a pretty significant jump considering that the Control – the original Optin – was pretty strong to begin with.

4. Next Steps

My next step is to do the same with the other major categories of my blog – work through 4-5 new eBooks and customize each Optin to be relevant to the article the reader is actually reading. If I see a 25% bump for each of these category – and I’m sure I will based on these initial results – I’ll be very happy with the results.

If you read this far, you should follow me on G+:

http://meta.baeldung.com/how-a-contextual-cta-will-lead-to-a-higher-conversion-rate/feed 0
Is It Worth having an Email Optin Form at the End of the Page? http://meta.baeldung.com/email-optin-form-at-the-end-of-the-page http://meta.baeldung.com/email-optin-form-at-the-end-of-the-page#comments Fri, 15 Nov 2013 13:48:41 +0000 http://meta.baeldung.com/?p=85 The Premise

Until very recently, I’ve been using a single Sidebar Email Signup Form on my blog – front and center.

This has been performing relatively well – over the last couple of months, I was able to improve its conversion rate from 0.37% to 1.14% through a series of changes (all documented on this blog).

I recently started using an “End Of Article” second Signup Form on the blog – basically appearing after all of my articles. I was motivated by the well understood idea that the reader usually needs reminding in order to take action.

Before going into details about the experiment, here is the Optin Form I’m currently using:

Signup Form at the Bottom of the Page

Signup Form at the Bottom of the Page

Because the form is positioned at the very end of the page, by the time they get to it, the reader would have already presumably read the whole article. Also, given most of my articles are over 1000 words, the fact that they actually reached the end indicates that they found it useful enough to stick around – or else they would have stopped reading by that point.

So – the premise is that visitors that see this second Signup Form is much more qualified and interested in the content than the visitor who has just landed from Google and happens to see the main form simply because it’s above the fold.

Let’s now take a look at the actual conversation rate I’m getting on this form, and see if we can validate the premise for the experiment.

The Results

The data for this new Optin option is as follows:

Visits: 6044

Conversions: 141

Conversion Rate: 2.33%

How can these results be interpreted?

First – the conversion rate for this new Optin is much better then the Main Sidebar Form – more than 2x better. The premise of visitors being more qualified at the end of the page is validated and now has hard numbers behind it.

Then – given the fact that only half of my readers reach the end of the article, but the conversion is double – it basically rounds up very nicely to another 100% increase in signups.

It will be an interesting November!

http://meta.baeldung.com/email-optin-form-at-the-end-of-the-page/feed 6
Improve Email Opt-In Conversion by 229% – Case Study. http://meta.baeldung.com/improve-optin-conversions-with-new-book-cover http://meta.baeldung.com/improve-optin-conversions-with-new-book-cover#respond Tue, 12 Nov 2013 15:02:00 +0000 http://meta.baeldung.com/?p=80 How the Test Worked

I’ve been working to improve the email conversions on my blog lately. Up until now, every test I ran resulted in a triple digit increase in my conversions – but I didn’t expect this trend to continue. I’m happy to say – I was wrong.

After my previous test – when I saw a huge 750% increase by having my signup optin scroll with the visitor – I am now testing a completely new design of the form against the existing one.

To set up the test, I used the Split Testing feature of OptinSkin to server version A to some users and version B to others in a simultaneous A/B test. Once I reached statistical significance (over the course of several days) – I turned off the under performing version.

The initial version of the Optin Form looked like this:

Initial Email Optin Form

Initial Email Optin Form

The new version being tested looks like this:

New Signup Form

New Signup Form

I kept the copy exactly the same so that I can later on pinpoint the results to the design change and nothing else.

Obviously the one significant element that stands out from the new design is the new book representation – this didn’t exist in the old design – so it’s interesting to see how much of an impact that had on the overall conversions.

Which Test Won?

If you guessed – version 2 – then you guessed right.

Here is the data:

design 1 – 38271 impressions – 142 conversions = 0.37%
design 2 – 11309 impressions – 138 conversion = 1.22%

So the new version not only outperformed but absolutely dominated version 1 of the design – by 229%.

If there’s one conclusion that you can pick up from this experiment is that it’s a spectacular idea to include a visual artifact – such as a book – in the design of your Optin Forms.

I’ll continue to test various elements of my site and bring you the results – sign up for email updates from me (see what I did there?).

http://meta.baeldung.com/improve-optin-conversions-with-new-book-cover/feed 0
How a Simple Visual Tweak improved my Singups by over 750% – Yet Again http://meta.baeldung.com/visual-trick-increased-my-email-signups http://meta.baeldung.com/visual-trick-increased-my-email-signups#comments Fri, 25 Oct 2013 14:11:34 +0000 http://meta.baeldung.com/?p=64 1. A New Experiment

I said in my last article that I’m no longer expecting an experiment to yield drastic improvements in my Email Signups. I was wrong.

My first experiment – offering a free eBook – resulted in a 625% increase, and my second – a completely new Signup Form – a more modest 125%.

So, my third experiment was simple – I fixed the position of my Email Signup Form (via plugin) when the reader is scrolling down. The idea is straightforward enough – with the form staying on the screen more, the reader will have more of a chance to actually read what it says and choose to sign up. You can of course check out the blog to see it in action.

What I didn’t anticipate was just how much this small change was going to affect my conversion rate – nothing short of a 763% spike in signups.

2. Unexpected Results

The “Before” of this experiment is of course the “After” of my previous article:

Email Signups before fixing the form position

Email Signups before fixing the form position

Now, here is the data after I have made the change:

Email Signups after Fixing Optin

Email Signups after Fixing Optin

And finally, just for the visual impact this always has on my – here’s my Mailchimp:

Mailchimp - October

Mailchimp – October

3. Learning

The key takeaway from this experiment is that a single, uncluttered Call To Action works wonders, especially when the reader will keep eyeballs on it longer.

The “single CTA” point is – for me – also very relevant. Notice that my optin form does not contain any additional, secondary options – such as social buttons or badges. These would give the reader to many alternatives and less valuable ones.

Once a reader does go for one of these alternatives – for example they follow me on Google+ – it’s highly unlikely that they would also go back and signup for Email.

There are of course more “in your face” ways to collect emails – using a popup for example (on my TODO list) – but what’s interesting about these results is that this change does not qualify as “in your face” – just the opposite in fact.

Live and learn.

http://meta.baeldung.com/visual-trick-increased-my-email-signups/feed 5
How Less Options On My Optin Improved the Conversion Rate http://meta.baeldung.com/new-optin-form-increases-email-signups http://meta.baeldung.com/new-optin-form-increases-email-signups#respond Mon, 21 Oct 2013 12:26:54 +0000 http://meta.baeldung.com/?p=57 1. The New Experiment

In my last experiment, I looked at the results of offering a free eBook and how that affected my conversion rate for email signups. In that experiment, I kept the exact same Optin Form and only changed the copy – adding the eBook as an incentive.

This time, I did things a little bit different – I changed the old Optin with a new one, where I have more or less the same copy but less options (no more social sharing).

The result – a nice 115% increase in signups over the course of a 2 week period. That is less than the 625% bump I got after the last experiment, but then again I was getting almost no signups before that, so I wasn’t expecting anything of that magnitude.

2. The Before and After

Here is how my Signup Form looked before the change:

Email Form with free eBook

Email Form with free eBook

The new Optin form is simpler – the social sharing options are gone – and there is now only one thing you can do – Subscribe:

New Email Optin Form

New Email Optin Form

Notice that, besides some minor modifications, the main Call To Action copy is similar on both Optins. This is mainly so that I can clearly pinpoint any change in my signups to the new design and eliminate other possible influences as much as possible.

Next, notice that the new form is actually collecting more data than the old – I am now asking for the Name of my subscribers as well. It is a well known fact that the more information you ask for, the lower the conversion rate, because less people will actually bother to fill in everything.

But ultimately – the most important change is that the new form presents the reader with less options – the old form contains social buttons for Twitter and Google Plus, as well as a RSS feed link. The new form removes all of these so that the Call to Action is clear and there is no confusion about what I want the reader to do.

3. The Results

First, let’s look at the data from the previous experiment: I had 29 signups over the course of 28 days.

Now, let’s see how the new form changed things:

Email Signups after New Form

Email Signups after New Form

Over the course of 13 days, there were 28 new signups to my email list.

This represents a 115% increase in signups as a result of the current experiment.

4. Conclusion

This is another good step towards the end goal – which is a high performing email signup process.

My next experiment is going to be a proper A/B test – pitting several Email Optin forms against each other to see which one performs better.

Stay tuned (by signing up the my email list – go figure).

http://meta.baeldung.com/new-optin-form-increases-email-signups/feed 0
How Giving an eBook Away Skyrocketed my Mailing List Signups http://meta.baeldung.com/free-ebook-increases-email-signups http://meta.baeldung.com/free-ebook-increases-email-signups#comments Mon, 14 Oct 2013 09:46:02 +0000 http://meta.baeldung.com/?p=39 1. The Experiment

I have been thinking of writing a small eBook for quite some time – and my new focus on growing my email list was a great excuse to do just that.

Put two and two together and I wrote the book on Building REST Services with Spring, and am now giving it away to readers that trust me enough to signup to my email list.

Since I don’t like hiding the actual numbers and I like scrolling even less – here’s the end result: my signups jumped a cool 625% after giving the eBook away.

So let’s jump right into it.

2. The Before and After

Here is how my signup form looked like before the eBook:

Email Form with no free eBook

Email Form with no eBook

The copy was generic and entirely untested:

Subscribe to receive email updates or follow us on stuff:

And here’s the new form after adding the eBook:

Email Form with free eBook

Email Form with free eBook


Notice that I am using the exact same widget – the only change is the new copy:

Free eBook on Building REST Services with Spring + regular content about building stuff ( ~1 email a week. )

3. The Results

On to the numbers – first, let’s look at the last 30 days before adding the eBook:

Email Signups Goals BEFORE eBook

Email Signups Goals BEFORE eBook

This comes out to a grand total of 4 signups to the mailing list – impressive, I know.

After the change, the new signup form with the free eBook was live 28 days. Here are the results:

Email Signups Goals AFTER eBook

Email Signups Goals AFTER eBook

Over the 28 days the new form, there were 29 signups to my email list – just over one signup per day. That represents the 625% increase in my email signups I mentioned early on – which is nothing short of cool.

4. The Conclusion

This experiment was a good step towards converting more of the casual visitors of my blog into into regular and engaged readers, getting actual value out of the content I put out.

And finally, it’s good to finally have hard numbers for this – giving an eBook away does wonders for your email signups.

http://meta.baeldung.com/free-ebook-increases-email-signups/feed 5