This Is What Happens When You Blog Based on AB Testing

By: | March 20, 2016 | Tags: , , , |

AB Testing Guarantees #Blog TrafficDo you want the opportunity to make blogging decisions that are driven by data?

If you could make decisions based on testing, you would have concrete evidence that your actions will result in a high ROI (Return on Time Invested) for you.

A/B Testing is a way to do that testing so that you don’t waste your time.

A/B testing is an easy, free, fun, relatively fast time-saving strategy that will give you concrete proof that your decisions will result in a positive outcome for your blog.

How fast is “relatively fast”? You will have evidence within four hours of whether your actions will be effective.

I sleep during the four hours when I test. You don’t even have to be awake.

What do you need in order to do A/B Testing?

In order to do A/B Testing, you need to use an Email service that offers that feature.

The “subscribe box” in your sidebar is not enough to catch Emails. You should use an Email service. It’s an additional way to capture Emails, and the Emails are yours in the event WordPress or another blogging platform closes your blog down or goes under.

According to Julie Kalungi, having an Email list will help you increase your subscribers. “You should do whatever it takes to learn how to create killer emails.”

Your problem is that unless people open your Email, they will never read your content. What was the point of having an Email list if the people on it don’t read your articles?

Do you need a better subject line? Do you need to send your Emails at a different time of day or night? A/B Testing is a way to take the uncertainty out of blogging and get these questions answered before you send the majority of people on your list your Emails.

What is AB Testing?

According to marketer Mark Newsome, you should not invest money in advertising your blog. You should test and let the results help you make decisions. A/B Testing is a way to do that test.

AB Testing is also called A/B Split Testing since you split your Email subscribers into groups in order to test.

AWeber offers a description of A/B Testing:

You divide your email list into two segments, sending one email to segment A and after changing an element in the email you send a second email to segment B. In the context of email marketing split testing is often used to test different subject line text or message content to increase response rates.

Simply put, you come up with a question about your blog with two possible answers. You don’t know the definitive answer, so you don’t know which action to take. A/B testing allows you to test both ideas on two different segments of your Email list.

You don’t even have to take your time to divide your list. Your subscribers are randomly divided by your Email service.

This post will tell you how to do A/B Testing using the MailChimp Email Service. However, AWeber allows you to A/B test as well.

Why is A/B Testing Valuable?

According to AWeber, by testing your theories on your subscribers on your Email list, you can get to know them better.

Once you have the data, your time will be saved going forward since you won’t have to weigh the pros and cons of the various decisions you make. You will know the answer after just one test.

It is not just important to keep your subscribers updated. You need to analyze the data that comes from doing so. Blogger Noah Kagan explains it is important to test assumptions.

According to blogger Harleena Sing, you should send your Emails to your list consistently and see which Emails perform the best. The analysis you get from A/B Testing will give you this information.

According to Kalungi, you should always test and check to see which style of Email got more opens and clicks on your links.

How to Conduct A/B Testing

Group A is a Control Group and Group B is a Treatment Group. Treat them differently and see which group opened and clicked your links more than the other.

MailChimp offers four ways to do A/B Testing. There are four types of theories you can test.

    • Different Subject line
    • From Name
    • Content
    • Emailing your respondents at different times of the day

This article addresses how to test different subject lines, but MailChimp lets you test your possible theories about all four.

How to do A/B Testing of subject lines:

The subscribers in Group A get Emailed one subject line while Group B gets a different subject line. I tested on half my list although you could do the test with as little as ten percent of your subscriber list. The members of the groups are randomly chosen.

A/B Testing Example #1

I needed to send my post 2 Things About SEO You Need to Know to Get Traffic to my MailChimp Email list. WordPress does not send my posts to my subscribers on that list.

In order to test my subject line, I had to come up with two possible subject lines. I did but didn’t know which to choose.

Jon Morrow advises starting subject lines with verbs like Discover and Learn. On the other hand, Neil Patel raves about the value of using a question.

Which to pick? I didn’t know, so I used both on different groups in my Email list:

Email Subject 1: Discover how the two kinds of SEO can help you get blog traffic

Email Subject 2: Do you know how the two kinds of SEO can help you get blog traffic?

Do you want to have some fun? Before I share the results, see if you could predict which subject line was more effective in getting my subscribers to open my Emails. Which would you have opened?

A/B Testing offers bloggers data

Did you guess right? As can be seen from the screenshot, Email Subject 1: Discover how the two kinds of SEO can help you get blog traffic produced better results.  More of my subscribers clicked the link to my article that had the statement, not the question.

Note: This test occurred during the four-hour period following my scheduling the Email. These results reflect that time period.

A/B Testing Example #2

Okay, so I tried it again. A good researcher repeats the test to make sure the results weren’t flukes.

This time, I had a more challenging decision regarding my subject line since I was mailing two articles to my Email subscribers, 17 Reasons Buffer Will Blow Your Mind and 5 Gifts For the Writer in Your Life.

I was torn between Email Subject Line #1: Discover 17 reasons you should use Buffer to blog and 5 perfect gifts for writers and Email Subject Line #2: How to save time blogging with Buffer and what the perfect gifts are for writers.

My rationale: Merits of Email Subject Line #1: It contains the word Discover, and 17 and 5 are odd numbers and prime numbers. They both supposedly do well in headlines.

Merits of Email Subject Line #2: The words how to are effective in headlines. Maybe they’d be effective in my subject line.

Once again, have some fun with me. Make a prediction. Which subject line do you think won? Which was more effective in motivating my readers to open my Email?

A/B Testing helps #bloggers

Did you guess correctly? I didn’t. The subject line with “discover” won the first A/B test but lost the second A/B test. What did I learn from this? I need to keep testing! I can not make assumptions about my subscribers.

I don’t have to be right. MailChimp will send the Email with the winning subject line to the rest of my subscribers after the four-hour period of testing is over.


In conclusion, thanks to A/B Testing, there is no longer any need for speculation when it comes to the behavior of your subscribers.

I find A/B Testing fun. I preschedule my Emails before I go to sleep since I like people to receive them at 2:00 am California time when I’m sleeping. When I wake up, I am in suspense wondering which subject line got the most clicks. It’s like waiting for the results of a race.

I hope reading about my theories that I’ve tested will inspire you to come up with your own theories and ways to test them.

Please share these case studies and the results in order that you might inspire others to test variables that might lead to blogging success.

Readers, were you able to predict the outcomes? What do you think of A/B Testing? Are you doing it now? What is your opinion of it? If not, do you think you might try it? Do you have any questions about A/B Testing? I look forward to your views.

Related Posts:

How to Easily Increase Page Views with an Email List

How to Easily Install a Click to Tweet

How to Easily Get Your Blog Found on Google (The post will explain how to find words with high search volume to put in your Email subject lines.)

  1. John Doe

    If it saves time in the long run it is worth the initial time in the short run. And naturally the results are the most important thing.

    • Janice Wald

      I agree. That’s the point of A/B Testing. Don’t waste your time doing what doesn’t work. Thanks for writing. I appreciate your comments.

  2. Carol Graham

    You are always so CLEAR in your instructions and it is appreciated. You must be a teacher 🙂

    • Janice Wald

      Hi Carol,
      I am a teacher! I was afraid it was too hard! I went back in and asked if there are any questions. You read my mind! Thanks for coming by and giving me that assurance.

    • Janice Wald

      HI Darrin,
      I visited your site too. You will see my avatar, but my comment isn’t there. I complimented you on your noble cause and told you about the three dogs I had until recently.
      Thank you for coming by my site. I am glad you appreciated my MailChimp A/B Testing article.
      You indicated you are a relative “novice”. I’d love to encourage you to subscribe. I help new bloggers here. I am having a big blog party next Saturday. You’d get notified and could come get new subscribers.

  3. 1010ParkPlace

    What great information! I will try this! Thanks, Janice! xoxox, Brenda

    • Janice Wald

      Hi Brenda,
      Thanks for writing to tell me you are going to try it. Have fun! It is definitely Janice fun. I was a sociology major and love testing variables against a control group.

  4. Soumya Roy

    You are absolutely right, A/B testing is really important in all aspects of online marketing. Until and unless we do AB testing and data analysis, it is not possible to measure the better one. It’s always recommended, especially in email and paid marketing campaigns. Liked reading the post and thanks for sharing it.

    • Janice Wald

      Hi Soumya,
      I tried to visit your blog too, but it said comments were restricted to team members. This is what I wrote:
      Hi Soumya,
      You visited my blog today. I am glad you enjoyed my A/B Testing article.
      In response to your post, I may link to it next weekend, I like your article so much. I especially agree with #2 and #3. I write about the importance of networking. I never realized it could help people in their workplace. I would love to encourage you to subscribe to my blog. I have articles like the A/B Testing article all the time. I was a sociology major. I love to test theories and bring the results to my readers.

  5. Melinda

    Oh no! Since I had to quit teaching, I’m now allergic to testing!
    Glad you can do it in your sleep! That’s very effective time management!!
    Your BBFFM

  6. Keesha

    Great article Janice! I have tried A/B testing myself but not with email marketing. I have used it to test how well different pop ups work on my blog. The results have been surprising to say the least and I didn’t realize how small changes can make a huge difference to how readers respond. Thanks for reminding me about the importance and value of A/B testing :)!

    • Janice Wald

      Hi Keesha,
      Your validation means a lot. Thanks for writing to tell me you liked my article and found my tips valuable.

  7. Debbie

    I love this! And I have Mail Chimp so I can just follow your instructions. By the way, I think you always have great headlines.

    • Janice Wald

      Hi Debbie,
      Thanks for writing.
      1. Your comments made me glow when I read you loved the article.
      2. Then, I read you like my headlines! Frosting on the cake! Thank you!

  8. Kimsea Sok

    Hey, Janice! This is another awesome post of the day.

    Well. As far as I knew over 90% of bloggers and internet marketers are currently using email marketing. Over 50% of those marketers are paying high attention on implement email marketing in the right way. They tried to develop personas, traffic report, and analyst previous performance to tweak out ain’t working. I agree with what you mentioned A/B testing is great way to develop a right email marketing strategies. Thanks for sharing…

    • Janice Wald

      Hi Kimsea,
      Thank you so much for the share on Facebook. I shared your comprehensive article about Email marketing on Twitter. The “loop” term was new to me, so thank you.
      I am glad you agreed with my strategy that I advocated in my A/B Testing post. I always use A/B Testing. It’s enlightening, and I find it fun trying to predict the winner. I am happy for our connection. I followed you on Twitter.

  9. Bella D. @ Self-Publishing Made Easy Now

    Thanks for sharing this article, really helpful for bloggers. It made me realize the value of AB testing. I hope I can give it a try soon.

Would you like to share your thoughts?

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.