A/B Testing for Beginners

What is A/B testing in email marketing? 

Say you’re commuting from Place A - Place B. You have multiple routing options, where you try taking different routes, finally, you choose the one that saves you time and energy. 

A/B testing is something similar. It is a statistical method used by marketers where you split two versions of the same email. For example different subject lines, the preview text, “from” name. etc)

This way you understand what campaigns perform better and helps to double down your efforts there.

A/B testing gives insights into your customer behavior and helps you improve your overall email marketing strategy.

Why is A/B testing important for email marketing

As an eCommerce brand, you have your own set of challenges. For example, your campaigns might have poor open rates. The first thing you should A/B test is your email subject lines. It’s the first thing that they notice. Here’s an example from Nykaa

Similarly, if you see decent open rates and your emails are not resulting in conversions, you need to optimize

• The email narrative 
• Check if your CTAs are enticing enough for your customer to take an action
• Discount percentages 

It’s all about getting to know where the leakage is and trying to optimize the components of your email.

Here are some steps that you can take into consideration while A/B testing:

Step 1: Set a goal

Decide what results you want to achieve with your next campaign. It can be : 

• Increasing your open rates leading to conversions
• Improving your click-through rates
• Conversions: Making people buy
• Understand if coupons/bundle works
• Emails with count-down timers vs without
• Increase your average order value of the customer
• Improve my abandoned cart recovery

Step 2: Identify levers and choose your recipients.

As discussed before, Subject lines, discounts, BOGO offers, CTAs, social proof, etc are your levers to accomplish your goals. 

Let’s take a different example this time. Say your goal is to make one-time buys to make their second purchase. The levers you can use would be ‘discounts’ or ‘free shipping’ or even BOGO offers.

Create a segment for shoppers who transacted only once. Then play around with your levers by splitting the segments into three. 

Step 3: Run and Review the metrics

Send out different versions of your email to the segments of recipients and track the performance based on your defined goal. 

Run your A/B test campaign. Wait for at least 24-48 hours to get sufficient data.  Compare your open rates, click rates and revenue between the 3 versions and see which one performed better. 

Pro Tip:

A/B testing is not just limited to one-off campaigns you send. You can also A/B test email automation like welcome email sequence, replenishment flows and even your abandoned carts. For example, you can have a separate abandoned cart sequence for first-time shoppers and vice versa. 

The first step is about identifying the problems in your email marketing– check where you can improve. Use the levers to your advantage to get a better understanding of your customers. 

Happy A/B testing!