UPDATE: Subject line AB testing is now live. You can use this article to set up other types of AB tests, such as on send time, best subject line, CTA, or a combination of all.
What Is an A/B Test Campaign?
An A/B test campaign is a campaign with 2 or more versions being tested on one dimension – typically subject line or send time. A multi-variate test is a test with more than 1 dimension, for example if you are testing 2 different subject lines with 2 different versions of your message body you have 4 total versions and a multi-variate test, to determine the best combination of subject line and message body.
Why Use AB Tests?
AB tests can help you optimize your campaign engagement, by determining best time or subject line to use. If you have a large list (over 50k) you can test a subject line on 20k and use the best one on the rest of your list, improving your engagement overall.
You can setup AB test to determine:
- Ideal send time for your brand or list – test early on, then use that knowledge forever!
- Best subject line
- Best converting CTA in your email body
- A combination or all of the above!
Besides producing better results (sales, conversions), higher engagement helps improve your future inboxing.
How to Setup an A/B or A/B/C Test
Step 1: Setup up your A/B or A/B/C segments
Determine how many variations do you want to test – that’s how many segments you need to split your list into. We suggest to define reusable segments using “starts with” condition on one of your custom fields, for example first name.
Then, create a segment that’s basically “not A” to ensure you don’t make a mistake with rule overlap. The example below shows a segment C that’s basically doesn’t match ether A or B, and segment B would be setup similar to A but with different rules.
TIP: Make sure each segment has at least 10,000 contacts in it, so your experiment results are statistically significant.
Step 2: Setup your campaign variations.
Create your bulk/promo campaign, then create copies with your variation – subject line, preview, or message body. Assign a different segment to each campaign, see example in the screenshot below.
If you are testing on send time, then your campaigns will be identical, but will just be scheduled to send at different times.
Note: The example above is for a B2C list that’s 60%+ Gmail, so those landed in Promotions tab.
TIP: You could create a test using small parts of a larger list, then use the results for the remainder of your list.
Step 3: Start Testing (almost)
Reach out via chat and request to update your account to allow you to send more than 1 campaign at a time, if you want to test on content and send your campaigns at the same time. By default, accounts have 1 queue and campaigns go out sequentially. If you are on an Agency plan, you already have ability to send concurrent campaigns.
That’s it. If you are a BigMailer customer you can reach out for assistance with this or anything else via chat 7 days a week.
Not a BigMailer customer? Why not try BigMailer for free, with no features locked? We are sure you will love it.
