How to create an A/B Test

An A/B test allows you to create multiple versions of a Campaign. This means you would be able to have the same Campaign but with different subject lines, calls to action, etc, and then automatically send those different versions to a subset of your total recipients.

The version that results in better metrics based on the criteria you define will then be sent to the remainder of your recipients after a designated period of time. Or if you would rather manually select the winner, you can also stop the test and have it sent to the remaining recipients immediately.

Please note: We recommend adding all of the recipients to your Campaign before proceeding with the steps below so that you can easily see the sample size.

How to create an A/B Test

1. Go to the Content step of an in progress Campaign.
2. Just below the subject line, click the Show text link next to the Advanced heading.
3. Click the Create A/B Test text link.

4. Click OK, and a second version of your Campaign will automatically be created as an exact copy of the first. You might want to rename the tabs to better represent the changes you are making.

5. Modify the content of the second version as desired such changing the subject name. Click the Add Version text link to add additional versions. You can also rename the default Version 1 and Version 2 by clicking the down arrow icon next to the version name.
6. Click the A/B Test Settings text link.
7. 
8. By default, the different versions of your Campaign will be setup to be sent to equal segments of random recipients representing 20% of your total recipients. Drag the arrow icon to adjust the sample size. A minimum sample size totalling at least 1,000 recipients is recommended for optimum results.

8. Select the criteria on which the winning version should be by selecting from options such as View rate, Click rate or Clicks on specific link i.e. Forward to a Friend or Unsubscribe. 
9. By default, the A/B test will run for 4 hours after the Campaign has been sent. You can enter a different duration if desired, and/or you can manually select a winner and stop the A/B test (see step 13 below).
10.  When using the A/B testing feature, you can now choose which version of your Campaign to send as a test. This means you can thoroughly test all versions of your A/B test Campaign before pressing send to all of your recipients.

    

    Just like the normal testing feature, you will be able to send a test Campaign of your Version to up to five recipients.

11. After your Campaign has been sent, you can view the results in real time via Reports. To do this click the Reports > Name of the Campaign. You will see the A/B Test at the top right of the page and the amount of time remaining along with the Select Winner link. You can click the A/B Test tab at the far right to view the metrics over time.

12. At the end of the A/B test, the winning version will be sent to the remainder of your recipients.
13. Or if you would prefer to suspend the test and immediately send a selected version to the remaining recipients, go to the Campaign's reports and click the Select Winner link at the top right of the page.

Please note: Campaigns that are copied will keep their settings so ensure that A/B Tests are turned off when copying to an account without the functionality.