How to create an A/B Test in Microsoft Dynamics

An A/B test allows you to create multiple versions of an Email Campaign. This means you can create slightly different versions of the same campaign, such as editing the subject line, images, content, etc., in order to judge which version receives the best statistics. The A/B test will automatically send the different versions to a subset of your total recipients.

The version that results in better metrics after this test will be sent to the remainder of your recipients. You're able to define which criteria you'd like the reporting metrics to be based on, i.e. views, clicks, etc., and also decide on how long you'd like the test to run before the final version is sent out. Or, if you would rather manually select the winner, you can of course stop the test before the designated time period and have it be sent to the remaining recipients straight away.

Please note: We recommend adding all of the recipients to your Email Campaign before proceeding with the below steps to ensure you can easily see the sample size.

You will need to have the Versions setting activated in your account in order to send an A/B test. Contact [e-mail] who will be able to activate the A/B split test setting in your nominated account.

How to create an A/B test

1. Go to the Content step of an in progress Email Campaign.
2. Just below the subject line, click the 'Show' text link next to the Advanced heading.
3. Click the Create A/B Test text link.

4. Click OK, and a second version of your Campaign will automatically be created as an exact copy of the first. This can be seen as a new page, with both pages automatically labelled 'Version 1' and 'Version 2' respectively.

5. Modify the content of the second version as you'd like, such as changing the subject line and altering your placement. Click the Add Version text link to add additional versions. 
6. Click the 'A/B Test Settings' text link.

7. By default, the different versions of your Campaign will be set up to be sent to equal segments of random recipients representing 20% of your total recipients. Drag the arrow icon to adjust your sample size. A minimum sample size totalling at least 1,000 recipients is recommended for optimum results.

8. Select the criteria by which the winning version of your email will be chosen. The options include View rate, Click rate or Clicks on specific link, i.e. Forward to a Friend or Unsubscribe. 
9. By default, the A/B test will run for 4 hours after the Campaign has been sent. You can enter a different duration if desired, and/or you can manually select a winner and stop the A/B test. To stop the test, go to the Email Campaign's report and click the Select Winner link at the top right of the page.
10. After your A/B test has been sent, you can view the results in real time via Concep Reports. To do this, click on the name of the email campaign and then you will see the A/B test in the top right of the page and the amount of time remaining along with the Select Winner link. You can click the A/B test tab at the far right to view the metrics over time.

Please note: Campaigns that are copied will keep their settings so ensure that A/B Tests are turned off when copying to an account without the functionality.