Increasing campaign results with A/B testing

IMPORTANT NOTE: This article may contain outdated information. For the most up-to-date information related to this topic, please refer to the Hungarian version of this page.
In case you do not speak Hungarian, you can always translate the page with Google Translate.

Purpose of the function

A/B testing (also called split testing) helps you to test which version of your campaign content brings the most results. After the test you can use the version that was the most effective. You don't have to settle for only one version of your ideas, but you can try the best ones and real life tests and statistics will show you which one to choose.

What you can test?

With A/B testing you can test the following parameters:

• Subject line: You can test two, three, four or five versions.
• Sender name and email: You can test two different sender name and email pairs.
• Email content: You can test two different email contents.

Process of A/B testing

1. Choose the type of the test
2. Create the email content(s)
3. Create the A/B test delivery schedule
4. Run A/B test
5. Wait while the test is running
6. Evaluate test results
7. Active the delivery schedule of the winner version

Using the A/B test step by step

In the left menu click on the email whose subject line or sender name you want to test. If you want to test email content click on any of the emails you want to test. Then, click the "Create A/B Test" button.

The following screen will appear.

The "A/B test name" will appear in the delivery schedule list.

In the "What would you like to test?" select what you would like to test. If you select subject line you can test 2,3,4 or 5 versions. If you choose E-mail sender or E-mail content you can test two versions.

Depending on which option you chose, you'll see some extra fields appear where you can enter your alternate subjects or sender details. If you've chosen the subject line option the following fields will appear:

Type the different versions of subject lines. If you want to test more than two versions click the "New subject line version" button.

After you finished typing the different subject lines you will see something like this:

Select the email that you want to test for content. If you test the subject line or the sender the email is already selected. However, you can choose a different email for which the A/B test will be created.

Select to which list you want to send the A/B test. If you want to send the A/B test to a specified segment of the list select the segment as well.

Click the "Next" button. On the next screen you can set the sizes of the the groups:

Set the size of the test groups with the slider. The two test groups cannot be bigger than 50% of the selected list's or segment's size.

You also have to define how the winner is selected, whether it is from most openings, most unique clicks, the most clicks on a given link in your email, or manually. Based on the winner version a new delivery schedule will be created, that will send the email to the rest of the list.

If you've selected the "Manually"" option, it depends on you which version you use to create the delivery schedule that sends the email to the rest of the list.

If you select one of the other options SalesAutopilot will automatically create a delivery schedule based on the winner (the version that brought the best results based on the selected parameter). However, this delivery schedule will be inactive, you will have to active it!

Finally, you'll need to select how long to wait before deciding the winner (from one hour up to several days). The result of the A/B test will be displayed inside this time frame, even if you've selected the "Manually" option.

Click the "Next" button to display the last step of the A/B test creating process.

This screen is similar to the normal delivery schedule setup screen. Set the start of the A/B test. If you test the sender or the subject line you don't have to setup that here.

If you want to active the A/B test click the "Save and activate" button. The test will be activated and will be sent at the time you've set. If you don't want to activate the test now click the "Save but activate later" button. The A/B test will be created but it won't be sent even when the time you've set arrives until you activate the A/B test. Chose this option if you probably have to modify the settings before the A/B test starts.

If you've selected the "Save but activate later" option you will see the following screen:

Click the "Activate test" button to activate the test. Click the "Back to the overview" link at the bottom of the screen to go back to email overview.

You can reach the A/B test results from the email overview screen anytime.

If the test is activated you can see the remaining time in the result screen.

The test is finished you can see the winner version.

A delivery schedule is created automatically - but with inactive status - based on the winner (except if you've chosen to determine the winner manually).