What is A/B Testing?

To perform an A/B split test properly you need to plan it and measure the outcomes carefully. You also need to make sure you remove any external bias to the results. For an extreme example it is no use testing a message that offers a special weekend discount if you send half the emails out on Friday and half on the following Monday morning after the discount period is over. The response rate is bound to be different. 

There are other factors that can bias results: time of day, location of reader, day of week and so on. To deal with this situation it usually makes sense to alternate the offer you are presenting to sales prospects so that half see one version and half see the other in strict rotation.

One technical issue that needs to be addressed is to make sure that someone seeing offer A, sees the same offer A if they return to your website. Likewise, someone seeing offer B needs to be shown offer B if they return. This degree of control over testing can usually be done with cookies.

Split testing can be applied to email marketing too. For example, the autoresponder Aweber allows you to split test email broadcasts. When you send out a broadcast as part of an email marketing program, you can write up to 4 different versions of your emails. Aweber will then track the conversion rates of each version.

An effective way of using the results of an A/B split test is to run a test on a subset of your email list.

Suppose you have 50,000 names on your list. You can do a split test on 10,000 of them (you need a good number to generate statistically valid results). Once you have determined which email message generates the greatest conversion rate you then use that version to email the remaining 40,000. This will maximise your profitability.