Testing all aspects of your email campaigns is the only way of ensuring you extract full value out of your email marketing software. The statistics that emerge are clear, precise and, unfortunately many feel, inarguable. Testing proves, and that is its strength.

You will be able to access a considerable quantity of data through your email marketing software analytics during and at the conclusion of a campaign. For instance it will show the open rate clearly. You will be able to compare the figure with that of the previous campaign and if you changed the content of the Subject Line, perhaps making it more impactive, you might well feel the change justified. 

Unfortunately, this is not so. There might have been any number of reasons why the open rate changed even if you kept all other aspects exactly the same. 

For instance, if you were offering holidays along the Amalfi coast the name might have been more recognisable than the the one in the previous campaign. Perhaps it was featured in a popular TV travel programme or, more helpfully, a media personality might well have been photographed nearby. These triggers could well have changed the point of view of your subscribers.

The way to eliminate these variables is to split your email marketing list.

With figures picked for no reason other than convenience, let us assume you have some 10,000 subscribers. You have decided that your new campaign will be sent to 4000 of these. You now need to subdivide this reduced list to give a ‘statistically significant’ number of subscribers to provide a test group.

You might think the most sensible option is to go for a 50:50 split but this has dangers associated with it. If you have been in email marketing for some time you will have already tested your Subject Line and worked out that your current subjectline style provides a good return. It could be that your change could give a lower open rate and you need to reduce that risk.

If your experimental change is rather dramatic then it might well be that any improvement will be shown by a low trial percentage, say 400. So send off the two different emails, one with the normal Subject Line and the other with the more radical one. Once you get the returns, any improvement will be significant.

But if your change is more subtle, perhaps using the surname in the Subject Line rather than a forename, then to get a dependable result you might well need a greater figure, perhaps 1000. If the modification is rather limited then it is probable the difference in the stats will be as well.

If the test group gives a noticeable improvement in open rates compared to the normal group then it is unfortunate that even then you cannot say for certain that the new Subject Line is a permanent improvement. It is best to test again, especially if the change was radical. There is always the risk that the reason the open rate was higher was because the Subject Line was novel. This time go for a higher split percentage to be sure.

Test and test again: the email marketing mantra.

With WizEmail you can automatically a/b split test your campaign content and/or subject lines with just a couple of clicks – Wiz will even send the winning campaign to the remainder of your data based on opens or clicks – clever little WizBots!