Split testing is just about as dependable a tool as we have for validating a change in an email marketing campaign. We have an original, the control, and we compare it to one, the variant, that has a single feature that is different from the control. It stands to reason that if the variant gives statistically better returns than the control, it is the way to go. QED and probably other initials.
It’s not quite that straightforward though. You should, must in reality, believe the returns. If you ignore them, then you’ve wasted all the time and effort of the split test, but worse, you’re reducing your RoI. 
However, as with most returns, they need to be interpreted.
If you split tested an email marketing campaign where the purpose was to obtain sign-ups for a conference, and the factor you changed was to include pictures of happy previous attendees, then it does not necessarily follow that pictures of exultant office staff will have the same benefit for an email marketing campaign for printer toner. You need to test again for such different products.
You will probably have split your email marketing lists and offered a certain product to one list, and then to those picked on slightly different criteria. If the alteration enthused one group, the others, by definition, are different, with different triggers. Say you split the lists on age grounds. Nostalgia might work wonderfully on one, but be incomprehensible to the other
What, then can you believe?
Firstly, the returns are impeccable. They cannot lie. If they prove the change of colour of the CTA button gave a 4% increase in click-throughs, then go with it.
The caveat, isn’t there always one, is that interpretation is required. There are variables and they might have affected the returns. That specific split email marketing list had specific metrics that probably won’t apply to others. You don’t have to guess though. If you run other split tests on other lists, then you will know what to believe.
Split testing is ongoing. One good result does not mean you’ve cracked it.
