A subscriber can enjoy a beautifully designed newsletter and still act on a short, plain-looking reminder. That is not dishonesty. Different messages have different jobs, and a preference expressed in a survey is not a prediction of every purchase.
Plain text vs HTML email marketing testing is useful when it answers a practical question: which presentation helps this audience take this next step? You do not need to settle the argument for the entire internet. Your next campaign will do.
Decide whether you are testing code or appearance
A true text/plain email contains text and line breaks, without HTML formatting or embedded images. An HTML email can include styled text, links, buttons and pictures. It can also look remarkably plain.
This matters because a message described as “plain text” may actually be a simple HTML template. If both versions are HTML, your experiment compares two presentations, not two coding formats. Label it accurately in the test record.
For a first design test, you might compare your usual illustrated HTML email with a text-led HTML version. That keeps the technical format consistent while changing the visual approach. If you genuinely want to compare HTML with text/plain, check how your sending setup creates and reports on each.
An HTML message can also carry a plain-text alternative. These are alternative representations of the same message, not two randomly assigned test groups. Keep their essential information consistent.
Choose a campaign with a clear decision
Pick an email where the intended action is easy to identify. A software company’s product briefing might aim for registrations. An international charity’s supporter update might invite people to book an online event. A retailer’s new collection might need images to explain what makes the products different.
The question is not whether pictures are fashionable. It is what useful work they do here.
Imagine a business software provider inviting existing customers to a briefing about a reporting feature. One version uses its usual branded header, illustration and button. The other presents the same invitation in a restrained, text-led layout with a clearly labelled link. Both give the same date, agenda and registration destination.
That is a manageable comparison. Replacing the invitation with an entirely different offer would answer a different question.
Make the comparison fair
Write a short test plan before building either version. For the briefing example, it could contain:
- Question: does a text-led presentation increase completed registrations compared with the usual illustrated layout?
- Audience: customers eligible for the same briefing, randomly divided into two non-overlapping groups.
- Held constant: sender, subject line, preview text, offer, core wording, destination and send time.
- Changed: the presentation, including the illustration and treatment of the main action.
- Decision: assess registrations after the agreed response window, alongside complaints, unsubscribes and production effort.
A design comparison may change several related visual elements. That is acceptable if you describe it as a comparison of complete presentations. It will not tell you whether the illustration alone caused the difference. Test that separately if it becomes important.
In a multi-market programme, randomise within comparable language or market groups where the audience supports it. Sending the illustrated version to established UK customers and the plain-looking version to new overseas contacts mixes presentation with very different circumstances.
Keep the approved offer and translation consistent within each comparison. Give one person responsibility for the final test record, so local edits do not quietly turn one experiment into several.
More recorded opens do not necessarily mean more people read the message.
Measure the action, not just the open
Choose the result that matters before the email goes out. For the briefing, that means completed registrations, not simply a visit to the registration page.
Recorded opens are a poor basis for a format verdict. Standard image-based open tracking cannot work inside a true text/plain message. In HTML messages, privacy protections can load remote content without a person reading the email, while blocked images can prevent an open from being recorded.
More recorded opens do not necessarily mean more people read the message. A tidy chart is not always a useful answer.
Use the same measurement method and response window for both groups. Record completed actions against comparable denominators, such as delivered emails, and inspect delivery differences rather than hiding them. Check the underlying counts as well as the percentages.
Clicks can help explain the journey, but automated security checks may visit links too. Confirm useful actions in the booking, order or enquiry system where possible. Do not assume every reported click is a person expressing interest.
Give the result room to be uncertain
Agree the sample, response window and success measure in advance. Let recipients have a reasonable opportunity to act; a business enquiry may need longer than an event booking.
Do not stop as soon as one version takes the lead. If there are very few completed actions, an apparent winner may tell you little. Use an appropriate statistical comparison where the data supports it, and report an inconclusive result honestly where it does not.
Repeat the comparison on similar campaigns before changing every template. A text-led reminder can work well without proving that a visual product launch should lose its pictures.
There is useful information even when neither version clearly wins. If results are similar and one is easier to produce and check, that may justify using it for that campaign type. Record the operational reason rather than claiming a conversion improvement you have not demonstrated.
Keep preferences and quality checks in place
A test result is not permission to ignore a subscriber’s explicit choice or accessibility needs. Respect relevant preferences, and do not make either version deliberately awkward in pursuit of a cleaner experiment.
Check both messages as received: reading order, links, essential conditions, sender identity and the unsubscribe route. Removing pictures is not a guarantee of inbox placement. Delivery still needs its own checks.
For help writing the text-led version, our plain text email copy guide focuses on making the offer clear without relying on photographs.
Choose one campaign, write one testable question and agree what would count as a useful result. The prize is not proving that plain text or HTML is universally better. It is making your next decision with something firmer than a preference for blue buttons.
The WizEmail Team
