< Back to stories

Why is split testing a waste of time?

7 December 2018
by Adam Oldfield

Blog post picture

OK, it’s a bold statement to make – as marketers, we’ve relied on the ‘proof is in the pudding’ results of split testing for years.

Instead of endlessly scratching our heads over what’s truly the best creative or the most eye-catching email subject line, for example, we’ve traditionally whittled it down to two (or more) favourites, and let recipients tell us!

In some respects, it’s great that we haven’t just relied on presumptions – or worse still, personal taste. We’ve let actual interactions drive our decision making.

But have we just been sitting on the fence, for all this time?

Many people ask us how we go about split testing during demos, and the question is easy to answer – with Force24 the process is quick and simple, and we show them, so they can see for themselves.

But that doesn’t mean we’re fans of split testing – certainly not any more. And the reasons are two-fold.

Firstly, you’re duplicating effort, and given marketers’ time is so precious, you should be striving to be as efficient as possible. We know you care about ROI – so do we! But think about it with a level head and ask yourself if you’ve ever really been 100% torn between two approaches? You almost always have a gut feeling as to which is best. If you’ve ever been so torn, surely it means either is good enough to go?! And given you shouldn’t just be blasting a message out to recipients once, why not save the other approach for another stage of the journey?

Secondly, the results of a split test are rarely conclusive – a marginally higher open rate may be because of slightly more engaged data, for instance. And, a higher open rate doesn’t necessarily mean a higher number of enquiries. A killer subject line may get some initial engagement but if it’s nothing more than click bait it won’t lead to any real results for your business!

So as professional marketers – when our time (and salary) can represent a significant proportion of the marketing budget – is it smart to duplicate effort?

No. That’s why, when we’re torn with an approach, we simply don’t worry. Instead we test as we go – after all, we’ve got the Force24 platform to play with throughout. We call it iterative testing.

It’s our belief that marketing departments should execute campaigns that educate the audience with content over a number of touches, rather than a single email. With this in mind we use automated journeys to send campaigns, and we tend to push the same topic on email between 3-6 times – each email considering the engagement of the contact and deciding if it’s appropriate to continue the user on the campaign journey.

This process also sees us gradually explore our creative options over time.

Let’s say you have 10,000 contacts and you wish to answer a simple CTA debate. If you split test, 5,000 may receive an email with ‘buy now’ and 5,000 with ‘more info’. Using an iterative approach, however, you’re talking to the same 10,000 contacts over a six-week campaign, meaning you’ll touch them six times. Three emails will have ‘buy now’ and three will have ‘more info’. The stats will then be based on the actions of 30,000 touches each.

Now, obviously if you detect a significant difference after email one, it’s really not necessary to run through the whole test – it would make sense to update the remaining emails to reflect these findings. But otherwise it’s a case of basic maths – the audience will give you the data you need to support your argument.

If you’re in need of MORE help, then it’s worth mentioning that Force24 clients have access to an intelligent ‘test feature’ which enables them to tag all of their campaign email and SMS stages. This could therefore be used in the CTA example above, for instance, to reduce effort, assess performance and still gain the insight needed to improve campaigns. It’s even possible to retrospectively look at where the CTA has been used in the past, to add greater depth to the test data!

Fancy a demo so you can see how easy it is to build automated journeys? We’ll even show you some of our own campaign stats if you like! Contact us to pencil something in the diary!

If you’ve enjoyed this blog and are interested in more tips to boost your ROI x 12 in 2019, download our latest guide.

Photo of Adam Oldfield
Adam Oldfield
Managing Director & Founder

Book a Demo

Reading about a platform is all well and good, but to see it in action is another thing entirely! Get in touch and book a free one-to-one demo to see how you could get the most out of Force24.

Book a Demo