#21 - Our Learnings From Running 269 A/B Tests (in Feb)

Okay this is just ridiculous.

Can you believe that we’re already one week into Q2?

Guess I’m having too much fun (they do say time flies when you’re having fun after all 🤷‍♀️).

For today’s newsletter, I’m going to be walking you through some of our learnings from running 269 a/b tests in Feb.

A quick heads up to not take this as gospel - use this as a foundation, because it’s going to differ for each brand.

Let’s begin.

P.S. Please note some images are cropped on purpose for confidentiality

1) On CTAs

We ran a couple of interesting CTA tests for several brands. and here are some of the results:

Note: Try to ensure your tests are statistically significant, so you have the confidence that the same results will be produced across multiple campaigns/flows.

a) Shorter CTAs (winner) vs longer CTAs (loser)

b) CTAs with icons (winner) vs CTAs without icons (loser)

c) Transactional CTA (winner - 2/3 times) vs personal CTA (loser)

d) CTAs in product blocks (winner) vs without (loser)

2) On Product Blocks

a) Product blocks with just discount (winner) vs price slashed with discount (loser)

a) Product blocks without price tag (winner) vs with price tag (loser)

3) Having a dedicated lander for sales emails

Always an interesting one to test vs driving traffic straight to a PDP/collections.

Additional tip: Use these learnings to leverage and drive more sales during ‘special occasions’

4) Sender name vs brand name

Adding some form of personalization always helps make the brand feel more ‘human’ vs a corporate face, and tends to work well.

Test also switching up your FROM sender names.

5) SMS (w/ vs without GIFs)

For this test, the SMS without a GIF actually performed significantly better (5.38% vs 0.5% click rates) vs the SMS with a GIF.

Funny because MMS costs more, but as always, this differs from brand to brand.

You wanna focus on the click rate, place order rate, as well as ROI when running this.

And that’s a wrap!

Testing is an integral process to growing, and should be part of your retention marketing stack.

You should never be worried about a test ‘failing’, as it gives a direction you do not want to head in.

Don’t underestimate these tiny wins too. The 1%’s compound really quickly, and over time it’ll shock you how much they impact deliverability/engagement/revenue.

Also, this might be helpful - it’s a link to an a/b testing cheat (we use the same structure for our clients) click here.

Have a great weekend!