Profiting Market Copyright 2019

How To Run Great A/B Tests

A/B testing can be valuable because different audiences behave, well, differently. Something that works for one company may not necessarily work for another. In fact, conversion rate optimization (CRO) experts hate the term “best practices” because it may not actually be the best practice for you.

But A/B tests can also be complex. If you’re not careful, you could make incorrect assumptions about what people like and what makes them click — decisions that could easily misinform other parts of your strategy.

To run an A/B test, you need to create two different versions of one piece of content, with changes to a single variable. Then, you’ll show these two versions to two similarly sized audiences and analyze which one performed better over a specific period of time (long enough to make accurate conclusions about your results).

How to Conduct A/B Testing

  • Pick one variable to test. …
  • Identify your goal. …
  • Create a ‘control’ and a ‘challenger.’ …
  • Split your sample groups equally and randomly. …
  • Determine your sample size (if applicable). …
  • Decide how significant your results need to be. …
  • Make sure you’re only running one test at a time on any campaign.

How To Run Great AB Tests 2  700x312 - How To Run Great A/B Tests
If you want to have your test mean anything at all — in other words, be statistically significant, which we’ll go over in the next bullet point — you’ll have to promote the heck out of your content. Send your email to a large enough list, promote your landing page across social, or even throw some PPC behind the blog post link to get enough people to see your test.

Keep in mind that if you’re running an A/B test for a specific audience, you need to keep your promotions tailored to only that audience. For example, let’s say that you’re curious if Twitter followers will like something on a landing page, you wouldn’t want to promote the A/B test content anywhere other than Twitter. Not on Facebook, not through email — just through Twitter.

Okay, so now you know if your experiment worked or not for the metrics you originally set. Awesome! But you can’t stop there. Just remember that your A/B test can have larger implications than just the test metrics themselves. Now, you’ve gathered your data — significant or not — and checked to see if your A/B test had any other unintended consequences. You’re done!

Whatever your experiment’s outcome, use your experience to inform future tests and continually iterate on optimizing your app or site’s experience.

Please Leave Feedback

Profiting Market Logo. 300x300 - How To Run Great A/B Tests

Digiprove sealCopyright secured by Digiprove © 2019 David Leduc