How to Automate Your A/B Testing

By ​Jonathan Duch

October 24, 2017

In the world of personalization and ecommerce optimization, A/B testing does not get a lot of attention these days. It is, however, still one of the most popular and widely adopted optimization practices used by digital marketers--and for good reason: ideas for testing are typically easy to generate; tests are quick to deploy, and results are simple to understand. AB tests are not, however, typically easy to automate.

In our experience there are two obstacles that prevent marketers from automating their AB tests: 1) most tests do not reach statistical significance, and 2) marketers lack the technology.

Most AB testing tools offer an auto-promote functionality capable of promoting the best test variant to 100% of the audience once a winner is determined. That sounds great, but in a world where most tests do not reach significance, that capability gets little use. In fact, if you are like most marketers, you often override tests and manually promote a variant before significance is reached. That can have serious consequences, but what else are you to do? You are pressed for time, and you have to make the best decision you can with the information you have in the moment.

Aside from the auto-promote functionality, marketers typically do not have any other automation options. That leaves marketers in a constant cycle of creating, launching, and manually monitoring test performance in order to optimize their website. It is the manual monitoring that takes so much time. You at least have to check each test’s performance once a day if not more often.

Majority Fit ExperiencesLuckily, we can alleviate a lot of that work: Monetate’s Majority Fit capability automates your tests and reduces your managerial load without sacrificing results, all while delivering useful insights to your team.

By deploying Majority Fit with your tests, Monetate will evaluate test performance every few seconds, and gradually allocate more visitor traffic to the winning test variant. This allows more visitors to see the best variant and see it faster than in a normal AB test. It also frees you up from having to constantly monitor performance. The technology makes constant adjustments, including reallocating traffic as variants may perform best at different times, and the algorithm is still in search of a long-term winner. In other words, the algorithm is constantly monitoring, learning, and adjusting so you do not have to.

Frontgate, a Monetate client and early adopter of the Majority Fit capability, ran a simple yet powerful Majority Fit Experience on their Product Description Pages. Using the algorithm, Frontgate experimented with the placement of their product recommendation boxes. The goal was simple: maximize engagement with the product recommendations by placing them is the best location. The result was a 10% lift in engagement without having to manually monitor performance. Frontgate was able to sit back with confidence that the algorithm was driving the best result.

Want to do what Frontgate did? Download the Frontgate case study to learn more.

The good news about automation of tests is that,  not only is it possible, Majority Fit Experiences are available today to Monetate clients. You can start running them now. Learn more about how to do so in the Knowledge Base or talk to your Client Success Manager.

If you are not a Monetate client, we are happy to chat with you about how you can automate your AB testing activities - contact us.  Also learn how you can improve your other optimization practices with the The Top 10 Testing, Targeting, and Optimization Best Practices Ebook.

Jonathan Duch is the Product Communication Czar for Monetate, where he oversees and directs cross-departmental product launches. Jonathan is a 10-year veteran of marketing technology software, and has pursued a career that has taken him from the US, to Europe, to Southeast Asia, and back again. He currently lives in the Washington, DC area with his wonderful wife, and can often be found running the local trails, or sailing in the Potomac River.

Experience the future of ecommerce personalization

Book your personalized demo today.