The complete glossary for marketers
You can think of a webpage as a collection of elements: images, buttons, text, etcetera. In an A/B test, you can only test one element. For example, an “Add to Cart” button is an element on a webpage that can be changed. It could be altered to be green, orange, or red. Each separate color is a separate variant of that same element.
In a multivariate test, multiple variables are tested together at the same time in order to determine which combination produces the best outcome.
The two main ingredients of a MVT are elements and variants. Elements are the content or creative you want to test. Variants are the different versions of each element you are testing.
Because a MVT is used to find the single best combination of elements and variants, it has to explore and test all possible combinations. Calculating the number of combinations before running a MVT is a matter of simple math. Here’s the simple MVT equation.
[Number of variants of “A” element] X [Number of variants of “B” element] ... = [Total number of combinations to be tested]
By way of example, if we were testing two elements, each with 2 variants, the total number of unique combinations would be 4.
Because of the math, adding new elements or variants to a MVT quickly increases the total combinations being tested on your webpage. For instance, let’s assume you’ve chosen to MVT the following elements and you want to know how many combinations you’ll have based on the number of variants you’re creating.
If you’re testing 2 variants for each element, you will have 8 combinations of content (2 x 2 x 2 = 8).
If you’re testing 2 variants for “A” and “B” and have 3 variants for “C.”, you will have 12 content combinations (2 x 2 x 3 = 12)
If you’re testing 3 variants of each element, you’ll have 27 content combinations (3 x 3 x 3 = 27).
Relatively simple MVT tests can produce a large number of test combinations and can become complex quickly, so it’s worth finding an expert to carry it out.
In the marketing world, testing is a great way to understand your audiences better, learn how to cater to them, and optimize for your desired outcomes. Here are a few ways multivariate testing offers an advantage:
Because of the number of content combinations, multivariate tests require a lot more traffic than A/B tests. Here’s a quick checklist to see if the page you want to test qualifies for an MVT:
If you can say “yes” to the checklist above, a multivariate test may offer the fastest way to improve your webpage user experience. Otherwise, a simple A/B test may be what you need to do the job.
Historically, multivariate tests are difficult and time consuming for marketers to create.
With Monetate, you easily build multivariate tests using the same, sentence-based campaign structure that answers Who, What, When, and Why. That means your tests are centered around who actually sees them as opposed to complex testing matrices that focus on where the test is shown.
With the ability to test any page of your website, including running a multi-page A/B/n or multivariate campaign, Monetate Test & Segment™ is the most powerful, yet easy to use, testing solution ever built.