January 10, 2012
If you've ever shopped for a new vehicle, you've made a mental list of the features that most interest you. Is it red? A two-door? And does it have the 19" alloy wheels?
What's great about these lists is that they make the buying experience simple. If the car isn't red, look no further. The challenge, though, is when safety, reliability, and performance are on your list. To evaluate these, you need to be behind the wheel, under the hood, and in between the seats. Unfortunately, when a car doesn't measure up, it's not always obvious at first glance.
And so it is with testing solutions. KPIs, including conversion rate and AOV, rank only among the more visible ways of measuring a tool's impact on your business. The others? Get ready to look under the hood at ease of initial setup, ease of ongoing use, and how risky use of the tool is in the first place.
But how can a tool be "risky"? Like a car without airbags, some tools simply aren't built with the proper fail-safes to protect companies against the unexpected, including:
1. Cloud-based, Client-side Architecture: Sharp and sudden increases in traffic can overload a server, quickly taking a website down. Your testing solution shouldn't be cause for concern. Cloud-based, client-side tools operate independently of your server, hosting center, and ecommerce platform. They utilize vast, redundant, globally-distributed server networks that offer near-infinite scalability, and perform all necessary tasks (data collection, decisioning, content rendering, etc.) via a single round-trip. Some testing solutions require multiple server calls for each individual test element. Others reroute your traffic through a proxy, creating a single point of potential failure should that proxy ever fail. But the world's biggest, highest-traffic websites employ cloud-based architectures. Your testing solution should too.
2. "Look Before You Leap" Campaign Audit: Quality Assurance should have redundancies, involving both the client success professionals you work with on a regular basis, as well as automated controls that detect website changes that might conflict with active campaigns. When your testing solution can auto-detect site changes that are incompatible with a campaign, it's able to disable that campaign in real time and alert your services team. Without this ability, the shopping experience can "break," yet go unnoticed for an extended period of time.
When it comes to selecting a testing solution, many vendors speak of potential return on investment. To be sure, campaigns that prove their worth through incremental annual revenue and high statistical significance are essential ingredients. Still, another dimension of ROI includes the impact on revenue when testing solutions don't always perform as they should. When these risks materialize, your site can lose money—and lots of it.
A great testing solution, then, emphasizes both sides of the coin. Business-relevant features on "Heads" and proper fail-safes on "Tails." Make it your New Year's Resolution to demand this in 2012.