How to Create a Culture of Testing and Optimization

By Evan McGonagill

July 26, 2018

I’ve listened and talked to dozens of companies to discuss how they use testing to drive higher revenue, leads and traffic. As a result, I’ve become convinced that some have developed what I will call a ‘Culture of Testing’ and others have not. And I’ll bet if I were to do a big statistical research project, I would find that the companies that have a culture of testing dominate those that do not. So what is a culture of testing and how do you get there?

To get back to basics, a culture of testing means your organization commits to working in a way that is both testable and measurable. Analytics from these tests are what drive future business decisions.  

Organizations that have built a culture of testing have incorporated testing into their strategic decision making processes at a deep level. Rather than a side project or a temporary undertaking, it is integrated into their operations on a consistent and ongoing basis, and is owned by everyone as a shared responsibility. A culture of testing lays the foundation for success with personalization.

Outlined below are ten ways to build a testing-centric organization.

The first three suggestions below discuss the cultural aspects of testing; specifically, how you embed experiments into the backbone of your organization so that

  • Everyone understands the role of testing
  • The vicious practice of back seat driving decisions in lieu of testing is stopped, and
  • The core team is in place to design and implement tests.  

The remaining seven suggestions are the roadmap for your core team to adopt a disciplined, ongoing practice around testing that will help your results to soar.

Together, these ten steps will guide you to a smart and rigorous testing practice that will fuel your personalization efforts and deliver business impact for the long term.

1. A Culture of Testing Starts at the Top

Believe it or not, in this data-rich and technology diverse time, many companies are still making user experience decisions based on what top management thinks they should do. That’s right. Even Presidents of sizable companies are weighing in on website copy, offer types, button colors and more!

It really is the most damaging kind of back seat driving, and it’s the opposite of a culture of testing. Executives need to ensure that their professional rank and file have the tools, training and resources needed to perform data-rich tests such as ab testing on their marketing efforts. And they need to step out of the way and let the results of data-driven testing be the foundation for change—and not personal opinion.

2.Testing Requires a Cross Functional Team

Testing is not a solo operation. In order to be fully effective, it requires a team that probably resides in more than one department. If you are a manager, you need to work with your peers in other groups to pull together the key players from Marketing Operations, Creative Design, User Experience (UX), Marketing, Email Marketing, Product Marketing and perhaps even other groups as your organization dictates. And if you are still just setting up your testing solution, you may need to have someone from Marketing Operations and/or the IT group involved to make sure the data is flowing properly, and the system is operating as designed.  

3. Make Testing Part of the Team’s ongoing Responsibility

Establishing the team certainly comes first, but please know that this is not a one-time taskforce that can be disbanded after a couple of tests. Testing needs to be a regular part of the core team’s responsibility. If you are a company that has formal performance goals, then the testing duties should be included in the job descriptions and annual HR plan. How much time each member will need to devote to testing can vary dramatically depending on the size of the website, the number of products, the larger marketing plan and more. You will have to determine the job responsibility time allocation as you go along if this is a new program for your organization.

4. Separate out the Hypotheses and Testing Ideas from the Actual Testing

Ideating on what to test should be a distinct phase, separate from the actual testing process. This is the time to get creative! Think about all aspects that can be optimized: images, text, fonts, colors, screen location and much more are all things that can be tested.

This is a great time to review your Google Analytics, heatmaps, your DMP and other data to fully understand the current situation and use this data to help inform your brainstorms. Resist the urge to immediately narrow down your options to what you are going to test. Instead focus on creating more ideas. This is where you should be asking yourself and your teammates lots of questions such as:

  • What do you want to learn?
  • What are all of the possible ways you can improve the user experience?

Step beyond the obvious elements and consider some new approaches. Don’t shoot down even the most outrageous ideas at this point. You might even uncover a whole new vein of traffic and revenue gold to mine by going with a completely different design method. These cross functional idea-generating sessions could be the epicenter of great things for the whole department, so don’t cut them short.

5. Clearly Establish What the Test is and How it Will Take Place

Once you have a boatload of ideas, now is the time to narrow down your opportunities and establish your ground rules. Try answering the following questions:

  • What are you trying to learn?
  • What specific elements will you test? How will you test?
  • When will you test?
  • And for how long will you test?

These are all valuable questions that must be answered before you commence your test. While some of these elements can be changed, you’re best off establishing the test criteria and game plan before the test begins.

6. Don’t go Down Rabbit Holes

Don’t let distractions take over your team and your testing. If you’re not careful you will begin stepping away from your original objective and start testing something else. Do it a few more times and you will at some point wake up and realize that you are very far from your intended destination.

Let this serve as your reminder to remain crystal clear about what you are testing and how you will approach the test and don’t deviate from those objectives. Of course, you can always do another test, just don’t fool yourself into thinking that your test drift is going to answer your original question.

7. It’s Great to be Wrong

It’s human nature to want to be right and be right as frequently as possible. We’re all just wired to avoid having to say we made a mistake. But everyone on the core team needs to realize that perhaps the biggest learnings come from the times when we were completely wrong. Don’t be afraid to have one of your hypotheses be completely blown out of the water. If you’re not having failures, you’re not pushing the envelope enough. And with ever improving and changing competitors, technology, and user preferences, we all need to be aggressive in our testing. Instead of feeling like an aggressive test was a failure, consider the tests where you didn’t learn anything to be the big mistake.

8. Don’t Re-debate a Test

It’s like debating a great ball game after game day and disputing the outcome. Nothing will change the score and the result that happened that day with those players, coaches and fans. Sure, you can reminisce all you like, but the score is in the history books. Same thing with website, email or other UX testing. Finishing a test does not mean opening up the floodgates to disputing the results or just doing what you want to anyway because you feel like it. You need to honor the test and the test results, or do another test. Use the facts you learned in the test to stand strong with your decisions and the actions you are going to take as a result of the test. And if you feel like your hypotheses or test should be re-examined, do another one.

9. Incorporate all Possible Learnings in Subsequent Rounds of Testing

Testing is all about getting smarter and not having to start at the beginning every time. For every test you are about to perform, recap what you have learned so far. Make sure if you are doing a closely related test, that you incorporate what you have learned. If you already have learned something, don’t reinvent the wheel. Instead go back to ideating some other things that you can test (See #4 above). Quarterly, create a key learnings document for your teammates that are not intimately familiar with the details of what you have learned. That way they can evolve as well.

10. Have Fun!

Testing and optimization are all about improvement. It is a methodology to consider multiple variables and uncover the facts and the truth. It gives you guns in your bullets to refute the backseat drivers and act very powerful in meetings. It allows you to get intimately familiar with your visitors, readers, buyers and clients behavior and thinking. And it allows you grow in your career.

Evan McGonagill is a content writer for Monetate, where she researches and produces whitepapers, blog posts, and other material about commerce and personalization. Evan has a background in libraries and archives, and she uses her interest in the structure and flow of information to think about how brands can harness data to build more personal connections with their customers. When she isn't in the library or learning about personalization, you can find her playing music in West Philadelphia.

Experience the future of ecommerce personalization

Book your personalized demo today.