May 30, 2013
Lately, I’ve been thinking a lot about observational bias. You might be sitting there thinking: Bruce, what could observational bias possibly have to do with website optimization and creating a great online experience for my visitors?
But observational bias actually has a huge impact on the world of website optimization, because it involves looking for the answer to a problem where you think you can look instead of looking for the answer to a problem where the answer actually is.
Sound complicated? It is, but let me share a story with you that will clear things up.
A policeman is walking by a bar one night, and he sees a drunk man crawling around on the ground beneath a lamp post.
“What are you looking for?” the cop asks the drunk man.
“I’m looking for my house keys,” the man says. “I lost them around here.”
“I’ll help you,” the cop says. Together, they begin to look around under the streetlight.
But after a few minutes, neither one of them can find the keys.
“Are you sure this is where you lost your keys?” the cop asks.
“No, I’m not sure of that at all,” the man says. “I might’ve lost them in the alley.”
“Then why aren’t you looking in the alley?” the cop asks.
“Well, this is where the light is,” the drunk man says.
This story is a great example of observational bias, and it’s also why observational bias is more commonly called the streetlight effect. And it points to exactly what’s wrong with the current mindset around website optimization.
Most brands, unfortunately, are using data to see if their assumptions about their visitors are holding up instead of asking really important questions that may not be answerable using the data they already have. They are looking where the light is, and typically that light is on the website. We have all of these web analytics, and they all tell us about the website.
The problem, of course, is in the alleyway, and it’s the customers—and we’re completely in the dark about them.
Case in point: If I asked the VP of Marketing at a huge ecommerce brand what the most popular page on the company’s website is right now, that person would likely be able to answer me immediately.
But if I asked: What page do your most valuable customers visit the most? That same VP of Marketing would have to go back to their web analyst and wait to find out. Why? Because currently, the entire website is being run based on where the light is shining, not where the answers actually are.
So what’s the solution? Well, the tools aren’t perfectly in place yet, but they are coming. And they’re going to change the face of website optimization.
If you’re interested in seeing what that future looks like, we’ll be at IRCE 2013 previewing Monetate LivePredict next week. Feel free to stop by Booth 712 to see what’s in store. And if you won’t be at IRCE, follow Monetate on Twitter for some LivePredict sneak peeks.
Streetlight photo courtesy of ShutterStock.