Data—can’t live with it, can’t live without it. There’s an old saying that I’m sure most of you have heard: “We manage what we measure.” Other variations go something like this: “If you can’t measure it, you can’t manage it,” or “Without data, we’re flying blind.”
The information age has given us the ability to collect massive quantities of data. In a 1960s episode of the original Star Trek series, there was one particular episode in which the crew of the Enterprise discovered a deserted planet, and on it a computer. Spock reported to Captain Kirk: “Captain, there’s over a terabyte of data here, more information than the world has ever known.” Today we can store a Terabyte of data on a single disk drive.
And so companies are overflowing with data that is used to make million – and billion – dollar decisions every day. Having data to support our conclusions gives us a greater sense of confidence. But should it?
Data acts like a drug for many executives. It gives us a warm fuzzy feeling; it helps us feel confident in our conclusions; it helps us justify our decisions; and it insulates us from blame should our plans go awry.
Unfortunately, all too often today, I see executives using bad data—and they don’t even know it. The reasons are many, but at the most fundamental level it’s because no one has done the grueling work of figuring out what the right data is.
One of the most common misuses of data is estimating the size of a new market. I’m sure many readers have heard the one about the shoe company whose business was slowing in the U.S., so it sent two marketing managers to Africa to explore the market there. The first called back to headquarters in the U.S., reporting, “There’s no market here for us. No one wears shoes.” The second called in reporting, “There’s a huge market here. No one wears shoes.”
Given the same data, one could conclude that there is either a tremendous market or none at all. And one could use the same data to kill any plans to take the business to Africa, or to make a huge investment there. So what data is needed in this case? The needed data is “How many people want or need shoes and can they pay for them?” It’s not a question of how many have or do not have shoes.
Obviously my story is an anecdote, and no established company would make a decision about entering a new market based on such simple reasoning. But even when more data is collected and analyzed, and when the team believes its decision will pay off, sometimes it’s just a failure. So what happened? All the data in the world couldn’t save us. We thought we were managing what we measure.
A big part of the problem is that we often start with the data and ask, “What can we learn from it?” But that’s simply backwards, because available data can severely bias our conclusions. Instead we need to ask, “What do we need to know? What decisions do we need to make? And what data is necessary to support those decisions?” Then we can search out the data, if it’s available, or we can install a means of collecting it. What we can’t do is settle for data that might be available and force-fit it to our problem.
That’s no different than abusing drugs to make us feel better for a time – until we suffer the crash of a hangover.