Dangerous Metrics and Risky Thinking

In a recent posting on e-consultancy the author questions the validity of the results we get when we use mathematical models to consolidate the mass of data we get from our Big Data. It’s a really good question. After all, people are not particularly good at dealing with large masses of data, not to mention the fact that (as the author notes) no corporate executive is going to sift through thousands of tweets in order to extract and track trends.

So, this becomes especially important when we’re supposed to be running “data-driven businesses.” If the summary data is flawed (for whatever reason), then chances are the data-driven decisions will be little more than drivel.

It also means that, as marketers, we need to delve into something that most of us can’t stand; mathematical models. If we want to make valid decisions (as I am sure most of us do) then we need to take the time to understand how the data we’re examining came to be. What was the underlying data? How valid is that? How was the summary produced and what assumptions were used in those formulas?

These are vital, and very difficult, questions to answer. One major reason that they are so hard is that our brains just aren’t wired to deal with this kind of information very well. In Daniel Kahneman’s great book, Thinking; Fast and Slow he goes deeply into why this is true and what to do about it. Greg Satell has written an excellent post about it, if you don’t want to take time to read the book.

But, the bottom line here is that this kind of problem is extremely dangerous, if we want to make good decisions based on good data. Because the modeling problem mentioned above plays right into this wiring of our brains; our smart (slow) brain will readily accept erroneous conclusions fed to it by our instinctive (fast) brain if they seem reasonable. This is ameliorated when we have a lot of experience in that particular field.

But, the major problem here is that none of us has a lot of experience in the interpretation of Big Data yet. And in this highly accelerated business climate, it’s so easy to look at the numbers, come to a conclusion, and jump into action. But there’s a reasonable chance, given this scenario, that that action is going to be off base, maybe far off base. So, if we don’t take the time to understand it, then we’re basically back to deciding by our gut. And then what’s the value of all the investments we’ve made into gathering, analyzing, and interpreting Big Data?

Until we get the experience and the validated tools (which, again, takes time) to make good, Big Data-based, decisions quickly, the only answer is to slow down, deeply understand the data and the problem, and let our smart (slow) brains take the time to make good decisions, based on both our gut and what the Big Data is really telling us.