Beware the dangers of data

Subscribe to enjoy similar stories. Managers are better equipped than ever to make good decisions. They are more aware that human judgment is fallible.

They have oodles of data about their customers and products. They can use artificial intelligence (AI) to analyse, summarise and synthesise information with unprecedented speed. But as the pendulum swings inexorably away from gut instinct and towards data-based decisions, firms need to be alive to a different set of dangers.

In a recent paper Linda Chang of the Toyota Research Institute and her co-authors identify a cognitive bias that they call “quantification fixation". The risk of depending on data alone to make decisions is familiar: it is sometimes referred to as the McNamara fallacy, after the emphasis that an American secretary of defence put on misleading quantitative measures in assessing the Vietnam war. But Ms Chang and her co-authors help explain why people put disproportionate weight on numbers.

The reason seems to be that data are particularly suited to making comparisons. In one experiment, participants were asked to imagine choosing between two software engineers for a promotion. One engineer had been assessed as more likely to climb the ladder but less likely to stay at the firm; the other, by contrast, had a higher probability of retention but a lower chance of advancement.

The researchers varied the way that this information was presented. They found that participants were more likely to choose on the basis of future promotion prospects when only that criterion was quantified, and to select on retention probability when that was the thing with a number attached. One answer to this bias is to quantify everything.