Examining noise: a flaw in human judgment: a penetrating study of decisions
Daniel Kahneman, Olivier Sibony and Cass R Sunstein
This is the first book I read that has a chief of staff. The usual acknowledgments at the end begin with thanks to the chef. This suggests that the author team or subject needed a coordinating presence – a guiding voice to iron out differences of opinion or interpret complexities.
It is a tribute to the authors that none of these difficulties are evident in Noise. But this is the least of their achievements. Carefully developed argument and careful sifting of a wide range of evidence creates a penetrating study of an important dimension of human judgment.
Considering the status of the authors, such an achievement should come as no surprise. If these sociologists were footballers, they would be the team of Ronaldo, Messi and, of course, Harry Kane.
Daniel Kahneman holds the Nobel Prize in economics. His book Thinking Fast and Slow explains different forms of decision making with such clarity that it has become an unexpected bestseller. Cass Sunstein led the development of Nudge Theory, explaining how small changes in the way decisions are presented and structured can have a big impact on results. The last member of this triumvirate, Olivier Sibony, is an academic and former management consultant.
Their assembled intellectual firepower explains the differences in outcomes when individuals make decisions on similar issues. This is defined as system noise. It is the “unwanted variability of judgments that ideally should be the same.” This is not a trivial problem – it “can create widespread injustice, high economic costs and errors of all kinds”.
The noise openness example demonstrates the human cost and the challenge. A prominent American judge, Marvin Frankel, drew attention to the very different rulings of his judicial colleagues. The examples are revealing. Two men, both first-time offenders, were convicted of fraud for taking small counterfeit payments. One was sentenced to 15 years. The second was sentenced to 30 days.
This variation occurs for judgments about both our past and our future – “it is the variability of historical statistics on conviction decision and underwriting premiums. It is the uncertainty in the forward-looking prediction you make about a particular outcome. ”
This is further demonstrated by an insurance case study. Insurance executives are asked about the variation that can be expected in expert judgments. A 10 percent difference was the most popular answer. In fact, an audit of those decisions found the actual difference to be 55 percent. An insurer could set a premium for a policy at $ 9,500. A colleague assessing the same claim could set the premium at $ 16,700.
The noise argues that the defining characteristics of such a difference is that it is undesirable and underestimated. The authors conclude that “wherever there is judgment, there is noise, and more than you might think”.
A frequent claim is that groups can reduce noise. A “wisdom of the crowd” can emerge with a sufficient number of decision-makers. The authors disagree, due to the impact of “information cascades”. Polarization can set in because “when people talk to each other, they often find themselves at a more extreme point in accordance with their original inclinations.”
A more efficient approach is to look for a common or average result in several independent judgments. This may involve the use of “silent rules”, simple mechanical calculations of historical data to develop predictions about future behavior.
This leads to a surprising conclusion. The authors cite research based on the development of statistical models describing the judgment of individuals. The model surpasses the individual.
A humble conclusion is reached: “You may believe that you are more subtle, more insightful and more nuanced than the linear caricature of your thinking. But in fact, you are especially noisier.
Fortunately, the solution is not our robotic replacement. The concept of “decision hygiene” is introduced. This includes advice on how to organize information and develop guidelines for decision making.
The concept of hygiene illustrates the clinical tone of this book. The risk is that this will be done to the detriment of the diversity of thought and perspective which is at the heart of human action. Yes, consistency is a strength. but steering decisions towards a sterile central average can also create vulnerabilities.
Noise is dense text too – concepts like mediation assessment protocols should perhaps be left to a very carefree weekend. This is also a narrower topic than the earlier publications of Kahneman and Sunstein.
But that does not diminish the depth of wisdom. The authors carefully develop their proposition, leading the reader to complex conclusions. Persistence is definitely worth it – it is a great learning and communication achievement.
Judge Frankel, in his efforts to introduce judicial consistency, argued for the use of “computers as an aid to orderly thinking in sentencing.”
This book makes that prospect a little less likely.
Paschal Donohoe is the Minister of Finance and President of the Eurogroup