The Challenge Network www.chforum.org |
go back |
Summary: two distinctive 'thinking styles' characterise expert views of future-oriented and complex problems. A new book shows that one of these is markedly superior to the other in predicting events and adapting to new information. Unhappily, the superior approach is less able to capture the high ground of debate, suggesting that active measures may be needed to make space for it.
Philip Tetlock has published "Expert Political Judgement" (PUP 2006), in which he shows the idiosyncrasy and fallibility of 'expert' judgements about future events.
The book is complex and subtle, but it is fair to say that it offers two forms of analysis. First, it describes the characteristic ways in which such judgement tends to be framed - described as 'cognitive styles' - and the personality types which are most prone to make use of these. Second, it assesses the effectiveness of these modes of judgement and ways of thinking. The research points to stark differences in the effectiveness of the two styles.
Isaiah Berlin once drew on European folk tales for the metaphor of the fox and the hedgehog. Hedgehogs, he said, have just one, powerful response to a threat: they roll themselves into a ball, presenting spikes to predators (and to cars.) They 'know just one big thing'. Foxes, by contrast, have no single response to challenges, for they 'know many little things'. They react to challenge by drawing on a pattern of general, pragmatic understanding, often making mistakes but seldom committing themselves to a potentially catastrophic grand strategy. Berlin was concerned with the roles of the expert and the generalist, and Tetlock has used this metaphor to name the two ends of the dimension which he has extracted from his research.
People who rely upon the Hedgehog cognitive style need closure - a sense of finality, of "that's settled, then" - in order to feel happy. That is, they need an unambiguous model to support their decision-taking, and the data against which to calibrate this. They like their model to be actually simple and conceptually parsimonious, decisive - that is, delivering a binary verdict, not a balance of probabilities - and repeatable. Hedgehog experts have a tendency to reach for formulaic solutions, for precedent and for the approbation of their peers, and to resent and resist challenge to their model. They prefer to capture a sub-set of the problem in a tractable form than to reach for a less precise, but perhaps more comprehensive, overview of the issues that are involved.
Experts who think in the 'Fox' cognitive style are suspicious of a commitment to any one way of seeing the issue, and prefer a loose insight that is nonetheless calibrated from many different perspectives. They use quantification of uncertain events more as calibration, as a metaphor, than as a prediction. They are tolerant of dissonance within a model - for example, that an 'enemy' regime might have redeeming qualities - and relatively ready to recalibrate their view when unexpected events cast doubt on what they had previously believed to be true.
In contrast to this, Hedgehogs work hard to exclude dissonance from their models. They prefer to treat events which contradict their expectations as exceptions, and to re-interpret events in such a way as to allocate exceptions to external events. For example, positive aspects of an enemy regime may be assigned to propaganda, either on the part of the regime or through its sympathisers. Tetlock makes the point that this is neither an exclusive characteristic of the political Left or the Right, but a feature of the Hedgehog ideologues within both.
Hedgehogs tend to flourish and excel in environments in which uncertainty and ambiguity have been excluded, either by actual or artificial means. The mantra of "targets and accountability" was made by and for Hedgehogs. Foxes, by contrast, use a style which works best where neither the interpretation of the operating environment nor the correct nature of or balance amongst targets is clear.
Tetlock collected data from nearly 300 experts in fields such as international affairs, economics, commerce, law and public policy over a five year period. These were asked to make assessments on a wide range of subjects, from the presence of absence of weapons of mass destruction in Iraq to the prospects for the Internet bubble. The quality of these projections were assessed after the event. The experts had been classified on the "Fox to Hedgehog" scale, making it possible to assess which of the two cognitive styles gave the best approach to making judgements about uncertain, complex events. The quality was assessed both as an ex ante forecast and as an adaptive commentator, absorbing and processing information as it became available.
It is, perhaps, a particular irony that the worst judges turned out to be Hedgehog experts holding forth on their own particular area of expertise. This group were particularly prone to overstate the likelihood of extreme changes: wars, financial crises and the like. They tended to calibrate their position against the peer group, creating "friends" and "foe" and over-stating their difference from their supposed enemies. They exhibited false memories, claiming to have said or believed things which they had not, in fact, espoused earlier. In general, Hedgehogs were less likely to re-evaluate their views - and in particular, extreme views - when these were challenged by new information.
Foxes were more commonly correct in their forecasts, and much more open to re-calibration of their views in the light of new information. Their media coverage was poor, however, because their statements were nuanced and took some effort to understand. They were less inclined to exclude extreme events - perhaps less focused on them - than were Hedgehogs, who typically received stronger media coverage and who were ready to make such exclusions. (This was the one area in which Hedgehogs performed better than foxes.)
One conclusion to be drawn from this analysis is, therefore, that we live in a world that needs a Foxy outlook, but that we tend to be faced with messages that derive from Hedgehogs, from Hedgehog-friendly criteria for commercial and public policy, and that these messages have the virtue of simplicity and closure. The problem for the Fox perspective is that it does not come with a neat, closed model, with defined goals and easy metrics. It is ambiguous, complex, open-ended and often self-referential, requiring a gradual iteration between tools and goals in order to find a satisfactory way forward. This puts it at a disadvantage, notably when faced with high-profile issues, urgency and governance which is so embedded in the Hedgehog style that it cannot (or will not) make the time needed to entertain new thoughts.
However - and it is an immense, overwhelming 'however' in the right circumstances - the Fox style tends to be the right one to use when the world has become too complex for the Hedgehogs. Rolling in a ball will not do when cars threaten. The Fox approach is demonstrated to give more right answers. It responds to new information. It creates the conditions in which future Hedgehogs can thrive. This symbiosis between the two styles, once so common in commerce and in the policy process, has all be disappeared from great swaths of public life and from whole industries. It may be that the knowledge economy demands the renaissance of the fox.
To top |