RichardHarlos.com

Between 1985 and 2005, Philip E. Tetlock collected thousands of expert opinions about the fate of dozens of countries. After the fact, these hundreds of experts and their opinions were evaluated for accuracy. How did they score?

Overall, the experts of positive opinions gave their predictions a 65% probability of occurring, while the negative opinion experts scored their opinions as 70% likely to occur. In fact, the ‘rosy’ opinions came true about 15% of the time and the ‘gloomy’ opinions only occurred 12% of the time. The complete details of Tetlock’s 20-year research project are in his book Expert Political Judgment: How Good is It? How Can We Know? (note: this is NOT an affiliate link; if I were going to use one, I would inform you in advance).

An interesting distinction Tetlock draws among those who gave their expert opinion is to refer to them as either “hedgehogs” or “foxes. What’s the difference? In an online commentary titled How Accurate Are Your Pet Pundits?, Tetlock writes:

We can see this process in sharp relief when, following the philosopher Sir Isaiah Berlin, we classify experts as “hedgehogs” or “foxes.” Hedgehogs are big-idea thinkers in love with grand theories: libertarianism, Marxism, environmentalism, etc. Their self-confidence can be infectious. They know how to stoke momentum in an argument by multiplying reasons why they are right and others are wrong.

Tetlock tells us that this ability to “stoke momentum” wins them high visibility in the media despite their tendency to make more mistakes. More mistakes than who, you ask? Why, more mistakes than the fox, of course! (Hedgehogs, Tetlock tells us, are the kind of experts most often found when one turns to Google for answers; Googlers beware!)

In contrast with our media-loving hedgehogs are the foxes. In the same online commentary, Tetlock characterizes the fox this way:

Eclectic foxes are better at curbing their ideological enthusiasms. They are comfortable with protracted uncertainty about who is right even in bitter debates, conceding gaps in their knowledge and granting legitimacy to opposing views. They sprinkle their conversations with linguistic qualifiers that limit the reach of their arguments: ‘but,’ ‘however,’ ‘although.’

I’ll bet you, like me, know your share of foxes, too.

Tetlock tells us that while foxes make fewer mistakes in forming their opinions, the very qualities that fuel an increased accuracy also tend to make them less popular than hedgehogs when it comes to making big headline stories in the media.

So it seems that Tetlock’s research favors the notion that the biggest stories in the biggest media conglomerates are more likely to be of the hedgehog variety: controversial, marketable and, unfortunately, less likely to be correct over time.

As for me, I’ll continue to search out the foxes in less-than-mainstream media because in the end, if I must choose, I’d rather read news that’s accurate, true, correct, right, etc. than to read news that’s merely marketable and popular.