The (Ex) Washington Realist

Napoleon promoted his generals on basis of how lucky they were.

If we take your suggestions AEI should be shut-down.

CFR should be pruned (Luttwak and Boot ought not be there - for example.)

Brookings ought to fire Pollack and his boss over there.
 

Another way of approaching this is to create an electronic "market" in which individuals would "buy" into these policy recommendations. Sort of like a rating system that is tied to performance. However, in this case, the performance will be judged by the "market" - e.g. subscribers to the Think Tank journals, guest speakers, former government officials (Foreign & Domestic).

A variation of this idea was discussed by Rumsfeld - he was not wrong on that.
 

Anonymous 10:20

The fact AEI is not shut down and CFR retains its position as the place for best and brightest shows the corruption of the system. Being right is not important; raising cash and appealing to prejudices is the way you get heard and get status.
 

Nikolas,

There is a way to do almost the same thing, and that is to point out whose views published in The National Interest have been vindicated by events. You have done this informally from time to time here on TWR. You only need to set aside a small section in each issue of TNI to recognize authors whose views turn out to be prescient.

If you wanted to be ecumenical and were willing to do the extra work, you could keep track of authors in maybe half a dozen peer journals (and maybe the occasional writer outside the peer group).

Personally I would do this in a way that stresses who is right and not who is wrong. Those who write about policy and become known for being prescient will get more attention.
 

I just remembered you are bimonthly now. I think it would be enough to do the roundup of who was prescient maybe once a year as a feature article by the editors.
 

David--that's a great idea--something to be considered.

Nick
 

The Middle East Policy Council has a brief on Iraq that chronicles the successful prognostications of its contributors on Iraq.
 

But... whether or not a Pundit is correct in his predictions or not is irrelevant, for a record of correct prediction is not a requirement of the Punditocracy. The purpose of the Punditocracy is to advocate a maximally belligerent US foreign policy and to label all advocates of a less aggressive policy as "appeasers". And this is nothing new. Don't you recall the Pundits who called Gorbachev "The most dangerous Soviet leader since Lenin"? Nobody cares that they got Gorby all wrong, and nobody will care about the Pundits who got Iraq all wrong.
 

Leon Hadar:

"I have been perplexed by the fact that so many smart and talented people with high IQs and advanced academic degrees from prestigious universities – have made so many wrong predictions about the future, have offered so much ill-advise for public policy, and generally have been proven wrong almost all of the time.

For an answer, consider a study by Philip Tetlock, a political scientist and psychologist at the University of California, Berkeley, who in Expert Political Judgment: How Good Is It? How Can We Know? (Princeton University Press, 2005) contends that there is no direct correlation between the intelligence and knowledge of the political expert and the quality of his or her forecasts.

Mr. Tetlock is very skeptical of our ability to predict future developments, as the odd assortment of path-dependency theorists, complexity-chaos theorists (recall the Butterfly Effect?), game theorists, and probability theorists demonstrate that the fundamental complex properties of the world make it impossible to achieve forecasting accuracy beyond crude extrapolation.

At the same time, the fundamental properties of the human mind – preference for simplicity, aversion of ambiguity and dissonance, belief in controllable world, misunderstandings of probabilistic processes – make it inevitable that experts will miss whatever predictability has not been precluded "in principle."

Mr. Tetlock has tried to prove, by gathering and analyzing more than 80,000 forecasts by academics, journalists, consultants and professional "futurists" about a variety of global issues, including the collapse of the Soviet Union, the end of the apartheid system in South Africa, and the outcome of the first Persian Gulf War (he explains the methodology he used in a long appendix to the study; which is rather complex itself) that the experts were wrong more often than blind chance.

What Mr. Tetlock considers to be an asset is the style of thinking, as there seems to be a direct link between how people think and what they get right and wrong. He applies in his study the prototypes of the "hedgehog" and the "fox" that the late British political philosopher Isaiah Berlin had proposed as a way of classifying political thinking and behavior.

Mr. Tetlock demonstrates the usefulness of classifying experts along a rough cognitive-style continuum anchored at one hand by Berlin's prototypical hedgehog and at the other by his prototypical fox.

The intellectually aggressive hedgehog knew one Big Thing and sought, under the banner of parsimony, to expand the explanatory power of that Big Thing to "cover" new cases. He or she toils devotedly within one tradition and imposes formulaic solutions on ill-defined problems.

The more eclectic foxes knew many little things and were contend to improvise ad hoc solutions to keep pace with a rapidly changing world. Drawing from a variety and sometimes contradictory array of ideas and traditions, he or she is better able to improvise in response to changing events and is more successful in predicting the future than the hedgehog."
 

I think it is more important to analyze and proof who profits from the implementation of certain policies, irrespective of the content and predictions...

Then examine the relationship of profiteers and promoters of, contributors to policies,...

As far as you are concerned with accountability,...

I am positive that profiteers will profit throughout different policies, showing that their predictions of being able to profit from policies were accurate, irrespective of the different contents of policies and the unfulfilled predictions regarding the content of those policies.
 

Pollack was not wrong - he was wrongright , which is a unique pundit category - His job was to be wrong, so his wrongness would be influential. That was his job - and he did his job well.
People who wanted to bomb Iraq but disliked Bush relied on Pollack to whisper sweet punditry in their ears. In his book "Threatening Storm" he wrote things that were quite unbelievable , even at the time. So if he wrote to be wrong, was he wrong? Or was he rightwrong?
 

Anonymous 6:52:

What a great category: "wrongright" !
 

For an answer, consider a study by Philip Tetlock, a political scientist and psychologist at the University of California, Berkeley, who in Expert Political Judgment: How Good Is It? How Can We Know? (Princeton University Press, 2005) contends that there is no direct correlation between the intelligence and knowledge of the political expert and the quality of his or her forecasts.

This makes perfect sense to me, and is eminently credible. Predictions are a small part of a pundit's job. The largest part of it is the pushing of value-driven agendas, which is in turn driven by a desire for popularity. Greater status.

The utility of a rating system based on predictions is that it might increase the relevance of predictions to a pundit's overall reputation, and therefore decrease the relevance of value-driven agendas. This would be a victory for rational thought.

However, you might also see very bad predictors continue to thrive- sort of a corrolary here to stock markets, where the most optimistic pundits are the most successful, not the most accurate.

Just another story of the spleen triumphing over the brain.

Jordan W. 02