Friday, March 02, 2007
Why Even Bother
I worked for a time after completing my doctorate in "the real world"--a small brokerage house.
Advice here matters. You tell someone to invest in a certain mutual fund or stock--and if they lose money--they take their business to someone else. There is accountability for the course of action you prescribe.
Fund managers were also accountable. I would love reading the Morningstar reports which would give you the history of a fund's performance and assign a rating.
Shouldn't pundits, academics and policymakers be subjected to a similar Morningstar five-star rating system?
If we take your suggestions AEI should be shut-down.
CFR should be pruned (Luttwak and Boot ought not be there - for example.)
Brookings ought to fire Pollack and his boss over there.
A variation of this idea was discussed by Rumsfeld - he was not wrong on that.
The fact AEI is not shut down and CFR retains its position as the place for best and brightest shows the corruption of the system. Being right is not important; raising cash and appealing to prejudices is the way you get heard and get status.
There is a way to do almost the same thing, and that is to point out whose views published in The National Interest have been vindicated by events. You have done this informally from time to time here on TWR. You only need to set aside a small section in each issue of TNI to recognize authors whose views turn out to be prescient.
If you wanted to be ecumenical and were willing to do the extra work, you could keep track of authors in maybe half a dozen peer journals (and maybe the occasional writer outside the peer group).
Personally I would do this in a way that stresses who is right and not who is wrong. Those who write about policy and become known for being prescient will get more attention.
"I have been perplexed by the fact that so many smart and talented people with high IQs and advanced academic degrees from prestigious universities – have made so many wrong predictions about the future, have offered so much ill-advise for public policy, and generally have been proven wrong almost all of the time.
For an answer, consider a study by Philip Tetlock, a political scientist and psychologist at the University of California, Berkeley, who in Expert Political Judgment: How Good Is It? How Can We Know? (Princeton University Press, 2005) contends that there is no direct correlation between the intelligence and knowledge of the political expert and the quality of his or her forecasts.
Mr. Tetlock is very skeptical of our ability to predict future developments, as the odd assortment of path-dependency theorists, complexity-chaos theorists (recall the Butterfly Effect?), game theorists, and probability theorists demonstrate that the fundamental complex properties of the world make it impossible to achieve forecasting accuracy beyond crude extrapolation.
At the same time, the fundamental properties of the human mind – preference for simplicity, aversion of ambiguity and dissonance, belief in controllable world, misunderstandings of probabilistic processes – make it inevitable that experts will miss whatever predictability has not been precluded "in principle."
Mr. Tetlock has tried to prove, by gathering and analyzing more than 80,000 forecasts by academics, journalists, consultants and professional "futurists" about a variety of global issues, including the collapse of the Soviet Union, the end of the apartheid system in South Africa, and the outcome of the first Persian Gulf War (he explains the methodology he used in a long appendix to the study; which is rather complex itself) that the experts were wrong more often than blind chance.
What Mr. Tetlock considers to be an asset is the style of thinking, as there seems to be a direct link between how people think and what they get right and wrong. He applies in his study the prototypes of the "hedgehog" and the "fox" that the late British political philosopher Isaiah Berlin had proposed as a way of classifying political thinking and behavior.
Mr. Tetlock demonstrates the usefulness of classifying experts along a rough cognitive-style continuum anchored at one hand by Berlin's prototypical hedgehog and at the other by his prototypical fox.
The intellectually aggressive hedgehog knew one Big Thing and sought, under the banner of parsimony, to expand the explanatory power of that Big Thing to "cover" new cases. He or she toils devotedly within one tradition and imposes formulaic solutions on ill-defined problems.
The more eclectic foxes knew many little things and were contend to improvise ad hoc solutions to keep pace with a rapidly changing world. Drawing from a variety and sometimes contradictory array of ideas and traditions, he or she is better able to improvise in response to changing events and is more successful in predicting the future than the hedgehog."
Then examine the relationship of profiteers and promoters of, contributors to policies,...
As far as you are concerned with accountability,...
I am positive that profiteers will profit throughout different policies, showing that their predictions of being able to profit from policies were accurate, irrespective of the different contents of policies and the unfulfilled predictions regarding the content of those policies.
People who wanted to bomb Iraq but disliked Bush relied on Pollack to whisper sweet punditry in their ears. In his book "Threatening Storm" he wrote things that were quite unbelievable , even at the time. So if he wrote to be wrong, was he wrong? Or was he rightwrong?
This makes perfect sense to me, and is eminently credible. Predictions are a small part of a pundit's job. The largest part of it is the pushing of value-driven agendas, which is in turn driven by a desire for popularity. Greater status.
The utility of a rating system based on predictions is that it might increase the relevance of predictions to a pundit's overall reputation, and therefore decrease the relevance of value-driven agendas. This would be a victory for rational thought.
However, you might also see very bad predictors continue to thrive- sort of a corrolary here to stock markets, where the most optimistic pundits are the most successful, not the most accurate.
Just another story of the spleen triumphing over the brain.
Jordan W. 02