Year: 2021

Note: Apologies to Phil Tetlock if he doesn’t want to be associated with the Taliban, I just couldn’t resist the alliteration. Also apologies to the Taliban if they don’t want to be associated with an American academic, though I assure them that Phil is one of the good ones.

Imagine that the US was competing in a space race with some third world country, say Zambia, for whatever reason. Americans of course would have orders of magnitude more money to throw at the problem, and the most respected aerospace engineers in the world, with degrees from the best universities and publications in the top journals. Zambia would have none of this. What should our reaction be if, after a decade, Zambia had made more progress?

Obviously, it would call into question the entire field of aerospace engineering. What good were all those Google Scholar pages filled with thousands of citations, all the knowledge gained from our labs and universities, if Western science gets outcompeted by the third world?

For all that has been said about Afghanistan, no one has noticed that this is precisely what just happened to political science. The American-led coalition had countless experts with backgrounds pertaining to every part of the mission on their side: people who had done their dissertations on topics like state building, terrorism, military-civilian relations, and gender in the military. General David Petraeus, who helped sell Obama on the troop surge that made everything in Afghanistan worse, earned a PhD from Princeton and was supposedly an expert in “counterinsurgency theory.” Ashraf Ghani, the just deposed president of the country, has a PhD in anthropology from Columbia and is the co-author of a book literally called Fixing Failed States. This was his territory. It’s as if Wernher von Braun had been given all the resources in the world to run a space program and had been beaten to the moon by an African witch doctor.

Meanwhile, the Taliban did not have a Western PhD among them. Their leadership was highly selected though. As Ahmed Rashid notes in his book The Taliban, in February 1999, the school that provided the leadership for the movement “had a staggering 15,000 applicants for some 400 new places making it the most popular madrassa in northern Pakistan.” Yet they certainly didn’t publish in or read the top political science journals. Consider this a data point in the question of whether intelligence or subject-matter expertise is more important.

Is the moon shot analogy fair? I think it probably strikes many people as odd, but I don’t see why it should. Surely, there were many political scientists who thought what the US was trying to do in Afghanistan given the resources invested was impossible, me among them, and maybe it’s simply the “experts” who were hired by NGOs, think tanks, and the US government that were delusional.

Yet I wonder what the field of civil engineering would say if the US went abroad and tried to build bridges based on principles that violated the laws of physics. I’d like to think the Pentagon would have trouble finding well-credentialed experts to help them, and those that did take a paycheck to help achieve the impossible would lose all credibility in their field. That of course has not happened to the pundits and social scientists who spent 20 years making a living off the idea that the US was doing something reasonable in Afghanistan.

Tetlock’s Discovery

Phil Tetlock’s work on experts is one of those things that gets a lot of attention, but still manages to be underrated. In his 2005 Expert Political Judgment: How Good Is It? How Can We Know?, he found that the forecasting abilities of subject-matter experts were no better than educated laymen when it came to predicting geopolitical events and economic outcomes. As Bryan Caplan points out, we shouldn’t exaggerate the results here and provide too much fodder for populists; the questions asked were chosen for their difficulty, and the experts were being compared to laymen who nonetheless had met some threshold of education and competence.

At the same time, we shouldn’t put too little emphasis on the results either. They show that “expertise” as we understand it is largely fake. Should you listen to epidemiologists or economists when it comes to COVID-19? Conventional wisdom says “trust the experts.” The lesson of Tetlock (and the Afghanistan War), is that while you certainly shouldn’t be getting all your information from your uncle’s Facebook Wall, there is no reason to start with a strong prior that people with medical degrees know more than any intelligent person who honestly looks at the available data.

I have a PhD in political science with a focus on international relations. Most people in my position would tell you that you should give my opinions on my topic of expertise more weight because of my credentials. I believe if anything, you should hold my degree against me, as getting a PhD is probably the most inefficient way to understand a topic, and a person seeking that credential has shown that they don’t understand that. I think I’ve been right on Afghanistan and other American interventions because of good intellectual habits, including a genuine concern with what is true. But that has little to do with any training I got from political science.

Continue reading…

From Richard Hanania’s Newsletter, here.