There are two ways of looking at this:
1. Statistics / numerical analysis / data science is a lot harder than you, the reader of this blog, might think.
2. Academia, like any other working environment, is full of prominent, successful, well-connected bluffers.
Someone sent me this:
and this followup:
I clicked around a bit, and there was a lot of appropriate distress this this sort of bad statistical analysis was being done at the highest levels of government.
But here I want to focus on something else, which is the general level of mediocrity, even at the top levels of academia.
When he is not “A/Chairman @WhiteHouseCEA,” the author of the above tweet is a tenured professor of public policy:
In his tweet, Philipson introduces the useful concept of “economist turned political hack.” My problem here is not whether Philipson is a hack but rather that someone can reach the heights of academia and government while being so ignorant as to think that this above curve fit could be a good idea.
OK, part of this is straight-up politics. There will always be prestigious and well-paying jobs for people who are willing to say the things that rich and powerful people want to hear.
But I think it’s more than that. Let me go back to points 1 and 2 above.
1. Statistics is hard. Sure, a decorated and tenured professor of public policy, “one of the world’s preeminent health economists.” Tyler Cowen says that graduate students in top econ programs all have “absolutely stellar” math GRE scores, and Philipson graduated from the University of Pennsylvania, which I think has a top program . . . But, again, statistics is hard. Really hard for some people. You could get a perfect math GRE score but not understand some basic principles of curve fitting.
Michael Jordan was a world champion basketball player but couldn’t hit the curve ball. So you’re surprised that someone who was good at taking multiple-choice math tests doesn’t understand statistics? C’mon.
Further evidence of the difficulty of statistics comes from Philipson’s economist colleagues from celebrated institutions around the world who famously flubbed a regression discontinuity analysis by massively overfitting—so, yes, the same sort of statistical error made by the Council of Economic Advisors and discussed above—in an analysis that was so bad that it motivated not one, but two papers (here and here) exploring what went wrong.
2. Academia, like any other working environment, is full of prominent, successful, well-connected bluffers. The striking thing is not that a decorated professor and A/Chairman @WhiteHouseCEA made a statistical error, nor should we be surprised that a prominent academic in economics (or any other field) doesn’t understand statistics. What’s striking is that the professor and A/Chairman doesn’t know that he doesn’t know. I’m struck by his ignorance of his ignorance, his willingness to think that he knows what he’s talking about when he doesn’t.
Some of this must be because bluffing is rewarded: it’s one way to advance in the world. If I wanted to be really cynical, I’d say that being not just wrong but aggressively wrong (as in the A/Chairman’s above tweet) sends a signal of commitment to the cause. I’m not quite sure what the cause is here, but maybe that’s part of the point too.
Again, this is not a problem that’s special to economists. I’m sure you’d see the same thing with sociologists, political scientists, psychologists, etc. Economists may be more of a concern because of their current position of power in government and industry, but the difficulty of statistics and the ease of bluffing is a more general issue.