Updates of bad forecasts: Let’s follow them up and see what happened!

People make bad forecasts, then they move on. Do the forecasts ever get fixed? Do experts learn from their mistakes?

Let’s look at three examples.

1. The economist who kept thinking that the Soviets were catching up

Paul Samuelson:


Yes, the above graph was from 1961, but “in subsequent editions Samuelson presented the same analysis again and again except the overtaking time was always pushed further into the future so by 1980 the dates were 2002 to 2012. In subsequent editions, Samuelson provided no acknowledgment of his past failure to predict and little commentary beyond remarks about ‘bad weather’ in the Soviet Union.” As late as 1989, the celebrated economist wrote, “The Soviet economy is proof that, contrary to what many skeptics had earlier believed, a socialist command economy can function and even thrive.”

This is not a left/right thing. Those of you who followed politics in the 1970s and the 1980s know that the supposed economic might of the Soviet Union was pushed by those on the left (who wanted us to imitate that system) and by those on the right (who feared we’d be overcome by it).

Anyway, for the purpose of today’s discussion, the point is that for twenty-eight years Samuelson neither corrected his error nor learned from it.

2. The contrarians who like to talk about global cooling

Steven Levitt and Stephen Dubner:

A Headline That Will Make Global-Warming Activists Apoplectic . . . It is curious that the global-warming arena is so rife with shrillness and ridicule. Where does this shrillness come from?

Yes, it was back in 2009 that the celebrated freakonomists promoted a claim that “The PDO cool mode has replaced the warm mode in the Pacific Ocean, virtually assuring us of about 30 years of global cooling”—but I don’t know that they’ve ever said a Whoops on that one, despite repeated news items such as the graph shown above.

Did Levitt and Dubner learn from this not to reflexively trust contrarian takes on important scientific issues? I don’t know. I did some searching and found this interview from 2015 where Levitt said,

I tell you what we were guilty of . . . We made fun of the environmentalists for getting upset about some other problem that turned out not to be true. But we didn’t do it with enough reverence, or enough shame and guilt. And I think we pointed out that it’s completely totally and actually much more religion than science. I mean what are you going to do about that? I think that’s just a fact.

So, as of 2015, they still seemed to have missed the point that you can learn from your mistakes.

3. Repeatedly biased forecasts from the U.S. Department of Transportation


The above graph was made in 2014.

So here’s my question: Has the Department of Transportation cleaned up their act? How are those projections going? I think the key problem is not the bad forecast, it’s the continuing to make the bad forecast even after it’s been destroyed by data.

We all make mistakes. The only way to avoid making mistakes is to do nothing. But we have the responsibility to learn from our mistakes. Science may be self-correcting, but it doesn’t self-correct on its own.