What do Paul Samuelson and the U.S. Department of Transportation have in common?
Phil Price points us to this news article by Clark Williams-Derry:
As the State Smart Transportation Initiative at the University of Wisconsin points out, the US Department of Transportation has been making the virtually identical vehicle travel forecasts for well over a decade. All of those forecasts project rapid and incessant growth in vehicle travel for as far as the eye can see. Meanwhile, actual traffic volumes have flattened out, and may actually be falling.
Each of the rising colored lines represents a forecast from a different year. The black line represents actual traffic trends on US roads—which never rose as quickly as the forecasters had predicted, and actually started a modest decline in 2007.
I’d like to see a label on the y-axis, and I’d recommend labeling the x-axis at 5-year intervals rather than every year, but the point seems pretty clear.
In this entirely USDOT’s fault, though. These forecasts are a “roll-up” of forecasts made by state DOTs. The US agency just collects the forecasts and reports them to the public: garbage in, garbage out.
But in a way, that’s even more sobering than if the fault were localized in USDOT, since it provides clear and compelling evidence that the nation’s entire transportation forecasting apparatus is completely broken. In the aggregate, all of those hard working forecasters in all of those state DOTs are just making up numbers.
Indeed. I’m reminded of this notorious graph from the legendary Paul Samuelson textbook (from 1961):
As Alex Tabarrok points out, the graph is even worse than it looks: “in subsequent editions Samuelson presented the same analysis again and again except the overtaking time was always pushed further into the future so by 1980 the dates were 2002 to 2012. In subsequent editions, Samuelson provided no acknowledgment of his past failure to predict and little commentary beyond remarks about ‘bad weather’ in the Soviet Union.”
The bit about the bad weather is funny. If you’ve had bad weather in the past, maybe the possibility of future bad weather should be incorporated into the forecast, no?
As Tabarrok and his commenters point out, this mistake can’t simply be attributed to socialist sympathies of the center-left Samuelson: For one thing, various other leftist economists did not think that the Soviets were catching up to us; and, for another, political commentators on the right at the time were all telling us that the communists were about to overwhelm us militarily.
As with the Commissar for Traffic, the problem was the linear model, the assumption that what has changed in the past will continue to change in the same way in the future.
What’s unforgivable, both in the case of Samuelson in the 1960s and the Commissars today, is not in making the mistake (surely, a linear extrapolation is quite reasonable in many settings) but in not recognizing the problem, even after the forecast has been completely destroyed by the data.