At Econbrowser, Dr. Menzie Chinn provides a succinct summary of my critique of CEA’s “Economic Impact of the Stimulus” report when he writes, “…these implied increments to growth rates do not jibe with the inferences drawn by Dr. Campbell — that the impact on GDP is much smaller than CEA asserts when using forecasts from the other agencies and firms.” (italics mine).
My critique was that the CEA’s method for estimating the economic impact of the American Recovery and Reinvestment Act (ARRA) cannot be used to make a meaningful inference about the economic impact of ARRA quantitatively (or qualitatively).
The CEA admits, as does Dr. Chinn, that numerous other events and actions were taking place that could explain the difference between the forecast and actual data. The economic analysis undertaken by the CEA either needed to use econometric tools to separate out and isolate the ARRA effect in the difference between the forecast and actual or they needed to run an impulse response of ARRA on their VAR model (a counterfactual analysis). That is, if they were going to use a VAR approach to study the impact of ARRA they needed to establish the baseline and then introduce ARRA to that same VAR model. For example, Christine Romer and David Romer used such an approach in their paper “The Macroeconomic Effects Of Tax Changes: Estimates Based On A New Measure Of Fiscal Shocks”.
My point is simple and hardly “difficult to understand”: the Administration approach did not, in fact, estimate the impact of ARRA.
The CEA just subtracted actual quarterly results for 2009 from the forecasted quarterly results. That’s a bad analytical move for at least two reasons: (1) Any number of things could explain the difference between a forecast and actual results. It is the job of the economist to separate out these effects in a statistically meaningful way. (2) The fact that using different forecasts provides different quantitative results shows that this methodology is inadequate for estimating the impact of ARRA.
Dr. Chinn’s criticism of picking forecasts before the stimulus was enacted is moot since I am not trying to show the impact of ARRA but rather the inadequacy of simply producing a forecast and subtracting the difference. However, the choice of forecast was intentional. If the forecast was not made before the stimulus bill passed, then the forecast would have included the impact of the stimulus and thus could not be used as a “counterfactual” of what would have happened without the stimulus.