How Seriously should We Take Economic Policy Analysis 1?

The short answer is: not much. We can take the qualitative analysis quite seriously, but in many areas, the quantitative results are far from as trust-worthy as most would like you to believe.

There are many reasons, and I have intended to blog about this for a long time, but I will only discuss one issue briefly.

In economic analysis, we do something called welfare analysis---that is we find the utility function for individuals, and then we find their utility under different policies.

The problem lies in finding the utility function. We know there is no way utility function is parametrically identified, thus, we need to assume a functional form for the utility and for that functional form, we fit the data. Almost surely (that is with probability 1), we will be using the wrong functional form and the utility function we claim to identify is a very crude approximation within that functional form. As it turns out the welfare calculation could be very sensitive to that. There is one paper that finds that the welfare improvements is infinity!! The authors admirably kept the results and explained that it is due to their functional form assumption and continued with other functional forms. I could imagine in other studies authors would just delete crazy results from the paper and pretend nothing happened.

The problem continues. For welfare calculation, we want an "experience utility", that is how people actually feel. However, we can only back out what I call "decision utility", that is agents behave as if they are maximizing their "decision utility". Unless we assume that agents do maximize their experience utility, there is no reason to believe that these two would coincide. In fact, the study of behavioral economics point to many problems. Any inconsistency problems or cognitive bias would break the link, and they exist.

Let me give another example. We all know about business cycle--that is the boom and bust, recessions. We kind of feel there is a big cost to it, and it would be quite desirable if we can eliminate it. In fact, it is so desirable that we will be willing to pay an insurance fee to eliminate it or protect ourselves completely from it. But if we apply a standard welfare calculation, an upper estimate of the welfare cost of business cycle is 1/5 of 1 percent. An individual with average consumption of \$50,000 would be willing to pay \$100 to eliminate fluctuations. This is still a very small amount compared to the implications of long run growth on income. Most economists feel this is weird, but that is what is spit out. I think this does point to the limitation of our standard welfare analysis. Perhaps, some habbit formation adjustments would give more plausible answers.

Thus, the utility function we get needs to be taken with a grain of salt and so does the policy recommendation.

What does this imply?

It does NOT imply we should stop doing welfare analysis in economics. We need to and we need to expand the scope. But we need to be careful in interpreting our results and realize the error margin is larger than it looks. I always think of the wisdom of Ariel Pakes:"A decision needs to be made in real time. This is not perfect, but this is the best we can do." On the other hand, I would not shoot for any drastic policy changes that promises a slight/moderate welfare improvements on paper.