Unanswered Questions About Cost-Benefit Analysis

We have only fragmentary evidnece about how CBA actually functions in government decision-making.

Considering that people have been debating cost-benefit analysis at least since Reagan mandated its use in 1981, you would think we would have the answers to some basic questions about how it works.  Yet we have very fragmentary information, generally based on the perspevtives of people at the agencies or in the White House Office of Information and Regulatory Affairs (OIRA), which oversees agency regulatory efforts.  Part of the reason is the OIRA’s activities are lacking in transparency, but part of the reason is that the government hasn’t really wanted to find out the answers. In short, there may be nothing as important to the regulatory state about which we know so little. as the actual operation of CBA within the federal governmennt.  Here are some of the key unanswered questions.

How important is cost-benefit analysis to the regulatory process?

At one extreme, CBA might really drive results in a direction they would not otherwise have gone.  On the other hand, CBA might just serve to rationalize results that the agency or the White House wanted to reach anyway.  Given the CBA involves a fair number of economic judgment calls, it may be hard to be sure of this from just seeing the CBA itself. Surely, there are pressures to push the CBA on way or the other. But there are limits to how malleable the analysis really, so it may still help channel decisions.  There are also a lot of regulations that are said to have unquantifiable costs or benefits.  We don’t know how those are handled within the government.

How much of OIRA’s role really involves CBA?

Some of the complaints about CBA are really complaints about OIRA – that it delays regulations it doesn’t look, sometimes until they die, that it sacrifices congressional goals to achieve preferred out comes, and that it provides a channel for partisan pressure and special interest influence.  OIRA’s role actually involves a lot more than reviewing CBA.  It acts as a channel for other agencies to have input on regulations, in a highly opaque way.  It oversees the legal judgments made by agencies.  It ensures that political staff at the White House are aware of what agencies are doing. It tries to vet regulations to make sure that all procedural requirements were followed.  And it tries to coordinate activities between agencies working on the same issue.  Even if CBA went away, OIRA would still be powerful, and arguably the same complaints would continue.

Would cost-benefit analysis itself pass cost-benefit analysis?

The costs of CBA are the additional data and analysis it requires, which require staff time and resources. If a regulation’s benefits outweigh its costs, then every day of delay is a loss to society. Putting aside all the philosophical questions about whether our goal should also be to balance monetized benefits and costs, we also don’t know how effective cost-benefit analysis is in achieving this goal. Retrospective studies show that at least sometimes CBA has done as promised, but we don’t have systematic evidence.

What about other possible justifications for CBA?

One justification for CBA is that it helps improve uniformity across the executive branch. That’s certainly possible, but in fact we don’t know if it has actually happened.  For instance, even if agencies all use the same “statistical value of life” as a gauge of the cost of extra deaths, they may differ in their risk assessment methods or how they evaluate individuals studies.  A second justification is that CBA improves transparency.  The issue seems to be whether it does so for policymakers or journalists, as opposed to economists.  That depends on how well they can understand the analysis or process the final numbers.  This seems susceptible to experimental investigation using groups of lay people, but no one ever seems to have made an attempt to find this out. In particular, it would be useful to know whether, as some critics fear, quantifying part of the analysis makes people give too little weight to factors that haven’t been (and maybe can’t be) quantified.

So what’s the bottom line?

As I said at the beginning, these are unanswered questions.  Empirical research might be able to shed light on some of them, but any real progress might require cooperation from OIRA itself.  There might also be room for some empirical work at the state level in those states that do make broad use of cost-benefit analysis.

Without clearer evidence, we all have to do the best we can with what evidence we have.  Given the incomplete nature of that evidence, unfortunately that means that our conclusions are likely to be driven by our background assumptions about how the regulatory state actually works.  That means that, in the absence of new evidence, the debate is unlikely to change anyone’s mind.

Cost Benefit Analysis, economic analysis, government regulation, Office of Management and Budget, OIRA, presidential power, presidential powers, regulatory state