On June 13th, the Environmental Protection Agency (EPA) issued an advanced notice of proposed rulemaking soliciting comments on how to improve the consistency and transparency of the cost benefit analyses that guide EPA’s regulatory decision making. Both are praiseworthy goals, particularly because executive orders issued by the Trump administration last year resulted in cost benefit analysis being used not only for determining the appropriateness of individual regulations, but also for choosing between regulations when deciding which to implement. In its solicitation for comments, EPA has hinted at a key part of the answer: retrospective reviews.
Many of the questions on which EPA is seeking feedback are in the weeds of policy analysis, such as how to grapple with the realities of inadequate data, model limitations, and uncertainty about how to weight various factors. As will always be the case, implicit or explicit decisions will be made to address those questions. The broader challenge is creating a system that continually learns and tweaks those decisions to reflect changes in knowledge, methods, and the state of the world. EPA’s interest in including a “systematic retrospective review element” in new regulation has the potential to provide a transparent and well-structured method for assessing which decisions worked well and which didn’t. If EPA can successfully implement retrospective reviews, it could serve as a role model for other regulatory agencies.
EPA asks a variety of good questions about how to set up such a retrospective review system: What criteria will determine when review is needed? How specific should the requirement be in terms of methodology, scope, and data? Beyond these details of the analysis itself, critical requirements that could contribute to success include requiring the reviewers to be independent and unbiased, insuring that the reviews directly influence future analyses, and developing a culture within EPA that values and rewards retrospective, high-quality reviews. None of this will be easy to accomplish.
Regulations affect what can and can’t be done at every step in the production of every product purchased or service hired. For that reason, there are often a lot of vested interests and monies riding on the outcomes of regulatory decisions. EPA should avoid reviewing its own regulations, and experts supported directly or indirectly by organizations impacted by the regulation being reviewed could also be excluded. This could create challenges for finding reviewers in the case of particularly broad regulations, but this is generally a surmountable problem. The independence of reviewers could prove critical in demonstrating that reviews are unbiased in both perception and reality.
In addition to being unbiased, the impact of reviews also could be considered. An unbiased review that simply sits on a shelf gathering dust is a waste of everyone’s time and money. Instead, the review process could carry real weight in influencing both the regulation in question as well as the broader regulatory analysis process used by EPA. To start, reviews could be made publicly available, without redaction, along with the required response from EPA. Ideally, a third party arbiter might assess whether EPA has sufficiently addressed the reviewer’s concerns, and could be given the authority to mandate further action if EPA’s response is deemed insufficient.
Finally, and perhaps most importantly, steps could be taken to create a culture that rewards high-quality reviews. The suggestions made above could easily cause retrospective reviews to be viewed as extra work and new headaches. Career regulators are already busy evaluating the impact of new policies, and no one likes having old work dug up to see what could have been done better.
Without proper incentives, retrospective reviews might be avoided or marginalized by those tasked with implementing them. To avoid this outcome, incentives could be introduced so EPA’s individual career regulators and managers directly and personally benefit from seeing through high-quality and impactful reviews. Rather than treating reviews as opportunities for punishing under-performance or errors, they could come to be treated as opportunities for rewarding career analysts for having produced work that reviews found to be high-quality, and for rewarding managers for shepherding though reviews that find opportunities for improvement. For example, the Society for Benefit Cost Analysis might hold an annual competition for recognition awards, or EPA staff might be rewarded with time to pursue novel research ideas or growth opportunities. Indeed, at certain career levels, successful implementation of retrospective reviews could be an advancement criterion.
These elements will not easily fall into place on their own, but could be well worth the effort. Retrospective reviews could help the EPA better protect the public interest by improving the consistency and transparency of its regulatory analysis.
Benjamin Miller is an associate economist at the RAND Corporation and a Professor at the Pardee RAND Graduate School.