Regulatory Field Experiments?

I really like Jonathan’s post about MIT’s J-PAL.  Permit me to offer a few points.

1.  The field experiment economists are randomly allocating stuff at the individual level.  Your household might receive a free newspaper, a report indicating how your electricity consumption differs from neighbors, a report indicating how your politician is performing on a series of criteria, a free bed net to fight malaria risk, a free bus ticket so that you can migrate to a big city,  admittance to an elite school for which you were on the waiting list.   There are academic papers that have been written about each of the topics I just listed.

2. In the case of regulation, we regulate a geographical location and the treatment is long term.  For example, Los Angeles was not randomly assigned to non-attainment status for the Clean Air Act for 1 day. It is assigned to this category until the air quality improves.  In a randomized control study, there are comparable individuals who are not treated and they represent the control group. The challenge for empiricists in the case of evaluating regulation’s consequences is that the regulation is not randomly assigned (would environmental lawyers who seek to improve pollution in the most highly polluted populated areas support randomization?) and there aren’t obvious control groups to be used for judging whether regulation is effective.

Funny reverse causality issues arise in regulatory evaluation studies.  Just as cities with more cops have more crime (so do cops reduce crime?), those cities with worse pollution have more regulation.  So, does regulation raise pollution or do those cities with more pollution have more regulation?  A randomization of regulatory stringency would allow us statisticians to make progress but would regulators willingly give up their power so that the applied evaluators could be happy?

3.  If the regulated firms know that they are participating in an experiment then a Hawthorne effect will ensue.  If firms anticipate that the experiment will last 3 months then they may just wait it out or act “nice” during that time.  If they expect the regulation in their area is permanent they will be more likely to take an action such as moving to a less regulated area (a domestic pollution haven) to reduce their regulatory burden.  Expectations matter!

4.  For a regulatory agency to run a field experiment, it must admit to itself and the public the ugly fact that “you know that you don’t know” the answers.  If you already know what is the right regulatory strategy, why would you run such an experiment.  I want more experimentation under AB32 but I worry that regulators feel that such pilots reveals “weakness” that they are not omniscient.  I am a modest man.  I know that there is a lot I don’t know. Field experiments offer a credible strategy for resolving many “chicken and egg” issues.  I continue to partner with some electric utilities who are willing to pilot energy savings programs and I am hoping that AB32 nudges them to experiment more.