Learning from Experience: Employing Retrospective Review to Improve Regulatory Policy Joseph E. Aldy Harvard Kennedy School Administrative Conference of the United States Retrospective Review Workshop May 13, 2015

Obama Executive Orders EO 13563: Improving Regulation and Regulatory Review EO 13579: Regulation and Independent Regulatory Agencies EO 13610: Identifying and Reducing Regulatory Burdens

Implementing Executive Orders Agencies had 120 days to develop plans May 2011: preliminary plans issued Final plans issued August executive branch agency plans 580+ initiatives identified Resource savings in tens of billions $ Time savings in tens of millions of hours Focus on reducing burdens Regular reporting every six months

Findings of Obama Retrospective Review Culture of retrospective review Need for retrospective review guidance Role of politics Promoting multi-agency coordination Challenge of counterfactual

Executive Branch Major Rules Agency Major Rules Monetized Benefits and Costs Result of Retrospective Review Plan for Retrospective Review of Rule in the Future U.S. Department of Agriculture4000 Department of Energy4400 Department of Health and Human Services Department of Homeland Security 2110 Department of the Interior5000 Department of Justice1000 Department of Labor4110 Department of Transportation5*500 Environmental Protection Agency 8*700 Office of Personnel Management 1000 Total * DOT and EPA issued one joint rule in 2013.

Recommendations Value of Retrospective Review Integrating Retrospective Review into New Regulations Prioritizing Regulations for Retrospective Analysis Performing Retrospective Analysis

Recommendations Interagency Coordination Promoting Outside Input Resources

Retrospective Triage: Review vs. Analysis Regulatory look-back increases regulator responsibilities without new resources Identify rules that merit detailed ex post analysis In practice, less-intensive review focused on smaller rules Rule changes focused on reporting burdens

Planning for Retrospective Analysis in New Rules Design rules such that implementation produces data for ex post analysis -- benefits, costs, efficacy -- compliance, administration Provide opportunities for third party access to data for replication, extensions Design implementation of rules to facilitate causal inference

Research Design in Rule Development Randomized Experiments -- Lacko and Pappalardo 2010 Difference-in-Differences -- Henderson Greenstone et al Regression Discontinuity -- Berry and Lee Bennear and Olmstead 2008 Additional quasi-experimental methods

Culture of Retrospective Review Integrating retrospective review planning in the design of the rule can leverage – and inform – prospective review Potential synergies in regulatory review and broader agency performance assessments Evaluating the performance of regulatory policy provides the evidentiary basis for political and policy debates on regulations

Further Reading ACUS Plenary adopted recommendations in December Full working paper can be downloaded from my publications webpage