1. Learning from Experience: Employing Retrospective Review to Improve Regulatory Policy Joseph E. Aldy Harvard Kennedy School Administrative Conference of the United States Retrospective Review Workshop May 13, 2015

2. Obama Executive Orders EO 13563: Improving Regulation and Regulatory Review EO 13579: Regulation and Independent Regulatory Agencies EO 13610: Identifying and Reducing Regulatory Burdens

3. Implementing Executive Orders Agencies had 120 days to develop plans May 2011: preliminary plans issued Final plans issued August 2011 26 executive branch agency plans 580+ initiatives identified Resource savings in tens of billions $ Time savings in tens of millions of hours Focus on reducing burdens Regular reporting every six months

4. Findings of Obama Retrospective Review Culture of retrospective review Need for retrospective review guidance Role of politics Promoting multi-agency coordination Challenge of counterfactual

5. Executive Branch Major Rules 2013-14 Agency Major Rules Monetized Benefits and Costs Result of Retrospective Review Plan for Retrospective Review of Rule in the Future U.S. Department of Agriculture4000 Department of Energy4400 Department of Health and Human Services 16430 Department of Homeland Security 2110 Department of the Interior5000 Department of Justice1000 Department of Labor4110 Department of Transportation5*500 Environmental Protection Agency 8*700 Office of Personnel Management 1000 Total492250 * DOT and EPA issued one joint rule in 2013.

6. Recommendations Value of Retrospective Review Integrating Retrospective Review into New Regulations Prioritizing Regulations for Retrospective Analysis Performing Retrospective Analysis

7. Recommendations Interagency Coordination Promoting Outside Input Resources

8. Retrospective Triage: Review vs. Analysis Regulatory look-back increases regulator responsibilities without new resources Identify rules that merit detailed ex post analysis In practice, less-intensive review focused on smaller rules Rule changes focused on reporting burdens

9. Planning for Retrospective Analysis in New Rules Design rules such that implementation produces data for ex post analysis -- benefits, costs, efficacy -- compliance, administration Provide opportunities for third party access to data for replication, extensions Design implementation of rules to facilitate causal inference

10. Research Design in Rule Development Randomized Experiments -- Lacko and Pappalardo 2010 Difference-in-Differences -- Henderson 1996 -- Greenstone et al. 2006 Regression Discontinuity -- Berry and Lee 2007 -- Bennear and Olmstead 2008 Additional quasi-experimental methods

11. Culture of Retrospective Review Integrating retrospective review planning in the design of the rule can leverage – and inform – prospective review Potential synergies in regulatory review and broader agency performance assessments Evaluating the performance of regulatory policy provides the evidentiary basis for political and policy debates on regulations

12. Further Reading ACUS Plenary adopted recommendations in December 2014 www.acus.gov/research-projects/retrospective-review-agency-rules Full working paper can be downloaded from my publications webpage http://www.hks.harvard.edu/fs/jaldy/Library/publications.html http://www.hks.harvard.edu/fs/jaldy/Library/publications.html