Learning from Experience: Employing Retrospective Review to Improve Regulatory Policy

Joseph E. AldyHarvard Kennedy School

Administrative Conference of the United States Retrospective Review Workshop

May 13, 2015

Obama Executive Orders

EO 13563: Improving Regulation and Regulatory Review

EO 13579: Regulation and Independent Regulatory Agencies

EO 13610: Identifying and Reducing Regulatory Burdens

Implementing Executive Orders

Agencies had 120 days to develop plansMay 2011: preliminary plans

issued

Final plans issued August 201126 executive branch agency plans580+ initiatives identifiedResource savings in tens of billions

$Time savings in tens of millions of

hoursFocus on reducing burdens

Regular reporting every six months

Findings of Obama Retrospective Review

Culture of retrospective review

Need for retrospective review guidance

Role of politics

Promoting multi-agency coordination

Challenge of counterfactual

Executive Branch Major Rules 2013-14Agency Major

RulesMonetized Benefits and Costs

Result of Retrospective Review

Plan for Retrospective Review of Rule in the Future

U.S. Department of Agriculture 4 0 0 0

Department of Energy 4 4 0 0

Department of Health and Human Services

16 4 3 0

Department of Homeland Security

2 1 1 0

Department of the Interior 5 0 0 0

Department of Justice 1 0 0 0

Department of Labor 4 1 1 0

Department of Transportation 5* 5 0 0

Environmental Protection Agency

8* 7 0 0

Office of Personnel Management

1 0 0 0

Total 49 22 5 0

* DOT and EPA issued one joint rule in 2013.

Recommendations

Value of Retrospective Review

Integrating Retrospective Review into New Regulations

Prioritizing Regulations for Retrospective Analysis

Performing Retrospective Analysis

Recommendations

Interagency Coordination

Promoting Outside Input

Resources

Retrospective Triage: Review vs. Analysis

Regulatory look-back increases regulator responsibilities without new resources

Identify rules that merit detailed ex post analysis

In practice, less-intensive review focused on smaller rules

Rule changes focused on reporting burdens

Planning for Retrospective Analysis in New Rules

Design rules such that implementation produces data for ex post analysis

-- benefits, costs, efficacy-- compliance, administration

Provide opportunities for third party access to data for replication, extensions

Design implementation of rules to facilitate causal inference

Research Design in Rule Development

Randomized Experiments-- Lacko and Pappalardo 2010

Difference-in-Differences-- Henderson 1996-- Greenstone et al. 2006

Regression Discontinuity-- Berry and Lee 2007-- Bennear and Olmstead 2008

Additional quasi-experimental methods

Culture of Retrospective Review

Integrating retrospective review planning in the design of the rule can leverage – and inform – prospective review

Potential synergies in regulatory review and broader agency performance assessments

Evaluating the performance of regulatory policy provides the evidentiary basis for political and policy debates on regulations

Further Reading

ACUS Plenary adopted recommendations in December 2014www.acus.gov/research-projects/retrospective-review-agency-rules

Full working paper can be downloaded from my publications webpage http://www.hks.harvard.edu/fs/jaldy/Library/publications.html