EVALUATING IMPACTS OF

REGULATORY REFORMS –

QUANTITATIVE AND QUALITATIVE

METHODS

4 February 2013, Bogotá 

Daniel TrnkaRegulatory Policy Division,Directorate for Public Governance and Territorial DevelopmentOECD

How to evaluate impacts?

• Outputs vs. outcomes• Improving social welfare through

better/smarter regulationvs.

• Saving costs for citizens/businesses/public sector and changing perception

• Ideal case – full RIA• Other options – calculating reduction

of regulatory/administrative costs (SCM), perception surveys

What is the Standard Cost Model?

• Invented by the Dutch Ministry of Economic Affairs in the mid 90s

• Used by all EU countries + Australia, Mexico, Kazakhstan…

• A tool to measure administrative costs and express them in monetary terms

• Advantages:– SCM makes costs ‘visible’ – Easy to measure, monitor & communicate– Uniformity, transparency, reliability and comparability– Commitment & awareness of policymakers– Enables to set a target distribute it across administration

• Disadvantages:– May be too costly– Focus only on one part of costs, not benefits

Different costs of regulations

The costs of regulation to businesses

Direct financialcosts

Compliancecosts

Long-termstructural costs

Indirect financial costs(substantive compliancecosts)

Administrative costs

What are administrative costs?

• Administrative costs/burdens (AB) are the costs imposed on businesses, when complying with information obligations stemming from government regulation

• Costs of paperwork – e.g. filling in forms, submitting them to the administrative authorities

• But also – gathering, processing and updating data, inspections

The methodology for AC measurement

The Standard Cost Model

Possible to differentiate between particular regulations, ministries, sectors of regulation

Methodological questions

• Full or partial baseline measurement, prioritisation – Beyond Pareto principle - 10% of the IO’s accounts for 90% of AC

• Business As Usual (BAU) costs • Costs of informing third parties

(customers, trade unions…)• Costs of material, equipment• Statistical relevance

Statistical relevance

• SCM based on assumptions, not on precise numbers

• 3 to 5 measurements per IO (and consultations of stakeholders) - a sophisticated rule of thumb

• If results converge, 3 to 5 is enough, if not, need a bigger sample

Main challenges

• Interagency co-ordination• Sustained momentum • Cutting dead wood • Last mile issue• Communication – with stakeholders, of results,

managing expectations• Possible discrepancy between the most

burdensome and the most irritating regulations• Evaluation• Potential squeezing-out effect• Integration with other policies• Integration into ex ante impact assessment

Reducing administrative burdens - does anyone perceive higher comfort?

• The shortest answer is: “No.”• A gap between “perceptions and reality”• But “What is reality?” • “If a majority of persons perceive the

same way consistently over time – maybe that’s the reality”

• Some possible reasons:– Absolute numbers vs. individual effects– Delays in impact– Cutting dead wood– Too much focusing on numbers, less on perception,

irritation– Lack of communication

Assessing perception

• Another way to evaluate effects of simplification efforts

• “happiness” is not our main goal but…• Getting feedback from users • Shaping and evaluating regulatory

reform programmes• Perceptions influence compliance with

regulations and investment decisions• Getting a well-deserved praise

Drivers of perception

• Irritation costs, • Service quality, • Different categories of costs, • Unawareness of goals benefits• Frequency of reform, uncertainty• Negativity bias, • Expectations, • Trust in government, political opinions• Communication, involvement

Danish burden hunter technique

• Identify 10-15 specific initiatives which addresses burdens experienced by business

• Develop a method that focuses on burdens experienced by business

Objectives

• Cutting the red tape that business experience as most irritating

• Allowing business to set the agenda and be heard

• User centered methods for collecting data and developing solutions

• Increase knowledge about why specific regulation is experienced as a burden to business – user behavior and efficient regulation

For more information see:

• OECD Cutting Red Tape series:– National Strategies for Administrative Simplification– Businesses' Views on Red Tape– Comparing Administrative Burdens across Countries– Progress in Public Management in the Middle East and

North Africa– Why Is Administrative Simplification So Complicated?– Country reviews: the Netherlands, Portugal. Poland, Viet

Nam• Overcoming Barriers to Administrative Simplification

Strategies: Guidance for Policy Makers

• SCM Network: http://www.administrative-burdens.com/

• OECD project on Assessing performance of regulations through perception surveys

www.oecd.org/regreform/perceptions

THANK YOU!

daniel.trnka@oecd.org

www.oecd.org/regreform