Embed Size (px)
Citation preview
1
INDICATORS OF REGULATORY QUALITY: DEFINITIONS AND MEASUREMENT
Claudio M. Radaelli, Fabrizio De Francesco
Centre for European Studies,
Bradford UniversityPresentation on the progress so far16 March 2004
2
IRQ – Indicators of regulatory quality
Project: • Stage 1 Review of
initiatives and indicators used by EU member states to measure regulatory quality (plus others with interesting experiences)
• Stage 2: Elaboration of a set of indicators to be used by the European Commission
• Timing: 14 months, interim report in May 2004, report on indicators in September 2004
IRQ Team • Bradford team
(Radaelli, De Francesco, Cameron, Jalilian)
• Advisor: Scott Jacobs• Advisor: Bruno Dente
Support network: 27 national experts on
‘better regulation’ policy from 16 EU M.S.
(incl. 6 CEECs)
3
Aims in this presentation
To discuss regulatory quality, the quality of indicators, and indicators of quality.
To collect feedback, suggestions, and information.
Please contact [email protected]
4
Table of contents
1. Getting to grips with the concept: what is regulatory quality?
2. Complexity of regulatory quality3. How does one measure quality?4. Classification of indicators5. From classification to the analysis
of the performance of indicators and system of indicators
5
1. Regulatory quality (RQ)
• Governments• How the EU defines RQ• Definitions provided by international
organisations• Academic approaches
6
UK: Better Regulation Task Force
Proportionality, accountability, consistency, transparency, and targeting
7
Commission
• RQ is part of the strategy for better regulation and good governance. This implies that, in addition to properties such as efficiency, effectiveness, coherency, simplicity, and clarity, good regulation should also respect:• Standards for the re-vitalisation of the acquis
(COM71 Final,) Commission 2003• Standards for the democratic use of expertise• Standards for consultation• And most importantly good regulation balances
environmental protection, social conditions, and economic growth sustainable development
8
Mandelkern Report (Council’s experts – EU)
Common principles:• Necessity• Proportionality• Subsidiarity • Transparency• Accountability• Accessibility• Simplicity
MR is focused also on how to achieve RQ. The main tools are:• Alternatives to regulation• IA• Consultation• Simplification• Access to regulation
9
OECD: evolution of ideas on RQ
• 1995: Good practices -> standards -> checklists• 1997: Three stages of development of regulatory reform:
de-regulation, regulatory quality (instrument by instrument approach), and regulatory management (more horizontal)
• 2002: From RQ to regulatory governance (transparency, citizens as partners, and bringing principles of better regulation across levels of governance and regulatory actors (legislature, judiciary) grounded in the wider approach to good democratic governance two main perspectives on regulatory quality: economic and juridical, both correlated with the design and implementation of policies, tool and institutions
10
Academics: Baldwin and Cave
They derive RQ from the claims to legitimacy [of regulation]:Legislative mandateAccountabilityExpertiseEfficiency: the legislative mandate is
implemented correctly
11
Baldwin, Hood, and Rothstein 'Assessing the Dangerous Dogs Act: When Does a Regulatory Law Fail?' Public Law Summer (2000)
Principles do not help much. Paradoxically, one can demonstrate that even bad legislation can pass the test of complying with the principles of the Better Regulation Task Force.
We need to evaluate the results achieved by regulation rather than use principles and key-words. More emphasis on systematic evaluation (ex-ante and ex-post) and on effective implementation of proposed regulation
Indicators, not slogans?
12
“Economists”
They take issue with the narrow notion of efficiency presented by Baldwin and Cave
They have a different benchmark: maximisation of the collective welfare
Quality regulation is about the economic impact of regulation, not about implementing the mandate
13
Dimensions of quality
Economists: They do not talk about RQ, they talk about the efficiency of regulation.
But efficiency is only one element of quality or one claim to legitimacy (in Baldwin and Cave’s typology).
Another element of RQ is the quality of the process through which regulations are produced.
Doubt: do economists measure quality or quantity (extent of regulation)?
14
To conclude on RQ: Advantages and limitations
• RQ tells us something about quality of governance
• RQ is neutral with respect to size of government or scope of intervention (claim made by OECD). Hence it does not express a judgment on how big government should be. Unless one follows the economists and ‘confuses’ quality and quantity.
• The notion of RQ is a bit fuzzy
• There are too many attributes in the current definitions of regulatory quality, but how does one recognise regulatory quality in the real world? What are our empirical sensors?
15
How our project approaches RQ
• Design of the process (structures, inter-organisational linkages, etc.)
• Activities and outputs
• Real-world outcomes
16
The puzzle of real-world outcome
Cross-country differences revealed by indicators: do they really mean differences in policies and their enforcement? Or do they simply register different models of capitalism (Hall and Soskice)?
Jaffe et al, JEL 1995: the evidence of the impact of regulation on competitiveness and locational choices of firms is far from clear. Same results for environmental regulation. But see Koedijk, K. e J. Kremers (1996), Market opening, regulation and growth in Europe, Economic Policy. Regulatory governance matters
Literature on regulatory competition: evidence tells us that countries do not compete by using regulation in the way predicted by simplistic economic models of ‘races’
OUR SUGGESTION: for us real-world outcome makes more sense if we look at changes in the behaviour of regulatory stakeholders. Regulatory governance as learning
17
Tools for RQ
Impact assessmentConsultationSimplification, codification, recastingAccess to legislation and regulatory
transparencyEx-post evaluation of regulatory
tools and institutions
18
2. Complexity of RQ
Quality is not a monolithic conceptQuality for whom?(A) Context shapes RQ
SEE PAPER ‘HOW CONTEXT MATTERS’ (Radaelli)
(B) Models of the policy process (C) Models of actors(A), (B) and (C) are often neglected in
the one-size-fit all approach to RQ
19
How different stakeholders look at RQ
Economist
Civil Servant
Politician Firm Citizen
Criteria Efficiency Conformity to rules
Consensus
Cost Mini-misation
Cost-effective protection from risk
Success Goals achieved
Following procedures
Outcome of negotiation
Profit ‘Enabling’ regulation
Logic of action
Logic of social sciences
Standard operating procedures
Logic ofNegotiation
Logic of influence
Logic of partici-pation
20
Implicit models of actors and implicit theories of the policy process
Different stakeholders bring into the discussion diverse logics and criteria of quality and quality assurance mechanisms.
It is not clear what is the model of stakeholders implicit in RQ programmes.
On theories of the policy process see the paper HOW CONTEXT MATTERS
21
Implicit theories (2)
Are these programmes based on the assumption that politicians are rent-seeking, hence quality assurance mechanisms should target this problem?
Or does ‘quality’ mean that a government is trying to curb bureaucratic power? Is RQ an approach to limit the power of business in the policy process or an instrument to provide more systematic access of corporate actors to the regulatory process?
What do corporate actors want in a regulatory process? Efficiency or protection? Profit maximisation or satisfying behaviour? Are they a unitary actor or in competition among themselves?
22
Implications
It is important to acknowledge the complexity of RQ up-front.
It is indispensable to discuss the criteria, theories, and models of actors upon which our approach to RQ is based.
23
3. How does one measure quality?
Tests
Indicators:
Quality of the RIA processQuality of the regulations produced via
RIA
24
4. Classification of indicators
LEVELS
TOOLS
TYPES OF
REGULATION
SECTORS OF
REGULATION
DEGREES OF
SOPHISTICATION
TYPES OF MEASURE
PROVIDERS
LEVELS OF
MEASUREMENT
PURPOSES
PRINCIPLES
25
BY LEVEL
Design of the process
Activities and Outputs
Real-World Outcomes
26
BY TOOL
Impact assessmentConsultationSimplification, codification, recastingAccess to legislation and regulatory
transparencyEx-post evaluation of regulatory
tools and institutions
27
BY TYPE OF REGULATION
ECONOMIC REGULATION
SOCIAL REGULATION
ADMINISTRATIVE REGULATION
28
BY SECTOR OF REGULATION
Regulation of Entry
Regulation of Labor
Environmental Regulation
Product Regulation
TAX and Social security law
Planning regulation
Etc…
29
BY DEGREE OF SOPHISTICATION
Simple measures Composite measures (Factor analysis,
aggregation of simple indicators on the basis of weighting).
Note that composite measures are not necessarily better than simple measures. For example, factor analysis may not be explicit enough on the variables that can be controlled by program administrators. Or it can be the statistical artifact of a questionnaire with too many questions.
30
BY TYPE OF MEASURE
Objective
Subjective
31
BY PROVIDER
Governments (self-assessed responses to regulatory quality questionnaires)
Experts (think tanks responding to Word Bank questionnaires or reviewing IA)
Stakeholders
32
BY CHARACTERISTIC
Categorical (ex. Dichotomous (Yes/No);
Ordinal (ex. 1 to 5 scales)Continuum (ex. Number --
no. of procedures, steps, etc.; Ratios; indexes)
33
BY PURPOSE: INDICATORS FOR WHAT?
Learning
Explanation
Transparency and accountability
Program management
Participation and ‘good regulatory governance’
34
By PRINCIPLEof better regulation policy
AustraliaCanadaUKUSAcademicsMandelkern ReportOECDSee Section 1 on the concept of RQ for the
list of principles
35
An example: combining values and levels
Values Process ActivityOutput
Real World
Necessity X X
Proportionality X X
Subsidiarity X (X)
Transparency X
Accountability X X X
Accessibility X X
Simplicity X X
Consistency X (X)
36
5. From classification to the analysis of performance
• Assessing quality of individual indicators• Assessing quality of systems of
indicators• Controlling for adverse effects• Crucial step: Managing indicators – using
indicators to improve management, to learn, and to produce better regulation.
37
From classification to the assessment of quality of each indicator (1)
EASY STEPS Validity (relation between indicator and
phenomenon) Quantifiable at regular intervals, so that the
indicators can be monitored Reliable (two different people produce the
same value given the same conditions) Comparability and cross-checkable Easy to gather, easy to communicate
38
From classification to the assessment of quality of each indicator (2.1)
ENTERING COMPLEXITY Efficient: a sophisticated indicator (arising out of
factor analysis) contains more information than simple indicators.
However, there may be a trade off between the quantity of information contained in an indicator and how easy it is to communicate it within the organisation and to the stakeholders.
It is important that the variables upon which the complex measure is built are easy to understand and coherent (no trade-off between the variables).
39
From classification to the assessment of quality of each indicator (2.2)
Relation with principles or norms. This is to avoid ambiguity. The indicator relates to an outcome that can be unambiguously judged satisfactory or not.
Do we measure the quality of the regulatory environment or the quality of better regulation programmes? In the long term they should converge. In the short-term, the environment can improve independently from the policy. Example: more people employed to issue licenses, the time needed to get a licence goes down, but the overall regulatory complexity is the same.
40
From classification to the assessment of quality of each indicator (3)
ENTERING PRINCIPLES Take the example of principles of better
regulation. Think of participation. Is this a dimension of quality? It depends on why one is designing an indicatorAlways ask the question: if the indicator is the solution,
what is your problem? Do you want to learn, to manage, to steer, to empower people,….?
This example tells us that the definition of quality is sensitive to the main purpose one has in mind when developing the system of indicators.
41
From classification to the quality of systems of indicators
• Coverage … but also simple and selective• Good balance among design, output, and real-
world outcome. Good balance between providers (stakeholders and governments)
• The system should be targeted and focused on those themes in regulatory reforms programmes that have large implications for decision-making
42
Dynamic systems of indicators
• Start with simple system of indicators, then in year two add more sophisticated indicators
• Use indicators to allow member states to measure their own progress year by year and in relation to other member states. Indicators and scoreboard of regulatory quality? Drawback: learning may be hampered by the ranking syndrome. This is not a race, but a mutually beneficial exercise.
• Use two sets of indicators. A simple set of those who use indicators to monitor complex regulatory reform programmes and another set of indicators for those who manage individual programmes.
