View
213
Download
0
Tags:
Embed Size (px)
Citation preview
Methodology and practical experience in elaboration of aggregated ratings and
indexes, measuring public management quality and efficiency
Vladimir Eliseenko
head of NPM technologies departmentHSE Institute for Public administration and public management
executive secretaryHSE coordination group for strategic development in PA research and teaching
Higher School of Economics , Moscow, 2011
www.hse.ru
Higher School of Economics , Moscow, 2011
Focus on indexes/ratings/grading/ranking in Russia (1)
photo
photo
photo
“Conception (program) of administrative reform in Russia Federation (2006-2010)” contained self-identification and target-setting toward 3 international indexes:
Governance Research Indicator Country Snapshot - GRICS (World Bank)
Global Competitiveness Report (World Economic Forum)
Corruption Perceptions Index (Transparency International)
Higher School of Economics , Moscow, 2011
Focus on indexes and ratings in Russia (2)
photo
photo
photo
Rankings and ratings, measuring new technologies of public administration and public management
I.Quality and scope of NPM technologies applied by subfederal governments (Ministry of Economic development/ Higher school of Economics)
II.Quality of financial management (Ministry of finance)
III.Quality of government cervices (Ministry of Economic development / Higher School of Economics)
IV.Quality of government bodies’ web-pages and e-services Administrative reform (Higher school of economics among 10 others)
Higher School of Economics , Moscow, 2011
Focus on indexes and ratings in Russia (3)
photo
photo
photo
Rankings and ratings of subfederal and municipal authorities according with President’s decrees №607 and №825
More than 230 performance measures for subfederal bodies (governors) and more than 140 performance measures for the heads of municipal authorities
Collected each year (actual for two latest years and predicted for three next years)
Covers both outcome measures and measures for government activities and services
Different measures provided by different sources (official statistics, data from federal and subfederal governments, own information, citizen surveys)
There’s obligation to construct ratings (Ministry of Regional Development for governors and, accordingly, subfederal governments for the heads of municipal authorities) and to give financial grants to best performers
Higher School of Economics , Moscow, 2011
Life after index – how to turn measurement into management (1)
photo
photo
photo
Current problems:
1) frustration with the results (dynamics) of Russia in international indexes:
- are we doing not enough?- are we going the wrong way?- maybe we need more time to see the consequences and to gain the results?
2)what’s the best decision(s) based on the index results?- is it correct to medal the governor, mayor or federal authority who is best “in average”- how we can indicate the “best practices” i.e. define what’s really useful in our effort to improve the quality of life?-if we focus on outcome measures – we can not exclude inertia (- contradiction “max competition – max similarity”
Higher School of Economics , Moscow, 2011
Life after index – how to turn measurement into management (2)
photo
photo
photo
Factor 1. Ensure the need Designing performance management systems to
fulfill multiple goals
Establishment of “agent of change”, capable of and
responsible for managing the elaboration of performance
measures and to coordinate the collection, analysis and use of
performance information
Factor 2. Provide a plan
Factor 3. Build internal support for change and overcome resistance
Factor 4. Ensure top-management support and
commitment
Factor 5. Build external support
Factor 6. Provide resources
Factor 7. Institutionalize change
Factor 8. Pursue comprehensive change
Overcoming measurement
challenges
Designing effective incentives
Managing the performance of
independent agents
Applying comprehensive implementation
strategies
Imposing the system of performance measures,
ensuring their quality and utility for a given purpose
The connection and use of additional benchmarking data
consumers
Dealing with organizational issues to create sustainable and effective benchmarking system
Defining the relations between performance benchmarking and performance contracting
Fernandez/Rainey (2006)
Forsythe (2001)Eliseenko (2007)
Higher School of Economics , Moscow, 2011
Life after index – how to turn measurement into management (3)
photo
photo
photo
Decision 1. Compare units on the basis of dynamics, not on the basis of absolute values
There’s too much economic and social differentiation in Russia, both across the regions and inside the regions (across)
It allows to compare the whole range of units without dividing them into subgroups (amplifies the competition)
It results in the “victory of the weak” but it is wonderful from the point of:
a) equalization (levelling)b) bringing the attention to the system
Higher School of Economics , Moscow, 2011
Life after index – how to turn measurement into management (4)
photo
photo
photo
Decision 2.Connect the weights of indicators (measures) with the difference of values across units
eliminates the effect of levelling (average of average)
shows clear priorities
stimulates to concentrate the efforts on “hard” measures not on the easiest to achieve
Higher School of Economics , Moscow, 2011
Life after index – how to turn measurement into management (5)
photo
photo
photo
Decision 3. Find the cause-effect chains across different levels of performance measures
Higher School of Economics , Moscow, 2011
Life after index – how to turn measurement into management (6)
photo
photo
photo
Decision 3. example of cause-effect chains across different levels of performance measures
Education Performance-based budgeting
“Money follows the pupil”
Financial grants for best teachers
Qualification of teachers
Percentage of interactive lessons
Unified testing results
Percentage of pupils who hasn’t got a certificate
Public housing and utilities
Performance-based budgeting
Subsidies to low-income families instead of losses compensation
Financial health of service providers (both private and public)
Quantity of complaints on the poor quality of the absence of services (water, heating etc)
Higher School of Economics , Moscow, 2011
Life after index – how to turn measurement to management (7)
photo
photo
photo
Decision 3. example:
SCHOOLEDUCATION
SCHOOLEDUCATION
HOUSING SERVICES AND UTILITIES
HOUSING SERVICES AND UTILITIES
PUBLIC HEALTHPUBLIC HEALTH
AOAO AOAO AOAOAEQAEQ AEQAEQ AEQAEQIII III III
Higher School of Economics , Moscow, 2011
Life after index – how to turn measurement to management (8)
photo
photo
photo
Decision 3. possibility to prove (by econometrics) the cause-effect chains across different levels of performance measures
Higher School of Economics , Moscow, 2011
Ongoing project: data (1)
photo
photo
photo
We use HSE ranking, based on the quality of administration in regional executive bodies (mostly the components of administrative reform) – measured in 2006 and 2007
А+ Excellent quality of PA
А Good quality of PA
В Middle quality of PA
С Poor quality of PA
Performance management and performance based budgeting (programming, internal and external, planning, evaluating the efficiency of government expenditures and government agencies and NDPB)
Quality of internal management (functional analysis, optimization of government regulation and control, anticorruption activities,
Treating citizens as clients/consumers (administrative simplification, quality of government services, openness and participation, “single window” and one-stop shops for government services, e-services)
Higher School of Economics , Moscow, 2011
Ongoing project: hypotheses (1)
photo
photo
photo
Main:
G1. The implementation of new PA tools and technologies influence the quality of regulation and administration in different branches of government policy. G1A. The degree of influence for each new PA technology can be different, depending on actuality (starting position).
G2. The quality of regulation and administration in a certain branch of government policy positively influence the efficiency and quality of private firms and government establishments, providing the services to the citizen and businesses
G3. The quality of services to the citizen and businesses positively influence the dinamics of outcomes in a certain branch of socio-economic system.
Higher School of Economics , Moscow, 2011
Ongoing project: hypotheses (2)
photo
photo
Alternative:
AG1. Quality and speed of adoption of new PA tools and technologies is not independent but dependent variable. It depends on the qualifications of higher-level management in subfederal government. Thereby, the participation in federal PA reforms distracts human and financial resources, which could be spent more effective way.
AG2. Only economic variables (per capita GDP, per capita budget revenues) can significantly influence the outcomes (the quality of living) and government efficiency.
AG3. Government is an evil itself so the reduction of public servants in any situation can positively influence the outcomes (quality of living)
Vladimir Eliseenkoe-mail: [email protected]
room k-20420, Myasnitskaya str., Moscow, Russia,
Phone.: +7 -916-596-6620