What are Actionable Governance Indicators (AGIs)?siteresources.worldbank.org/EXTPUBLICSECTORAND... · What are Actionable Governance Indicators (AGIs)? Definition of AGIs Actionable

1 02/05/2008

Actionable Governance Indicators – Concepts and Measurement1

What are Actionable Governance Indicators (AGIs)? ........................................................ 1 Definition of AGIs .......................................................................................................... 1 Governance Dimensions ................................................................................................. 1 Governance Systems....................................................................................................... 3

Why Should We Care About AGIs?................................................................................... 3 Objectives of AGIs ......................................................................................................... 3 Understanding AGIs Within the Normal Indicators Spectrum....................................... 4 AGIs and Modeling the Determinants and Impacts of Governance Systems................. 4

How Can We Actually Measure AGIs?.............................................................................. 7 Two Categories of AGIs ................................................................................................. 7

Governance Systems Features (Performance-determinants): r-indicators.................. 7 Governance Systems Performance: Y-indicators ....................................................... 8

Illustrations of AGIs ....................................................................................................... 9 Public Finance Management (PFM) AGIs: PEFA Indicators..................................... 9 Human Resource Management (HRM) AGIs........................................................... 11

Annexes............................................................................................................................. 16 Annex 1: Summary of AGI Features and Exclusions................................................... 16

Key Features of AGIs ............................................................................................... 16 Exclusions: What AGIs Are Not............................................................................... 16

Annex 2: What Distinguishes AGIs from Other Types of Indicators?......................... 17 Annex 3: HRM Diagnostic Tool................................................................................... 20

What are Actionable Governance Indicators (AGIs)?

Definition of AGIs Actionable governance indicators (AGIs) provide evidence on the characteristics and functioning of particular elements and sub-elements of the various dimensions of governance.

This note explains what AGIs are and illustrates them with a couple of examples. As the above definition requires clarity on what constitute dimensions, elements and sub-elements of governance, we begin with explanations of those terms. Annex 1 provides a bullet-point summary of key features of AGIs, as well as clarification of what AGIs are not, while Annex 2 provides additional clarification of what distinguishes AGIs from other types of indicators.

Governance Dimensions Governance dimensions could, of course, be defined in any number of ways. To avoid opening a hornet’s nest of argument about what constitutes a “governance dimension” and what comprises the full set of “governance dimensions”, we simply adopt a 1 Gary J. Reid, Lead Public Sector Management Specialist and Coordinator, Administrative and Civil Service Reform (ACSR) Thematic Group, The World Bank.

2 02/05/2008

schematic summary of the various dimensions of “good governance” that has evolved within the World Bank over the last decade:2

44

Political Accountability• Political competition, broad-based political parties• Transparency & regulation of party financing• Disclosure of parliamentary votes

Checks & Balances• Independent, effective

judiciary• Legislative oversight

(PACs, PECs)• Independent oversight

institutions (SAI)• Global initiatives: UN,

OECD Convention, anti-money laundering

Citizens/Firm

s

Citizens/Firms

Citi

zens

/Firm

s

Citizens/Firms

Decentralization and Local Participation• Decentralization with accountability• Community Driven Development (CDD)• Oversight by parent-teacher associations & user

groups• Beneficiary participation in projects

Civil Society & Media• Freedom of press, FOI• Civil society watchdogs• Report cards, client surveys

Private Sector Interface• Streamlined regulation• Public-private dialogue• Extractive Industry

Transparency• Corporate governance• Collective business

associations

Effective Public Sector Management

• Ethical leadership: asset declaration, conflict of interest rules

• Cross-cutting public management systems: meritocracy, public finance, procurement

• Service delivery and regulatory agencies in sectors

Dimensions of Good GovernanceDimensions of Good Governance

AGIs, then, are indicators that provide evidence on the characteristics and functioning of particular elements or sub-elements of any of the five governance dimensions identified in the above diagram:

1. Political accountability 2. Checks & balances 3. Civil society, media and private sector interface with politics and the public

administration 4. Decentralization and local participation 5. Public sector management

For simplicity of exposition, we will henceforward employ the term “governance systems” to refer to the elements and sub-elements of any given governance dimension,

2 World Development Report 1997: The State in a Changing World (The World Bank: Washington, DC, June 1997); Helping Countries Combat Corruption: The Role of the World Bank (The World Bank: Washington, DC, September 1997); Helping Countries Combat Corruption: Progress at the World Bank Since 1997 (The World Bank: Washington, DC, June 2000); Reforming Public Institutions and Strengthening Governance: World Bank Strategy, (The World Bank: Washington, DC, November 2000); Anticorruption in Transition: A Contribution to the Policy Debate (The World Bank: Washington, DC, November 2000); World Development Report 2004: Making Services Work for Poor People (The World Bank: Washington, DC, September 2003); Anti-Corruption in Transition 3: Who Is Succeeding…and Why? (The World Bank: Washington, DC, July 2006); Global Monitoring Report 2006: Strengthening Mutual Accountability – Aid, Trade and Governance, (The World Bank: Washington, DC, 2006); Strengthening World Bank Group Engagement on Governance and Anticorruption (The World Bank: Washington, DC, March 2007).

3 02/05/2008

whenever the distinction between elements and sub-elements is not essential to the argument.

Governance Systems Governance systems comprise the elements and sub-elements of the various governance dimensions identified in the above diagram. Examples of the “elements” of each governance dimension are provided in that diagram. But even within those identified within that diagram, one can still drill down, to what we are terming sub-elements. Thus, “cross-cutting public management systems” would include at least the following sub-elements:

• Public finance management systems (including budget, revenue, accounting, auditing, and procurement systems)

• Human resource management systems (including both civil service management systems, as well as personnel management systems for public employees not encompassed within a given country’s civil service)

• Policy management systems (i.e., policy formulation, policy coordination, and oversight of policy implementation in order to better inform both policy formulation and policy coordination activities and decisions)

Similarly, “service delivery and regulatory agencies” would include sub-elements, such as, specific service delivery agents (e.g., a Ministry of Education, individual schools, a Ministry of Health, health care providers, a health care financing institute, etc.), as well as specific regulatory bodies. Each such service delivery or regulatory agency would be a candidate for AGIs designed to provide evidence on the characteristics and functioning of that particular sub-element of the “service delivery and regulatory agencies” element of the public sector management dimension of governance.

Why Should We Care About AGIs?

Objectives of AGIs We care about AGIs for three basic reasons:

1. Design: Guide the design of specific governance systems reforms. 2. Implementation: Hold such governance systems reform efforts accountable for

achieving their objectives. 3. Learn from Experience: Facilitate systematic empirical research on:

a. Determinants of governance systems performance: What factors (e.g., reform design elements) contribute most significantly to achieving specific governance systems objectives?

b. Determinants of governance systems impacts: What impacts do the various aspects of performance of specific governance systems have on both broad governance quality as well as on specific aspects of government activities, such as production of public goods and services?

4 02/05/2008

Understanding AGIs Within the Normal Indicators Spectrum If AGIs are to be helpful for these three purposes, they need to be properly chosen. For this reason, it is important to understand how AGIs fit into the conventional set of performance indicator distinctions. The literature on indicators is voluminous. Standard typologies of indicators generally posit three broad levels of indicators – inputs, outputs and outcomes.

1. Inputs are the resources employed and activities undertaken in order to produce particular outputs. They can be measured either in monetary terms (dollars, euros, yen, rubles, lei, pesos, etc.), in terms of the magnitudes of particular types of inputs (e.g., teachers, chalk, books, civil servants, medications, etc.), or by monitoring the extent of particular types of activities (e.g., number of teachers/pupil, average class size, etc.)

2. Outputs are the products of those inputs and activities, and they are typically produced as a means to an end – i.e., individual outputs are generally things that policy makers believe are required to achieve other, higher level outcomes or results. Outputs are things like reports produced, laws revised, bridges built, students attaining some given level of education, vaccinations provided, etc.

3. Outcomes are the ultimate objectives of public policies. They are influenced not only by the outputs of public sector policies and programs, but also by other factors, like the economy, social norms, etc. Examples of outcomes include things like the health of the population (e.g., average life expectancy, infant mortality rates, etc.), economic well being of the population (e.g., average real income), environmental quality (e.g., average air quality, average water quality, etc.).

This three-level hierarchy of indicators, however, is problematic when applied to governance issues. This is because governance facilitates rather than directly delivers both outputs and final outcomes. Given this, one needs to employ an additional typology of indicators in order to be able to shed light on both (a) what factors contribute to improving the performance of any given governance system, and (b) what impacts those governance systems have on both outputs and outcomes of government policies and programs, as well as how those impacts are conditional on other factors (e.g., context).

AGIs and Modeling the Determinants and Impacts of Governance Systems Governance systems influence the ways and extent to which a given country’s public sector produces outputs and outcomes. This influence can be modeled through a system of four sets of equations (all variables are multi-dimensional arrays):

1. Y = y(r|s1)

2. G = g(Y|s2) 3. Q = f(G, x, z1)

4. O = m(G, Q, z2)

where

5 02/05/2008

Y = Quality (performance) of governance systems G = Quality (performance) of governance dimensions Q = Outputs O = Outcomes r = Factors that determine the performance of each governance

system, where such performance is the extent to which each governance system meets each of its functional objectives3

s1 = Exogenous (contextual) factors that affect the impacts of the r-factors on the performance of each governance system

s2 = Exogenous (contextual) factors that affect the impacts of each governance system on overall governance quality

x = Inputs z1 = Exogenous factors that impact output productivity independently

of governance systems z2 = Exogenous factors that impact outcome productivity independently

of governance systems y, g, f, m = Production functions for governance systems, governance

dimensions, outputs, and outcomes, respectively

In short, the outputs of public sector activities depend on three sets of factors: governance dimensions and systems (G), inputs (x), and exogenous factors (z1). Similarly, outcomes depend on governance dimensions and systems (G), outputs (Q), and a different set of exogenous factors (z2). In both of these relationships, however, the quality of governance dimensions and systems depend on an array of factors (r), whose impacts are conditional on another array of exogenous factors (s1 or s2).

Reduced forms of the output and outcome equations would be:

5. Q = f*(Y, r, s1, s2, x, z1)

6. O = m*(Y, r, s1, s2, Q, z2)

One of the reasons for developing AGIs identified above is to facilitate a better understanding of the above relationships. Accordingly, AGIs must provide systematic data on the Y and r arrays; i.e., the quality of particular governance systems (Y) and the factors that impact those qualities (r). Such information needs, in turn, to be complemented with evidence on how their impacts are conditioned on exogenous (contextual) factors, which requires information on those exogenous factors (s1 and s2). In the long run, of course, one would love to use AGI data in the estimation of any or all of the above six equations. This is not realistic in the near or medium term.

What is realistic is to generate Y and r data, complement it with s data, and employ such data for the three purposes already enumerated above: (i) guiding governance system reform design, (ii) facilitating more accountable implementation of such reform efforts by

3 To illustrate: Public sector HRM systems typically have four functional objectives: (i) depoliticized HRM practices; (ii) attracting and retaining required human capital; (iii) meritocratic and performance-focusing management of human resources; and (iv) wage bill sustainability. The various elements of the r-array for an HRM system would be those factors that contribute to meeting each of these four HRM objectives.

6 02/05/2008

continuously monitoring the extent to which those reforms achieve their objectives, 4 and (iii) facilitating empirical research on the determinants of governance systems performance5.

• Design and implementation of governance system reform efforts: AGIs will help to inform reform efforts on what they are accomplishing (through the Y-indicators), as well as on the extent to which the reform effort is making progress on meeting the prerequisites (the r-indicators) for improved governance system performance.

• Facilitate research on the determinants of governance system performance: AGI indicators should facilitate empirical research on the contributions of the various r-indicators to governance system performance, as measured through the Y-indicators.

Despite the extensive literature on monitoring issues, there is relatively little recognition that better understanding of either the contributions of governance dimensions to public sector performance, or the factors that actually determine the quality with which particular governance systems function, requires careful, systematic and replicable measurement of these Y, r and s arrays; i.e., of the quality of particular governance systems (Y), the factors that impact the various quality dimensions of those governance systems (r), and the exogenous factors (s) that often condition the relationships between Y and r.

In short, AGIs must provide evidence on the following sorts of variables:

• Determinants (r): Design features of governance systems • Performance (Y): Extent to which governance systems meet their functional

objectives • Exogenous factors (s): Other factors that can condition the impacts of:

o Governance systems design features on the extent to which they meet their functional objectives

o Governance systems functional performance on outputs and outcomes of government policies and programs.

This AGI initiative aims specifically at helping to address the need for systematic evidence on Y, r, and s. One helpful way of thinking of what this need amounts to is to consider that those of us in the development field, and in the World Bank in particular, devote considerable amounts of attention to monitoring the development aid we provide and its results. That monitoring typically focuses on one or more of the three standard types of indicators identified above: inputs, outputs and outcomes. What is rarely monitoring in the governance field are the intermediate phenomena that are the focus of governance reform support and efforts (Y and r); or what we refer to as the “missing middle” of the indictors spectrum:

4 This should be possible in the near term in any given country, as it only requires monitoring of relevant Y and r indicators. 5 This will only be feasible in a more medium term, as it requires such data for enough countries over a long enough time period to allow empirical research.

7 02/05/2008

• Inputs and outputs – i.e., inputs employed, actions taken and products produced as part of an effort to improve the functioning of some specific element or elements of one or more governance dimensions.

• AGIs (the missing middle) – i.e., evidence on the characteristics and functioning of particular governance systems. Details on the sorts of things such indicators can be designed to capture are presented below.6

• Outcomes – i.e., the final impacts of a country’s governance institutions on one or more of the five dimensions of governance identified above, or on political, social or economic phenomena about which citizens care. The most widely monitored sets of such outcome indicators for governance focus on things like corruption and its myriad manifestations, “rule of law”, “political freedom”, “democracy”, etc.

How Can We Actually Measure AGIs? A program that aims at measuring AGIs for various governance systems must be clear about the sorts of things that such indicators must capture. The above analysis helps to clarify what sorts of things will need to be measured. That analysis makes clear that the focus of AGI measurement must be the various elements of the Y and r factors that apply to any given governance system targeted by any given AGI monitoring effort.

Two Categories of AGIs Given the above reasoning, it follows that AGIs fall into two broad categories: Y- and r-indicators; where Y-indicators provide information on the performance of particular governance systems along their various performance dimensions, while r-indicators provide information on the determinants of the performance of given governance systems along their various performance dimensions.

Governance Systems Features (Performance-determinants): r-indicators r-indicators aim to capture the features of any given governance system that determine how well that governance system functions along its various performance dimensions. As a starting point for developing such indicators, it is typical to devise indicators that capture widely agreed prerequisites for a well functioning governance system. A useful typology of such r-indicators subdivides them into two broad sets – (i) rules of the game, and (ii) descriptors of capacities of the various agents involved in a given governance system:

• rules of the game – i.e., rules governing the actions of agents who are involved in the operation of a given governance system. These rules create (better or worse) incentives for agents to perform their roles. They do so in a variety of ways, including:

6 For more discussion regarding what distinguishes AGIs from input, output and outcome indicators, please see Annex 2, “What Distinguishes AGIs from Other Types of Indicators?”

8 02/05/2008

o circumscribing the behavior and actions of agents within a particular governance dimension by specifying rules that govern the actions/behavior of particular agents:

mandatory rules: rules that require particular actions by particular agents

permissive rules: rules that permit a range of actions/behavior by particular agents

o defining responsibilities; i.e., defining particular responsibilities of particular agents within any

given governance system and establishing the rules governing the exercise of each such

responsibility o assigning responsibilities, rights and authority; i.e., parceling out those

responsibilities, rights and particular types of authority to distinct agents, including, but not limited to:

policy-setting agents; i.e., agents whose roles are to establish the rules (policies, procedures) governing the actions of other agents within a given governance system

operational agents; i.e., agents whose roles are to do the work of a given governance system

oversight agents; i.e., agents whose roles are to exercise oversight of other agents within a given governance system,

separating conflicting responsibilities; i.e., assigning responsibilities across agents in ways that avoid creating conflicts of interest, e.g., by separating oversight responsibilities from operational responsibilities,

o requiring tracking, monitoring and disclosure of evidence on the exercise of particular responsibilities, rights and authorities: by, e.g.,

imposing records tracking (e.g., accounting) requirements for each set of responsibilities,

imposing reporting requirements for each set of responsibilities, imposing freedom of access to information requirements,

o creating checks and balances, both within and across governance systems and dimensions.

• capacity features – i.e., characteristics of resources that are widely recognized as important determinants of the capacity of a given governance system to function well

o types of resources, including descriptors that capture the quality of each resource

o quantities of each type of resource o production technologies employed and their defining characteristics

Governance Systems Performance: Y-indicators Y-indicators provide evidence on how well any given governance system is functioning along particular performance dimensions; i.e., how well that system is achieving each of

9 02/05/2008

its functional performance objectives. Such indicators capture the extent to which the immediate objectives of specific institutional reforms are being achieved. More specifically, such indicators measure aspects of organizational behavior and practices, whose variance reflects the extent to which a particular governance system is achieving its various functional performance objectives. For instance, one of the objectives of a human resource management system is to attract qualified human capital skills. An indicator of how well this objective is being achieved is the average number of qualified applicants per advertised position. Higher averages would reflect better performance on this objective than would lower averages. In addition, variance in such averages across sets of human capital skills (e.g., across professions or types of positions) would indicate that this objective was being better (conversely, less well) achieved for some skill sets than for others.

Illustrations of AGIs In what follows we provide concrete illustrations of AGIs for two specific governance systems: (i) public finance management, and (ii) human resource management.

Public Finance Management (PFM) AGIs: PEFA7 Indicators The Public Expenditure and Finance Accountability (PEFA) indicators provide perhaps the most fully developed, vetted and officially endorsed set of AGIs currently available.8 The PEFA methodology provides 28 indicators, covering six dimensions of public finance management systems. Some of the PEFA indicators capture determinants (r-indicators) of public finance management (PFM) system performance along particular PFM dimensions, while others capture performance (Y-indicators) along particular PFM dimensions. Many capture a mixture of these two types of AGIs, as many of PEFA’s detailed indicators are multi-dimensional (see Table 1).

The six critical dimensions covered by the PEFA indicators are9:

A Credibility of the budget - The budget is realistic and is implemented as intended

B Comprehensiveness and transparency - The budget and the fiscal risk oversight are comprehensive, and fiscal and budget information is accessible to the public.

C Policy-based budgeting - The budget is prepared with due regard to government policy.

D Predictability and control in budget execution - The budget is implemented in an orderly and predictable manner and there are arrangements for the exercise of control and stewardship in the use of public funds.

E Accounting, recording and reporting – Adequate records and information are produced, maintained and disseminated to meet decision-making control, management and reporting purposes.

7 Public Expenditure and Financial Accountability (PEFA): www.pefa.org 8 http://www.pefa.org/PEFA%20Website%20--%20CURRENT%204-9-03/www.pefa.org%20WEBSITE/PFM%20PMF%20Final%20June%202005%20for%20reprint%20May%202006.pdf 9 Source: Public Financial Management Performance Measurement Framework (PEFA Secretariat, The World Bank: June 2005 (reprinted May 2006), p. 2.

10 02/05/2008

F External scrutiny and audit - Arrangements for scrutiny of public finances and follow up by executive are operating.

Within each of these high level indicators, the PEFA methodology identifies between two and nine lower level indicators, which are summarized in Table 1.

Table 1: PEFA Indicators Indicator Acronym

Indicator Type: Determinant (r-indicator) or Performance

(Y-indicator)

A. Credibility of the budget

PI-1 Aggregate expenditure out-turn compared to original approved budget

Performance

PI-2 Composition of expenditure out-turn compared to original approved budget

Performance

PI-3 Aggregate revenue out-turn compared to original approved budget

Performance

PI-4 Stock and monitoring of expenditure payment arrears

Performance/Determinant

B. Comprehensiveness and transparency

PI-5 Classification of the budget Performance

PI-6 Comprehensiveness of information included in budget documentation

Performance

PI-7 Extent of unreported government operations Performance

PI-8 Transparency of inter-governmental fiscal relations


PI-9 Oversight of aggregate fiscal risk from other public sector entities

Performance

PI-10 Public access to key fiscal information Performance

C. Policy-based budgeting

PI-11 Orderliness and participation in the annual budget process


PI-12 Multi-year perspective in fiscal planning, expenditure policy and budgeting

Determinant

D. Predictability and control in budget execution

PI-13 Transparency of taxpayer obligations and liabilities

Determinant

PI-14 Effectiveness of measures for taxpayer registration and tax assessment

Determinant

PI-15 Effectiveness in collection of tax payments Performance/Determinant

11 02/05/2008

Indicator Acronym

Indicator Type: Determinant (r-indicator) or Performance

(Y-indicator)

PI-16

Predictability in the availability of funds for commitment of expenditures


PI-17

Recording and management of cash balances, debt and guarantees


PI-18

Effectiveness of payroll controls Determinant

PI-19

Competition, value for money and controls in procurement


PI-20 Effectiveness of internal controls for non-salary expenditures


PI-21 Effectiveness of internal audit Performance/Determinant

E. Accounting, recording and reporting

PI-22 Timeliness and regularity of accounts reconciliation

Performance

PI-23 Availability of information on resources received by service delivery units


PI-24 Quality and timeliness of in-year budget reports Performance/Determinant

PI-25 Quality and timeliness of annual financial statements


F. External scrutiny and audit

PI-26 Scope, nature, and follow-up of external audit Performance

PI-27 Legislative scrutiny of annual budget law Performance/Determinant

PI-28 Legislative scrutiny of external audit reports Performance

Human Resource Management (HRM) AGIs AGIs for human resource management (HRM) have been developed for monitoring both performance of HRM systems (Y-indicators), as well as determinants of the performance of those systems (r-indicators).

Performance HRM AGIs (Y-indicators) HRM reforms typically pursue one or more of four objectives:

1. Depoliticized personnel management 2. Merit-based, performance-encouraging personnel management 3. Attract and retain qualified staff 4. Fiscal sustainability of the wage bill

Table 2 identifies a set of indicators within each of these three broad objectives, each of which captures either a recognized prerequisite for furthering that objective or a

12 02/05/2008

dimension of organizational behavior that sheds light on whether that objective is being advanced.

Table 2: Human Resource Management Performance Indicators Objective Rationale Indicator Type:

Determinant (r-indicator) or

Performance (Y-indicator

1. Depoliticized personnel management within the Civil Service (CS)

Turnover within the civil service (CS) unrelated to changes in political leadership

Quarterly civil service turnover rates that spike immediately following a change in political leadership suggest that civil service appointments and departures are significantly influenced by political pressures.

Quarterly CS turnover rates plotted against changes in political leadership

Performance

In some countries, transfers, rather than dismissals, are the favored device for politicizing the civil service. Accordingly, quarterly civil service transfer rates that spike immediately following a change in political leadership suggest that civil service appointments and departures are significantly influenced by political pressures.

Quarterly CS transfer rates plotted against changes in political leadership

Performance

Quarterly turnover rates of political appointees plotted against changes in political leadership should exhibit larger spikes after changes in political leadership than is the case for civil servants.

Quarterly turnover rates of political appointees plotted against changes in political leadership

Performance

2. Merit-based, performance-encouraging personnel management

Staff actually work.

High absenteeism is a sign that staff are not adequately motivated to work.

Average absentee rate within a given cadre of staff.

Performance

Competition in recruitment and selection

Competitive recruitment and selection procedures enhance transparency, fairness and the odds of merit-based personnel management practices

% of vacancies filled through advertised, competitive procedures

Performance

Effective performance evaluation practices

Performance evaluations are a necessary but not sufficient condition for merit-based CS management practices that link

% of CS staff for whom annual performance evaluations were

Performance

13 02/05/2008

Objective Rationale Indicator Type: Determinant (r-

indicator) or Performance (Y-

indicator

some rewards to performance. completed

Variance in performance evaluations is a necessary but not sufficient condition for an effective performance evaluation process.

% of CS performance evaluations falling in each rating category

Performance

Continuously weed out poor performing staff

As CS management practices improve, poorer performing civil servants should exit the CS at non-trivial rates, thereby improving average quality of CS incumbents over time.

% of civil servants receiving the lowest performance rating in two successive years who have left the CS within the following year.

Performance

3. Attract and retain qualified staff

Attract qualified staff

Increases in average CS total remuneration relative to average economic sector wages suggest increasingly competitive (and hence, attractive) CS remuneration.

Average CS total remuneration as a % of average economic sector wages

Determinant

A CS salary structure that yields a consistent ratio of CS to private sector comparator salaries across Titles enhances capacity to recruit and retain qualified staff within all CS skill sets.

Ratios of average CS to private sector total remuneration by Title

Determinant

As CS positions become more attractive, the average number of qualified applicants per advertised CS opening should increase.

Average number of qualified (long-listed) candidates per advertised CS opening

Performance

Retain qualified staff

Low annual turnover rates of recent recruits suggest that staff find public employment attractive enough to remain within the public administration.

Average 3-year turnover rates for recent recruits.

Performance

Minimal variance in annual turnover rates among recent recruits across types of positions (e.g., Titles, urban vs. rural locations, etc.) suggests that employment packages are consistently attractive across

Variance in title-specific 3-year turnover rates among recent recruits.

Performance

14 02/05/2008

Objective Rationale Indicator Type: Determinant (r-

indicator) or Performance (Y-

indicator

types of positions; while, large variance in those same turnover rates suggests that the types of positions with higher turnover rates are less attractive than those with lower turnover rates.

A higher vertical compression ratio10 provides a reasonable indicator of opportunity for salary growth over a CS career.

Ratio of average Secretary General total remuneration to average Junior Officer total remuneration

Determinant

4. Fiscally sustainable wage bill

Civil service wage bill is fiscally sustainable

CS wage bill as a fraction of GDP should be consistent with Government’s fiscal program

Actual CS wage bill as a percentage of GDP.

Performance

Budget-financed wage bill is fiscally sustainable

Overall budget-financed wage bill (covering not just the civil service, but all budget-financed public employees) should be consistent with Government’s fiscal program

Actual budget-financed overall wage bill as a percentage of GDP.

Performance

FYR Macedonia and Albania are each monitoring “Actionable HRM Indicators”, from which many of the entries in Table 2 have been drawn. Albania has been monitoring such indicators since early 2000, while FYR Macedonia began monitoring more recently. Albania’s CS reform leaders have, from time to time, employed their monitoring data as evidence when they have made a case to the Prime Minister or the Government for particular interventions. For instance, in the early stages of implementation of Albania’s civil service reform effort, the entity charged with ensuring effective implementation of the CS Law, the Department of Public Administration (DoPA), noticed a significant increase in requests from Ministers for exemptions from the competitive recruitment procedures mandated by the CS Law. DoPA’s Director used data on that increase successfully to make a case for imposition of regulations that would make it more difficult to justify such exemptions. Similarly, in the early stages of Albania’s CS reform effort, a survey of public and private sector salaries was employed by DoPA to develop a proposal, which was ultimately adopted, for a new CS salary structure, which would ensure consistency in the competitiveness of CS salaries across types of CS positions. Finally, evidence on a rising incidence of qualified applicants per advertised CS position

10 Ratio of average total remuneration for staff in the highest rank to average total remuneration for staff in an entry level position.

15 02/05/2008

in Albania has helped to convince doubters about the efficacy of Albania’s competitive recruitment and selection procedures.

Determinants HRM AGIs (r-indicators) Tools have also been developed to measure what many HRM experts view as key determinants of HRM system performance. A good example of a well-defined and comprehensive set of such r-indicators is the HRM diagnostic instrument developed in the Europe and Central Asia (ECA) Region in the late 1990s, which is included as Annex 3. That instrument includes multiple indicators of “good practice” within each of six broad evaluation dimensions of a country’s HRM system:

A Legal and ethical framework B Institutional framework C Employment and pay policy management D Human resource management policies and practices E Training and career development F Management practices and culture

That instrument was applied in a number of candidate countries for EU accession, and proved to be quite robust. It is now being applied in the East Asia and Pacific (EAP) Region as well.

16 02/05/2008

Annexes

Annex 1: Summary of AGI Features and Exclusions

Key Features of AGIs • Track impacts that can actually be detected within a relatively short time span. • Capture extent to which the immediate objectives of specific institutional reforms

are being achieved. • Are well defined (i.e., reasonable people can agree on precisely what a given AGI

is measuring), while allowing that disagreement may exist regarding the empirical importance or normative implications of a given indicator.

• Are narrowly enough circumscribed and clearly enough defined that they facilitate deliberations about what sorts of actions might be helpful for improving the performance of a particular element of one of the above five governance dimensions.

Exclusions: What AGIs Are Not • They are not broad indicators of the quality of an entire dimension of governance.

• They are not unbundled, constituent elements of such broad indicators of governance

17 02/05/2008

Annex 2: What Distinguishes AGIs from Other Types of Indicators? Consider the process of trying to improve some particular element of a given dimension of governance. The Bank supports such efforts all the time. Projects aimed at strengthening public financial management are an example, as would be a project supporting civil service reform. Similarly, a Country Assistance Strategy (CAS) could include both analytic and operational work aimed at ensuring that a given country establishes more effective transparency and accountability requirements, through, for instance, enacting more sensibly designed asset declaration requirements, conflict of interest laws and freedom of information requirements. In any such reform process, there will be a need to monitor both implementation and results of the reform effort. Projects typically focus their monitoring efforts on inputs, activities and outputs; i.e.., the first two of the standard sets of indicators noted above. A CAS, on the other hand, is more likely to focus its monitoring on a mixture of outputs (e.g., laws passed) and outcomes. But it is rare for reformers or donors, such as the Bank, to design indicators that systematically monitor the “missing middle” identified above; i.e., the quality of particular elements of given governance dimensions (the y array noted above) and the factors that impact those qualities (the r array noted above).

AGIs vs. input and output indicators

Input and output indicators for such governance reform efforts track inputs employed, actions taken and products produced as part of that effort to improve the functioning of some specific element or elements of one or more governance dimensions. For example, a public financial management reform effort might include activities such as the following:

• design and install a computerized financial management information system (action; product),

• revise an organic budget law (action; product) • establish a new accounting system and chart of accounts (action, product) • strengthen the capacities of an internal audit unit by providing training to its staff

(action)

Most governance reform efforts, and World Bank-financed projects in particular, regularly monitor indicators of such activities, what inputs are employed to do this, at what cost, and what products are produced by those actions. This is important for managing such reform efforts, but it is not sufficient for determining whether such a reform effort is actually making progress on the underlying governance improvements being sought by those actions.

A key reason for this is that the impacts of most governance reforms depend not solely on those inputs, actions and products, but also on other factors, such as the ways in which those actions are undertaken, and whether those actions and their timing are sensibly adjusted during implementation to address constantly changing challenges to the reform effort (e.g., events which occur with some probability, rather than which can be anticipated with certainty during project design). In short, the agents responsible for implementing a reform agenda can, and often do, undertake the programmed actions in

18 02/05/2008

ineffective ways – e.g., in a pro forma fashion, or without adequately tailoring their actions to unpredictable sources of resistance if and when such resistance arises, etc.

AGIs provide an additional tool for addressing this risk, since they can provide systematic evidence on whether the reform agenda has been implemented in a way that actually furthers the underlying objectives. As such, they provide an important complement to input and output indicators, which are an important tool for managing a governance reform effort, but are not well tailored for ensuring that reform activities are actually contributing to the core objectives of the reform effort. We explain below in the main body of this note two distinct ways that AGIs can be designed to address this need.

AGIs vs. outcome indicators

Governance outcome indicators, on the other hand, focus on the final impacts of a country’s governance institutions on one or more of the five dimensions of governance identified above, or on political, social or economic phenomena about which citizens care. Examples of the former include some of the WBI governance indicators11, many of the summary indicators produced within the Global Integrity Index12, the summary indices included in the World Bank’s Country Policy and Institutional Assessment (CPIA)13, etc. Typical examples of the latter include indicators that focus on corruption and its myriad manifestations; e.g., the WBI governance indicators14, various indicators available from the Bank’s Business Environment and Enterprise Surveys (BEEPS)15, as well as Transparency International’s Corruptions Perceptions Index16, etc.

Governance outcome indicators, however, provide very little guidance on why a given country is performing well or poorly on any given governance dimension. They don’t shed light on which elements of each dimension are working well or poorly. Nor do they provide evidence on why any particular element of a given governance dimension is working well or poorly. AGIs are designed precisely to drill down to the elements and sub-elements of each governance dimension, and even further, to the various features of a given element or sub-element, so as to shed light on both which elements or sub-elements and what features of any given element or sub-element are working well or poorly. When coupled with evidence on context, as well as inputs and outputs, AGIs can facilitate research on how particular aspects of context, inputs and outputs of governance reform efforts interact and contribute to the performance of a particular element or sub-element of a given governance dimension.

11 Kaufmann, Daniel; Kraay, Aart; and Massiimo Mastruzzi, Governance Matters VI: Governance Indicators for 1996-2006 (The World Bank: Washington, DC, July 2007). http://web.worldbank.org/WBSITE/EXTERNAL/WBI/EXTWBIGOVANTCOR/0,,contentMDK:20771165~menuPK:1866365~pagePK:64168445~piPK:64168309~theSitePK:1740530,00.html 12 http://www.globalintegrity.org 13 http://siteresources.worldbank.org/IDA/Resources/73153-1181752621336/CPIA06CriteriaA2.pdf 14 Kaufmann, Daniel; Kraay, Aart; and Massiimo Mastruzzi, Governance Matters VI: Governance Indicators for 1996-2006 (The World Bank: Washington, DC, July 2007). http://web.worldbank.org/WBSITE/EXTERNAL/WBI/EXTWBIGOVANTCOR/0,,contentMDK:20771165~menuPK:1866365~pagePK:64168445~piPK:64168309~theSitePK:1740530,00.html 15 http://web.worldbank.org/WBSITE/EXTERNAL/WBI/EXTWBIGOVANTCOR/0,,contentMDK:20711397~menuPK:1928482~pagePK:64168445~piPK:64168309~theSitePK:1740530,00.html 16 http://www.icgg.org

19 02/05/2008

20 02/05/2008

Annex 3: HRM Diagnostic Tool Matrix 1: Evaluation of Civil Service Human Resource Management Ref Good

Practice Criteria

Score Performance Indicators Used and Benchmark Information

Evaluation Summary Action Required Technical Assistance

A. Legal and Ethical Framework Score: Desirable characteristics Specific legislation governing the civil service, with subsidiary legislation and/or regulation that elaborate rules/procedures/systems for personnel

management The behavior of civil servants and political appointees, including probity considerations, is governed by a Code of Conduct Merit-based rules and procedures for civil service personnel management The scope of the civil service is clearly defined Impartial treatment of citizens, transparent civil service operations and policies

Is there a civil service law? A1 The civil service is governed by specific legislation.

Are there subsidiary regulations describing procedures?

A2 The scope of the Civil Service is clearly defined.

Does the law define which categories and types of staff are civil servants?

21 02/05/2008

Matrix 1: Evaluation of Civil Service Human Resource Management Ref Good

Practice Criteria



A3 There is a code of conduct or equivalent framework that governs the behavior of civil servants.

Is there an adequate Code of Conduct or equivalent framework for Civil Servants?

Is explicit political activity prohibited by law?

Is there a clear distinction between political appointees and career civil servants?

A4 The civil service is politically neutral.

Is the ratio of political appointees to civil servants similar to good practice? E.g.: U.S., 1:394; Sweden, 1:2051; U.K., no political appointees

22 02/05/2008


Practice Criteria



Are there statements on the need to disclose conflicts of interest, which are in accordance with good practice?

A5 There are legal provisions to ensure probity, such as requirements to disclose conflicts of interest, and which agencies should enforce these provisions.

Are conflicts of interest inspected formally?

A6 The notion of merit based procedures established in law.

Is there an overall statement on the civil service operating on the basis of an individual’s merit? All EU countries recognize the principle of a civil service operating on the basis of merit.

23 02/05/2008


Practice Criteria



A7 Citizens have access to fair and transparent means of redressing grievances with the civil service.

Is there an Ombudsman provided for in law?

Is civil service policy and performance widely published?

A8 Open access to information.

Do civil servants have access to their personnel files? Civil servants have open access to their personnel files in the EU countries for which information was available

A9 Is there a data protection act? Average

Score

B. Institutional Framework Score: Desirable characteristics There are effective, dedicated institutions for civil service policy, management and oversight, with officially recognized legal status and clearly

defined role and responsibilities Independent oversight bodies ensure fair play in the civil service Existence of accountability and recourse mechanisms for citizens, employees, the legislature, and the executive

24 02/05/2008


Practice Criteria



Does the civil service law define all of the relevant institutions required to manage the civil service? Structures vary across EU; function rather than form, is essential

Is there a single institution to formulate civil service human resources policy? Single central institution formulates policy and establishes framework for CS management

Does the single institution have sufficient authority to ensure policies are adopted and complied with? Key government and civil society actors agree on policy and management organs for CS

Does the single institution have sufficient resources to ensure policies are developed and complied with?

B1 There are clearly defined institutions to develop and co-ordinate civil service policy.

Do line ministries Human Resources Departments have appropriate capacity in terms of numbers of staff and skills?

Is there a body with the mandate to undertake independent decisions on the fairness of civil service policies?

B2 There is an oversight body* to ensure fairness of

Does the oversight body have sufficient authority to enforce decisions?

25 02/05/2008


Practice Criteria



Is the body governed independently from civil service management and political institutions? Diverse high-prestige government and non-government composition of oversight body(ies).

Does the mandate of the oversight body cover all relevant policies?

Does the oversight body report on its activities and outcomes?

Does the oversight body have monitoring and evaluation capacities to monitor civil service management performance? Resources allocated to oversight tasks match scope of required activities, and capacity to monitor CS behavior installed

civil service policies. (*or bodies)

There is evidence of effective institutions (such as Legislative Committees, Judicial Reviews, National Audit Office) and published reports. Evidence of active operations, such as public hearings, audit reviews, published reports

Average Score

26 02/05/2008


Practice Criteria



C. Employment and Pay Policy and Management Score: Desirable characteristics Civil service wage bill is affordable and within overall fiscal framework, and does not crowd out other necessary expenses Remuneration is sufficiently competitive to recruit, retain, and motivate qualified staff at all levels The compensation system is simple, monetized and transparent, with rule- and/or market-based determinants of compensation The establishment control system is in place and linked to well-functioning computerized payroll and personnel information system to provide

adequate budget control of personnel expenditure Pensions system is sound and affordable

C1 Numbers of civil and public servants are in line with international practice.

Is the ratio of all public sector employees per head of population in line with EU member states? EU range: 2.7% (Greece) to 17.4% (Sweden); median 7.1%; average 7.7%

27 02/05/2008


Practice Criteria



Is the wage bill as a proportion of government expenditure in line with EU member states? EU members wage bill expenditures: As % of goods and services: range: 24.6% (Germany) to 82.2% (Greece); median 67.4%; average 58.9% As % of total expenditure: range: 5.6% (Sweden) to 29.7% (Portugal); median 11.2%; average 13.2% As % of total revenue: range: 5.7% (Sweden) to 36.2% (Greece); median 11.5%; average 14.9%

Are goods and services as a proportion of government expenditure in line with EU member states? EU members goods and service expenditures as % of total expenditure: range: 14.2% (Sweden) to 96.0% (Switzerland); median 19.1%; average 22.5%

Is there any evidence of overstaffing in the public sector?

Is there any evidence of shortages of goods and materials, which are affecting administrative performance or service delivery?

28 02/05/2008


Practice Criteria



Is the ratio of civil service to private sector pay in line with EU or OECD norms? Annual gross average earnings public sector relative to private sector: range: 1.02 (U.K.) to 1.64 (Portugal)

Are turnover rates comparable to EU norms? EU turnover rates: range: 0.9-1.2% (U.K. management-level staff) to 10% (Sweden)

C2 Levels of pay are sufficiently competitive to recruit, retain and motivate qualified staff at all levels.

Is remuneration at senior levels appropriate to skills, as indicated by compression ratios? EU compression ratios: range: 4.6 (France) to 9.8 (U.K.)

Is the grade structure sufficient to provide incentives to advancement without introducing undue complexity/non-transparency?

C3 There is a uniform and effective grading structure that optimizes decision making.

Is there evidence of grade creep to circumvent salary constraints?

29 02/05/2008


Practice Criteria



Are % of non-wage allowances and supplements in line with OECD ranges? Base salary as % of total remuneration in 19 OECD countries: 100% in 9 countries; 70-100% in 6 countries; 50-100% in 1 country; <30% in 3 countries

Is there a system of post-based establishment control?

Are regular reports produced to confirm accurate employment information, and credible patters of intake and attrition?

Is a computerized system operating to support establishment control?

Is the establishment system Year 2000 compliant?

Are there well defined lines of technology and systemic lines of communication between the Personnel Department and Budget Department in the Ministry of Finance?

C4 The Compensation system is simple, monetized, transparent and fair.

Is there predictability and containment of personnel costs in the budget?

30 02/05/2008


Practice Criteria



Is the construction of the pensions system in line with best practice?

Desirable characteristics include: “stable, reliable, sufficiently flexible to adjust to economic, demographic and other changes… easy to understand and administer… perceived as fair and there is a clear relationship between performance – as measured by earnings – and pensions. Benefits and costs are reasonable and predictable.” (Sigma Policy Brief No. 2, “Civil Service Pension Schemes”)

Is the pensions system based upon sound actuarial practice?

C5 There is a pensions system that is affordable, effective, and fair.

Is there independent audit of the pension fund and system?

Average Score

D. Human Resources Management Policies and Practices Score: Desirable characteristics Capacity exists to review current and projected staff resource requirements An appropriate personnel information system is in place Staff selection is conducted on the basis of merit, and promotion is based on open and transparent merit-based procedures and collective

management decisions A performance appraisal system is in operation with uniform objective criteria across the service, review at different management levels, and

feedback to individuals Disciplinary procedures based on transparent principles, and institutional procedures for judgement and appeal exist

31 02/05/2008


Practice Criteria



D1 There are effective planning arrangements for deployment of civil servants.

Is there a national planning strategy and procedure for deployment of civil servants throughout the structure?

Can a system be demonstrated that registers basic personnel information for the entire civil service, containing personal characteristics such as education and skills, personnel actions, appraisal results, training and career history?

Are Human Resources Practitioners trained in the use of Personnel Management Information Systems?

D2 Effective Personnel Information systems are in place.

Is the Personnel Management Information System used effectively in decision making?

Are all posts advertised publicly to ensure equal competition for posts?

Is there a competitive process including objective assessment for recruitment?

Are job descriptions prepared and utilized?

Job descriptions are required for each post, under the labor code. Usage varies.

MOI should provide technical assistance to line agencies in preparation and use of job descriptions.

D3 Recruitment is based upon merit.

Is the recruitment process checked by multiple reviewers?

32 02/05/2008


Practice Criteria



Are there consistent practices across the civil service?

Are performance appraisals undertaken on a regular basis for all staff? E.g., Belgium/Germany all employees evaluated every 1-2 years; Netherlands/U.K. each employee evaluated annually

Is there a formal unified performance appraisal system? Evidence of written appraisals for last two years available for review? EU countries’ appraisals kept on file, with open access to employees

Are categories of appraisal similar to best practice?

D4 Performance evaluation is objective, uniform, and focussed on performance improvement and career development.

Is the appraisal linked to other human resources procedures?

Is there a uniform process, with legal basis, for making promotion decisions?

Are the procedures open and transparent? Results of promotions published?

Is the process used consistently across the civil service?

Is the process overseen by a third party?

D5 Promotion is open and transparent, and based upon merit.

Is there recourse to appeal for promotion grievances?

33 02/05/2008


Practice Criteria



Are there common objective criteria for disciplinary action?

Are statistics held on the number of disciplinary hearings and their outcomes?

D6 There are disciplinary procedures based upon transparent and fair principles.

Is there a written process for conducting disciplinary hearings?

Average Score

E. Training and Career Development Score: Desirable characteristics Appropriate training system to re-skill staff and to prepare new recruits for ongoing work, with a budget and clear institutional responsibility

assigned for training policy formulation and implementation Is there clear responsibility for civil service training policy?

Is there a national training policy based upon training needs analysis?

E1 There is an appropriate training system to re-skill staff to prepare new recruits and develop existing staff.

Is there institutional capacity for developing training policies at different levels?

34 02/05/2008


Practice Criteria



Are there specific training budgets identified in the state budget?

E2 Training budgets are sufficient to implement the training policies, within fiscal constraints.

Is the level of training expenditure consistent with international practice?

Is there a sufficient range of public and private sector providers to compete for training provision?

Training provision is sufficient to meet the requirements of the civil service.

Have options been considered for contracting out training provision?

Are there defined procedures for evaluating training courses?

E3

Training evaluation is undertaken to

Does training evaluation examine relevance of training, and cost effectiveness?

Varies by ministry.

35 02/05/2008


Practice Criteria



identify the relevance and cost effectiveness of training.

Are trainees utilized and well-placed in the civil service upon completion of training?

Unable to assess.

Average Score

F. Management Practices and Culture Score: Desirable characteristics Decision-making structures ensure effective citizen-oriented service management and delivery, with delegation schemes that are similar to other EU

civil services Are budgets devolved to decision makers?

F1 Decision making is placed at the lowest possible level to ensure effective management and service delivery.

Are there clear mechanisms for delegating decisions and holding individuals to account for their performance (in terms of outputs and outcomes)?

36 02/05/2008


Practice Criteria



Are standards published for public services?

Yes. General service standards (response times for inquiries) are published in the Administrative Code while standards for individual services are set in regulations issued by individual ministries. For example, since passport issuance is handled by the Police, MOI regulations stipulate the three terms for issuance with corresponding fees. (e.g. the basic term is 4 weeks at a cost of 100 crowns, with 1 week and 24 hour issuance also available for set, higher fees). Compliance with standards for common services such as passport and drivers license issuance is reported to be timely, while more problems arise with less common services (e.g. land cadaster offices, patents and trademarks).

Monitor and publish performance relative to published standards.

F2 Service Delivery is based upon public consultation regarding priorities and standards of service.

Are standards based upon public consultation?

In some areas, such as high visibility passport-issuance, standards likely reflect popular demand e.g. for express service. No information available about other areas.

Public consultation in standard-setting and evaluation of performance relative to standard should be increased.

Technical Assistance to help develop customer service monitoring.

Average score

37 02/05/2008

38 02/05/2008

Gary J. Reid C:\Documents and Settings\wb51420\My Documents\Ops\PRMPS\Actionable indicators\Actionable Governance Indicators Concept Note.doc 01/10/2008 5:00:00 PM