How to measure governance:How to measure governance:Principles, frameworks, Principles, frameworks,

data collection techniques data collection techniques and challenges for Iraqand challenges for Iraq

Shipra Narang SuriShipra Narang SuriInternational Consultant, OGCInternational Consultant, OGC

Stakeholders’ Consultative WorkshopStakeholders’ Consultative WorkshopAmman 6-8 July 2010Amman 6-8 July 2010

Principles and frameworks for Principles and frameworks for governance assessmentsgovernance assessments

The concept of Governance: The concept of Governance: What are we talking about?What are we talking about?

Focuses on the relationships between the Focuses on the relationships between the State, civil society and private sectorState, civil society and private sector

Key aspects:Key aspects:• Governance is not government, a crucial Governance is not government, a crucial

distinction between the two being the notion of distinction between the two being the notion of civil society civil society

• Governance emphasises the processes of Governance emphasises the processes of decision-making, as much as, or even more decision-making, as much as, or even more than, the decisions themselves or their than, the decisions themselves or their outcomes outcomes

UNDP definition of GovernanceUNDP definition of Governance

Governance is the system of values, Governance is the system of values, policies and institutions by which a policies and institutions by which a society manages its economic society manages its economic political and social affairs through political and social affairs through interactions within and among the interactions within and among the state, civil society and private sector.state, civil society and private sector.

- UNDP Strategy Note on Governance for Human - UNDP Strategy Note on Governance for Human Development, 2000Development, 2000

The normative debate:The normative debate:What characterises “good” or What characterises “good” or

“democratic” governance?“democratic” governance?

ParticipationParticipation RepresentationRepresentation Rule of lawRule of law Transparency and accountabilityTransparency and accountability Effectiveness and efficiencyEffectiveness and efficiency ResponsivenessResponsiveness Equity….Equity….

Some global governance Some global governance assessment frameworksassessment frameworks

UNDP/IDEA: Framework of Democratic Governance

UN-HABITAT: Urban Governance Index Transparency International: National

Integrity System (NIS) World Bank: Governance Matters index ODI: World Governance Assessment

Some regional examples… [1]Some regional examples… [1]State of Local Democracy (SoLD)

in the Arab World

Based on International IDEA’s State of Democracy Approach

Implemented in 4 countries - Egypt, Jordan, Morocco and Yemen

Themes - Representative democracy (electoral processes, political institutions), Participatory democracy (participatory processes, civil society)

Some regional examples… [2]Some regional examples… [2]Palestinian Governance Survey

Main themes – Rule of law; Public sector employment; Elections; Performance of the parliament; Performance of the government; CSO performance; Access to information; Political parties; Women rights; Education, Health, Labour and Housing

Data collection – Household questionnaire Sample – 2544 households Users – Government, parliament, CSOs,

international agencies

Some regional examples… [3]Some regional examples… [3]Arab Democracy Index

Conducted twice in Conducted twice in ten Arab countriesten Arab countries, latest , latest results 2010results 2010

The study measures The study measures forty indicators forty indicators to gauge to gauge four major values and principlesfour major values and principles: : • strong and accountable public institutionsstrong and accountable public institutions• respect for rights and freedomsrespect for rights and freedoms• the rule of lawthe rule of law• equality and social justiceequality and social justice

Data gathered for the Arab Democracy Index cover Data gathered for the Arab Democracy Index cover three different dimensionsthree different dimensions: : • the legal aspectthe legal aspect• public opinionpublic opinion• practices of regimespractices of regimes

Data sources include administrative data and Data sources include administrative data and sample population surveysample population survey

Data collection Data collection techniquestechniques

Data collection strategies for Data collection strategies for governance indicatorsgovernance indicators

Mix of qualitative and quantitative, Mix of qualitative and quantitative, de factode facto and and de jure de jure datadata

A variety of approaches:A variety of approaches:• Comprehensive governance assessment Comprehensive governance assessment

approaches based on multiple approaches based on multiple stakeholder perspectivesstakeholder perspectives

• Governance assessments based on Governance assessments based on citizen (or single stakeholder) citizen (or single stakeholder) perspectivesperspectives

• Self-assessments by government Self-assessments by government institutionsinstitutions

Primary data collectionPrimary data collectionPrimary data

Household/ Individual Surveys

Focus Group Discussions

Face to face

Thematic

Stakeholders

Key informant interviews

Mail

Telephone

Internet

Secondary/ published data Secondary/ published data collectioncollection

Secondary data

Governmental sources

Non-governmental sources

National ministries/ departments

States

Local governments

International Organisations, Donors

International NGOs

Local NGOs

Private sector

Advantages and constraints of Advantages and constraints of primary data collectionprimary data collection

Advantages Direct stakeholder

involvement Captures opinions,

perceptions, experience

Can help in mobilising stakeholders for action

ConstraintsConstraints ExpensiveExpensive Time-consumingTime-consuming Requires significant Requires significant

local capacity in local capacity in survey design, survey design, sampling, execution sampling, execution and analysis and analysis

Advantages and constraints of Advantages and constraints of secondary data collectionsecondary data collection

Advantages Published data may

be more easily available

Cost-effective Can be collected in a

limited time through desk study

ConstraintsConstraints Mostly Mostly de jurede jure data data May not reflect May not reflect

actual situation or actual situation or multiple multiple perspectivesperspectives

Needs verification, Needs verification, triangulation for triangulation for meaningful results meaningful results

Some examples…Some examples… VietnamVietnam

Public Administration Performance Index – Examined six different dimensions of public administration and public services

Based on views of ordinary citizens as primary users of public administration services

Other stakeholders such as private enterprises or civil servants excluded

Civil society organizations also excluded Data obtained from a representative selection

of ordinary citizens, rather than from household-heads, in an effort to know about the perception from across the population, including gender and age-differentiated groups.

Some examples… Some examples… UgandaUganda National Service Delivery Surveys (NSDS) –

conducted (2001, 2004, 2008) Mainly collected information from a

sample of 10,000 households Also included a module for service

providers Supplemented by data on National

Integrity Surveys (1998,2003, 2008) Key role for national statistical

organisation

Some examples… Some examples… Sri LankaSri Lanka Urban Governance Index (UGI), conducted

2004-05 in six cities 25 indicators in 4 categories – equity,

effectiveness, participation, accountability Information gathered mainly from

published data Verified through consultative workshop

with active participation of governmental and non-governmental stakeholders

Challenges in post-conflict Challenges in post-conflict environmentsenvironments

Poor governance often the root of the Poor governance often the root of the conflict (finding the right entry points)conflict (finding the right entry points)

Social fragmentation and lack of trust Social fragmentation and lack of trust Need for skilled facilitation and negotiation Need for skilled facilitation and negotiation

(finding the right partners)(finding the right partners) Lack of political will and commitment Lack of political will and commitment

(finding committed interlocutors)(finding committed interlocutors) Low interest and capacity within Low interest and capacity within

government or statistical agencygovernment or statistical agency Low interest and capacity within civil Low interest and capacity within civil

society society

Challenges in post-conflict Challenges in post-conflict environmentsenvironments …[2]…[2]

Limited availability of data and Limited availability of data and information information

Limited financial resources for “soft” Limited financial resources for “soft” issuesissues

Indicator/assessment fatigue (especially Indicator/assessment fatigue (especially those driven by external actors)those driven by external actors)

Issues of sustainability and replicabilityIssues of sustainability and replicability

Challenges in post-conflict Challenges in post-conflict environmentsenvironments …[3]…[3]

Consulting the community….Consulting the community….• What community? What community? • Why engage the community? Why engage the community? • Who represents the community?Who represents the community?• Who NOT to consult?Who NOT to consult?

How to consult?How to consult? How to we avoid legitimising particular How to we avoid legitimising particular

ethnic/nationalistic/communal ethnic/nationalistic/communal agendas?agendas?

Thinking strategically in post-Thinking strategically in post-conflict situationsconflict situations

Be selective, rather than comprehensiveBe selective, rather than comprehensive Focus on priorities, spatial and sectoralFocus on priorities, spatial and sectoral Be flexible and responsive to changing Be flexible and responsive to changing

needsneeds Emphasise both process and product, be Emphasise both process and product, be

participatory and inclusiveparticipatory and inclusive Incorporate concerns of environmental Incorporate concerns of environmental

sustainability, gender, poverty and sustainability, gender, poverty and exclusion, formal and informal sectorsexclusion, formal and informal sectors

Thank you for your attentionThank you for your attention