1 Indicators based on representative surveys of firms, households and service providers Javier Herrera (IRD - DIAL) Mireille Razafindrakoto (IRD - DIAL),

11

Indicators based on representative Indicators based on representative surveys of firms, households and surveys of firms, households and

service providersservice providers

Javier Herrera (IRD - DIAL)Javier Herrera (IRD - DIAL)

Mireille Razafindrakoto (IRD - DIAL), and François RoubaudMireille Razafindrakoto (IRD - DIAL), and François Roubaud (IRD - (IRD - DIAL)DIAL)

Seminar on “The Empirics Of Governance“Seminar on “The Empirics Of Governance“

May 1 – 2, 2008, Washington DCMay 1 – 2, 2008, Washington DC

2

PRESENTATION SCHEME

DIALDéveloppement, Institutions et Analyse de Long terme

I. Uses of household surveys

II. Taking into account sources of bias

III. Objective vs subjective corruption indicators

IV. Conclusions & Perspectives

3

I. Uses of household surveys

• Household survey as a « voicing » instrument; public awareness on corruption (particularly in of authoritarian political contexts).

• Complex links with policy

4

Historical perspective

First estimation of the extent of corruption in 1995 in Madagascar (with Household survey)

Headlines in the press: « Outcry against corruption! »

It then became impossible to ignore the problem

5

1995 It then became impossible to ignore the problem

the Ministry of Justice took steps to introduce a system of sanctions. a draft law on the fight against corruption

BUT Draft law rejected by the Government Council in 1999 Importance of context & characteristics of institution (~ authoritarian regime and problem of governance)

2002Institutional change Stress put on transparency 2003 Creation of an independent council (CSLCC) 2004 Independent anti-corruption office (BIANCO)

“ … It is important to bear in mind that, on the basis of the statistical survey conducted in May 1995 by the MADIO project, co-financed by the French Ministry for Cooperation and the European Union: ‘the issue of corruption seems to be a recurring problem which haunts the capital’s inhabitants. 96% of them consider that it is a major problem in Madagascar’. ‘Over 40% of people aged over 18 in the capital had had to pay a corrupt civil servant during the previous year’. Whatever the credibility of this survey and the interpretations that have been drawn from it, there is no doubt that corruption is a social problem in Madagascar … and it is necessary to combat the practice of corruption as energetically as possible” .

MEASURE1995 1998

DEFINITION OF POLICY

QUALITY OF INSTITUTIONS

/ Political Regime

Positive Impact

Interactions

6

Headlines in the press in may 2005: « More confidence & less corruption »

after the public conference presenting the first results of the survey on « governance, democracy and fight against poverty »

7

Sources : Razafindrakoto, Roubaud (2002) and 1-2-3 Survey 2002-2004, INSTAT, DIAL, authors calculations.

An improvement of civil servant wages or an active anti-corruption policy

a sharp drop in the incidence of corruption

Civil servant salaries and corruption levels in Madagascar 1995-2004

GOVERNANCE

048

121620242832364044

1995 1996 1997 1998 1999 2000 2001 2002 2004

%

90

100

110

120

130

140

150

160

170

180

Bas

e 10

0=19

95

Level of corruption Civil servants real wages (right scale)

Political crisis

Active anti-corruption policy

8

II. Taking into account sources of bias

• Who answers the question matters (random sample selection within the household)

• But also it matters to know more about who don’t answer (non respondent profile)

• Taking into account the working of corruption (middlemen, social norms, etc.)

• Robustness needs to be assessed• Informal production units needs adapted

survey approach (the 123 surveys)

9

III. Taking into account sources of bias

• The wording and sequence of question matters (but not only when measuring corruption, ie. labor surveys). Pilot surveys are important.

• Non response to governance questions in HH surveys are lower than in standard questions (ie. Income)

• Unobserved heterogeneity (« over optimistic »/ »under optimistic », use panel data -ongoing research)

10

The surveys

Governance and Democracy Modules grafted onto official household survey conducted by National Statistical Institutes

Survey support (stratified multi-stage sample surveys: area/HH) for the Modules (Individuals):

• In AfricaIn Africa: 1-2-3 Survey (LFS, informal sector, consumption and poverty)

- Madagascar 1995-2004, Capital + urban areas (2000, 2001), rural, entreprises surveys (time series)

- West Africa 2001/2003, in 7 WAEMU capital cities (Abidjan, Bamako, Cotonou, Dakar, Lomé, Ouagadougou and Niamey)

35 594 persons interviewed

• In Latin AmericaIn Latin America: Standard National Household surveys (ECH, Bolivia; SIE-ENEMDU, Ecuador; ENAHO, Peru). National and regional inference for Peru and Ecuador.

- Peru 2002-, national level (18 000 HH sample in 2002; continuous survey from 2003 to date).

- Ecuador 2004, (20 000 HH)

- Bolivia 2004, national (1 700 HH).

-The survey is taking place in Colombia during the 2nd term of 2005.

More than 50 000 persons interviewed

11

Strong points / Basic principles:

Ownership

Reliability

Sustainability

Conducted by National Statistical

Institute

Integration in the National Statistical

System

Relevance

SystematicPresentation /

Publication of the result wider public

ValidationDemocratic debate

demand

Bottom-up approach Investment in capacity building

Light, flexible toolReconductible time-series

Marginal costHH survey « Voicing »

empowerment, accountability

Supply side Demand sideInteractions

12

Non response

There is (often) more willingness to answer questions on governance and democray than usual economic questions

Non response rate to selected questions

%West Africa Mada

Cotonou

Ouaga-

dougou

Abidjan

Bamako

Niamey

Dakar Lomé Antana-

narivo

Total

Opinion on governement functioning

0 3.6 0.9 1.1 2.7 5.2 3.5 0.1 2.2

Opinion on democracy 0 2.8 0.5 0.9 1.9 3.1 0.3 0.2 1.3

Income level (continuous) 40.2 54.3 40.9 43.3 51.6 56.3 37.2 47.3

Income level (continuous or discrete)

2.3 6.5 3.2 6.5 14.1 9.2 1.6 0.1

Sources : 1-2-3 Surveys, Phase 1, Governance and Democracy module, 2001/2003, National Statistical Institutes, AFRISTAT, DIAL, authors’ calculations.

13

RobustnessProbabilistic surveys allows a quantitative assessement of indicators’ precision (also relevant for group and intertemporal comparaisons)

rural urban Total

has increased 31,0 39,1 36,6

95% confidence interval [29,0 - 33,0] [37,4 - 40,9] [35,3 - 38,0]

has decreased 8,43 10,47 9,85


is the same 49,1 48,8 48,9


don't know 11,5 1,6 4,7


Total 100 100 100

In your opinion, corruption since last year?

Source: our estimation based on ENAHO july 2003-june 2004, INEI, Peru.

14

Taking into account the working of corruption: middlemen; failed corruption; “accepted” corruption, amounts paid, access to services; links with inefficiency…

Source: ENAHO 2004 HH survey Peru

15

Incidence and cost of corruption (2002/2003 and 2004 in Antananarivo)

En 2002/2003 En 2004

Total Income per capita Total Income per capita

1st quartile 4th quartile 1st quartile 4th quartile

Incidence (all households) 16,3 11,9 20,1 8,4 6,0 10,3

No contact with civil service 40,4 46,8 31,5 29,9 34,2 22,4

Incidence (households victims) 27,4 22,4 29,3 12,0 9,1 13,3

Total annual amount (1 000 Fmg)

Mean (HH victims of corruption) 381 219 682 300 77 484

Median (HH victims) 25 20 50 30 14 50

% of income (HH victims) 3,3 7,8 3,1 1,2 1,8 1,2

Source : Enquête 1-2-3, modules qualitatifs, 2003 et 2004, INSTAT, authors calculations.

Corruption incidence taking into account access to public services Incidence (percentage of households victims) has decreased Total amount paid has decreased in real terms The poor as well as the rich had benefited from this improvement

16

“Don’t know” answer is not random: it concerns mainly the poor, the rural, the less educated, the socially excluded

Importance of democracy

Do you think that having a democratic government is

important?:

Rural %

Urban %

Not poor %

Poor %

Total %

Yes 34.8 55.9 56.3 41.5 49.4 More or less 38.0 30.6 29.7 36.6 32.9 Not much 13.7 8.4 8.7 11.6 10.0 No 4.6 3.4 3.3 4.3 3.7 Does not know 8.9 1.7 2.1 6.0 3.9 Total 100.0 100.0 100.0 100.0 100.0 Authors’ own calculations from ENAHO 2002-IV.

17

Corruption incidence for the informal sector production units can be measured

Cotonou Ouagadougou Abidjan Bamako Niamey Dakar Lome Total

%Incidence of Informal Units that had a problem with civil servants last year

4.7 5.0 7.0 3.5 6.2 8.5 6.2 6.2

% that settled the problem:

By paying a fine 75.5 76.7 42.9 38.5 40.4 49.2 44.5 47.3

By giving a “gift” 13.9 5.7 50.2 32.6 24.7 38.1 45.1 41.3

other 10.5 17.6 6.9 28.9 34.9 12.7 10.4 11.4

Source: Alain Brilleau et al. « Le secteur informel : Performances, insertion, perspectives, enquête 1-2-3 phase 2 », STATECO n°99, 2005 p.82.

18

III. Objective vs subjective corruption indicators

• Expert’s surveys vs. HH surveys

• Do they measure the same phenomena? (petty vs. big corruption)

• Minding the gap between : -perceptions and objective

indicators -aspirations and perceived outcomes

19

The mirror survey

To round out the collection of surveys on Governance and Democracy in seven West African capitals and in Madagascar (Antananarivo),

an additional survey to get the opinions of a certain number of Southern and Northern experts (researchers, development practitioners, decision-makers, senior civil servants, politicians, etc.).

The aim --> to compare answers from the population surveyed in each country with the specialists’ point of view.

20

The mirror survey (…)

Two sets of questions:

- what the experts think the respondents answered on average. - their own answer to these same questions (“What is your personal opinion?”)

Questions :

- Specificity of the answers of the population / experts, specialists - Knowledge of Northern or Southern experts on what happens and on people’s thinking in the South

-Relevance and reliability of indicators based only on appreciation of panel of experts

Sample size : 250 experts (30 per country in average)

DIALDéveloppement, Institutions et Analyse de Long terme

21

How far can we trust the experts’ opinion on corruption?

Discrepancies between real extent of corruption and experts’ perception in Francophone Africa

On the level of corruption / To what extent corruption acceptable On the relative positions (rank) of the different countries

Sources: General Household Survey (35,594 persons interviewed; 4500 for each country in average); Expert panel survey or Mirror survey (246 persons surveyed; 30 experts for each country in average). * In Madagascar, results are drawn from the 2003 survey. Authors calculations.

0

10

20

30

40

50

60

70

Incidence of corruption

General population * (% of victims of corruption from HHhousehold surveys)

Expert panel (mirror survey)(what they believe could be thepercentage of victims of corruption)

HH Survey

Mirror Survey

Cotono

u

Ouaga

Abidjan

Tana

Bamako

Niamey

Dakar

Lom

é

Avera

ge

0

5

10

15

20

25

30

35

40

Population who believes that making bribe is acceptable

General population * (% from Household (HH) survey)

Expert panel (what they think could be the % of population whobelieves that making a bribe is acceptable)

Mirror Survey

HH Survey

The need for complementary approaches / indicators

22

0

10

20

30

40

50

60

70

80

90

100

%

Cotonou Ouaga Abidjan Bamako Niamey Dakar Lomé Tana

The six principles of democracy are:

Fundamental Respected

Perception of the main democratic principles by the population

Are they fundamental? Are they respected in the country?

Measuring the gap between aspirations and effectiveness as regards democratic principles

DEMOCRACY

Sources : Enquêtes 1-2-3, module Démocratie, 2001/2003, Instituts Nationaux de la Statistique, AFRISTAT, DIAL

authors calculations.

23

Measuring the gap between aspirations and effectiveness as regards democratic principles

DEMOCRACY

Lomé

0

50

100

Free,transparent

elections

Politicalfreedom (right

to choose party)

Freedom ofexpression, of

the press

Equality beforethe law

Freedom totravel

Religiousfreedom

FundamentalRespected

Dakar

0

20

40

60

80

100

Free, transparentelections

Political freedom(right to choose party)

Freedom ofexpression, of the

press

Equality before the law

Freedom to travel

Religious freedom

Fundamental

Respected

Antananarivo

0

50

100

Free,transparent

elections


to chooseparty)


the press


Freedom totravel

Religiousfreedom

Fundamental

Respected

Cotonou

0

50

100

Free,transparent

elections


to chooseparty)


the press


Freedom totravel

Religiousfreedom

FundamentalRespected

Ouagadougou

0

50

100

Free,transparent

elections

Politicalfreedom(right to

choose party)

Freedom ofexpression,of the press

Equalitybefore the

law

Freedom totravel

Religiousfreedom

Fundamental

Respected

Abidjan

0

50

100

Free,transparent

elections


to chooseparty)

Freedom ofexpression,of the press

Equalitybefore the

law

Freedom totravel

Religiousfreedom

Fundamental

Respected

Bamako

0

50

100

Free,transparent

elections


to chooseparty)


the press

Equalitybefore the law

Freedom totravel

Religiousfreedom

Fundamental

Respected

Niamey

0

50

100

Free,transparent

elections


to choose party)


the press


Freedom totravel

Religiousfreedom

Fundamental

Respected

Sources : Enquêtes 1-2-3, module Démocratie, 2001/2003, Instituts Nationaux de la Statistique, AFRISTAT, DIAL

authors calculations.

24

No correlation between Objective measure (incidence of corruption) Subjective perception of the working of civil service

EnsembleBamako

Dakar

Ouaga Abidjan

Tana

Cotonou

Niamey

Lomé

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

0% 2% 4% 6% 8% 10% 12% 14% 16% 18%

Incidence of corruption

% o

f n

egat

ive

op

inio

n o

n t

he

wo

rkin

g o

f th

e ci

vil

serv

ice

Cor=-0,48

The need for complementary approaches / indicators

25

The global lessons• Advantages of household sample surveys Transparency of measurement proceduresRepresentativenessQuantification, Comparability of indicators over time.

• In-depth policy-oriented analyses More appropriate than international indicators and aggregates.

• Both objective (behaviour, actual experiences) and subjective information (perception, satisfaction) Monitoring and relating the two fundamental dimensions of these phenomena.

• Socio-economic disaggregation These two dimensions can be combined with traditional variables related to the socio-economic characteristics of individuals and households (income, occupation, sex, age, ethnic group, etc.). Possibility to disaggregate information between different population categories (gender, poverty, ethnic groups, discriminated people, etc.

• International comparability

26

IV. Conclusions & Perspectives

• Measure different dimensions of governance

• A detailed understanding of different forms of

corruption is necessary to analyse its

determinants and consequences

• Different sources of potential bias should be

considered in designing & analysing surveys

• Both, objective and subjective, governance

indicators need to be collected to better

understand aspiration/outcome gaps and why

corruption may become or not a political issue.

27

END

For more works on Governance, Democracy and Poverty see our web site

http://www.dial.prd.fr

Our article:

“Governance, Democracy and Poverty Reduction: Lessons drawn from household Surveys”, International Statistical Review (2007), vol. 75, issue 1, pp.70-95

is available upon request

Documents

1 Indicators based on representative surveys of firms, households and service providers Javier Herrera (IRD - DIAL) Mireille Razafindrakoto (IRD - DIAL),