Impact evaluation of Climate Change interventions

Dr Virinder Sharma,

DFID India

Current state of play on IE of CC and Environment interventions

• 2008 Review- Adaptation: evaluations based on qualitative methods

» Stakeholder interviews» Lack baselines» Not integrated into project

• Mitigation: Desk reviews, interviews and short visits

• Lack of counterfactual analysis• Exceptions: PES, Protected areas and

Forest conservation

Challenges of conducting IE of CC

• Environment- Long term lag between intervention and impact ; Environmental & welfare outcomes

• Adaptation indicators and success?• Focused on CB & governance processes• Mitigation -how to assess impact on welfare and the

adaptation co-benefits

Possible approaches:• Use behavioural change as a proxy• Use indicators of development effectiveness to assess

impact on adaptive capacity

Determinants of adaptive capacity

Adaptive capacity is determined by access to and control over livelihood resources

Access and control over these resources are determined by policies, institutions and power structures

SL framework: Determinants of adaptive capacity

Livelihood resources

Examples

Human Knowledge of climate risks, conservation agriculture skills

Social Women’s savings and loans groups, farmer-based organizations

Physical Irrigation infrastructure, seed and grain storage facilities

Natural Reliable water source, productive land

Financial Micro-insurance, diversified income sources

Policies, institutions and power structures

Sustainable Livelihoods Framework: 14 Household vulnerability indicators

DFID RL projects: IE Methodology Objective: Comprehensive Impact Evaluation based on qualitative and quantitative analysis

Whether there is change, what contributed to change? Is is attributable to the project?

Design: Counterfactual analysis; Mix Design and Cross-sectional; difference in difference estimation

Instruments: HH schedule; FGD checklist, MoP, LIMES

Sample: Random selection of intervention/control groups.

32 MWS Project, 16 MWS Control

Project 1400 hh (64 villages), Control 700 hh (32 villages)

WBR category-wise distribution of households

Well Off9%

Manageable22%

Poor39%

Very Poor30%

Sample Profile Caste-wise distribution of the households

0.0

10.0

20.0

30.0

40.0

50.0

60.0

Schedule Caste Schedule Tribe OBC Others

% h

ouse

hold

s

Project

Control

BPL - 67% (P), 62% (C) ST- 49% (P), 29% (C)

Socio-economic, Climate/Environment outcomes

Increase in Livelihood Assets

Increase in Livelihood Assests

42.2 43.053.4 52.0

45.6 46.4 51.7 54.741.5 40.1

47.8 48.8

40.8 39.1

40.5 39.7

33.8 29.8

35.5 32.045.2 47.5 35.2 32.6

17.0 17.96.1 8.3

20.6 23.812.8 13.3 13.3 12.4 17.0 18.6

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Poor VeryPoor

Poor VeryPoor

Poor VeryPoor

Poor VeryPoor

Poor VeryPoor

Poor VeryPoor

Natural Physical Financial Social Human Total

% h

ouse

hold

s

Low Significant Highly Significant

Socio-economic, Climate/environment Outcomes Improvement in disaster coping capacity

Category Project (%) Control (%)

Marginal farmers 74 66

Small Farmers 75 77

Medium Farmers 88 88

Factors for improved disaster coping capacity

44

4

6

5

4

23

35

28

5

5

6

5

20

27

29

26Enhanced agriculture production

Drought proofing

Increased fodder availability for cattle

Increased access of drinking water for household & cattle

Increased availability of drinking water for household & cattle

Enhanced food availability from Grain Bank

Diversification of livelihood activities

Access to consumption credit

% marginal farmers

Control

Project

Critical reflection of IE in CC and Development programmes

• Climate Change: local vs. macro impacts regional/global) • CCA and development: Uncertainty, risk management,

vulnerability and resilience (transformative?)• Participation approaches (Low income) vs. heterogeneity• Political economy/governance issues• Application to Government programmes-

Physical and Financial achievements vs. Inputs & outputs

• Outcome budgeting (indicators) outcomes-impact• Results Framework Document: success indicators• Evidence of effectiveness of funds/interventions; What

works- what doesn’t- circumstances-cost• Positive examples: Watershed, NREGA/Green Jobs

Potential Impacts of Climate variability in Western Orissa: drought

Potential Impacts of Climate variability in Western Orissa- sand casting/flash floods