Measuring Conservation Impact Initiative World Parks Congress Durban, South Africa September 8-18,...

Measuring Conservation Impact Initiative

World Parks Congress

Durban, South Africa

September 8-18, 2003

Foundations of Success (FOS)

with

Wildlife Conservation Society (WCS)

Conservation International (CI)

funded by

The Moore Foundation

Topics

1. Provide a brief introduction to the work of Foundations of Success in general.

2. Describe the Measuring Conservation Impact (MCI) study.

3. Present some of the major lessons from that study.

4. Discuss current and future products.

Foundations of Success (FOS)Our Mission

To improve the practice of conservation by working with practitioners to develop and communicate tested knowledge about the conditions under which different conservation strategies and tools are effective.

FOS Objectives The “Foundations of Success”

Work with practitioners to:

1. Define clear and practical measures of conservation success.

2. Determine sound guiding principles for using conservation strategies and tools.

3. Develop and strengthen practitioner knowledge and skills in adaptive management.

Foundations of SuccessWhat do we do?

Learning Portfolios Adaptive management systems Neutral facilitation Training Applied research

Foundations of Success (FOS)

with

Wildlife Conservation Society (WCS)

Conservation International (CI)

funded by

The Moore Foundation

Measuring Conservation Impact (MCI) Initiative

Our Collective Inability to Define & Measure Conservation Impact Causes Many Problems

Difficulties in establishing priorities “Blind” management Obstacles to learning Problems in convincing donors and society

to support our work Many “competing” M&E approaches among

conservation organizations

Measuring Conservation Impact Review Objectives

Investigate history of M&E in various fields and synthesize lessons learned– Business & management

– Education & social services

– International development

– Public health

– Conservation

Develop framework for M&E in conservation Develop a database and computer-based decision tree for

conservation project indicators

M&E Publications Reviewed by Field (n > 350)

3559

76

52

99

0

10

20

30

40

50

60

Business &Mngmt

Education &Soc Serv

InternationalDvlpmt

Public Health& Pop

Envt &Conservation

Pu

blic

atio

ns

Re

view

ed

70

80

90

100

Conservation Family Tree

ClassificationSystems

BiologicalSurveys

ComplianceMonitoring

Eco-Certification

Organic-Certification

State of Env’tMonitoring

NationalReport Cards

Strategic EnvAssessment

Biodiv ImpactAssessment

Scorecards

LogicalFramework

AdaptiveManagement

Project CycleMngmt

Results-Based Mngmt

LearningNetworks

RapidAssessments

Env ImpactAssessments

PrioritySetting

PopulationMonitoring

Social ImpactAssessments

Prototyping

LogicalFramework

EconomicProduction

Econ WelfareIndicators

1700

1900

1970

1980

1990

2000

MEASURING EFFECTI VENESSACCOUNTING &CERTIFI CATI ON

STATUS ASSESSMENTBASIC

RESEARCH

Business & Management Family Tree

1500

1900

1970

1980

1990

2000

PROCESS IMPROVEMENTACCOUNTING &CERTI FI CATION

BalancedScorecard

CostAccounting

Activity BasedCosting

Double-EntryBookkeeping

EngineeringStandards

Auditing byAccountants

ProcessCertification

ScientificMngmt

Total QualityMngmt

Managing forResults

Six Sigma

ProcessReengineering

PerformanceBenchmarking

LearningOrganization

Communityof Practice

OutcomeEvaluation

OperationsResearch

ReflectivePractice

SocialLearning

STATUSASSESSMENT

Stock MarketIndex

EconomicProduction

Econ WelfareIndicators

OutcomeEvaluation

AccountingStandards

PerformanceManagement Chaordic

Systems

SocialCertification

Soc. WelfareIndicators

Education & Social Services Family Tree

AptitudeTests

ExperimentalStudies

CurriculumEvaluation

Outcome &Process Eval

Written SchoolExams

StandardizedObjective Tests

IntelligenceTests

Quasi-Experimental

Criterion-Referenced Tests

Utilization-Focused Eval

Benchmarkingfor Schools

PerformanceMeasurement

EmpowermentEvaluation

Project CycleMngmt

ScientificMngmt

PerformanceBenchmarking

ReflectivePractice

SocialLearning

MEASURING EFFECTIVENESSSTATUS ASSESSMENT

1800

1900

1970

1980

1990

2000

Theory-basedEvaluationGoal-free

Evaluation

International Development Family Tree

LogicalFramework

Project CycleMngmt

ExploratoryApproach

ExplanatoryApproach

CulturalAppraisal

ParticipatoryM&E Participatory

Logframe

CapitalApproach Results-

Based Mngmt

ProjectCycle

Outcome &Impact Eval

1800

1900

1970

1980

1990

2000

MEASURING EFFECTIVENESSSTATUS ASSESSMENT

EconomicProduction

National DevlIndicators

SocialBenefit-Cost

EfficiencyEvaluationEnv Impact

AssessmentsSocial ImpactAssessments

Public Health Family Tree

DescriptiveStudies

OperationsResearch

PerformanceMonitoring

Project CycleMngmt

1600

1900

1970

1980

1990

2000

MEASURING EFFECTIVENESSSTATUS ASSESSMENT

InterventionStudies Watchdog

Monitoring

Logic Model

LogicalFramework

BASICRESEARCH

ObservationalStudies

ACCOUNTI NG &CERTIFICATION

Large-ScaleStdzd Survey

RapidAssessments

Health RiskAssessment

Health ImpactAssessment

EfficiencyEvaluation

SituationAnalysis

Results BasedManagement

Certification /Accreditation

LESSON #1: Different M&E Needs Require Different M&E Approaches

1. Status assessment

2. Measuring effectiveness

a) Impact assessmentb) Adaptive management

3. Accounting and certification (compliance)

4. Basic research

Examples of These Types ofM&E Needs in Conservation

1. Status Assessment – Present State“How are things doing overall?”

2a. Measuring Effectiveness (Impact assessment)“What is the likely or actual impact of an activity?”

2b. Measuring Effectiveness (Adaptive mgmt) “How can we get better at our work?”

3. Accounting and Certification“Are we meeting pre-determined standards?”

4. Basic Research“How do we advance the frontiers of knowledge?”

LESSON #2: Trend towards integrating M&E into the broader project cycle

LESSON #2: Trend towards integrating M&E into the broader project cycle

LESSON #2: Trend towards integrating M&E into the broader project cycle

LESSON #2: Trend towards integrating M&E into the broader project cycle

LESSON #2: Trend towards integrating M&E into the broader project cycle

ImplementManagement &

MonitoringPlans

D

Develop aMonitoring

Plan

C

Develop aManagement Plan:Goals, Objectives,

& Activities

B

Start

Clarify Group'sMission

Iterate

Use Results toAdapt & Learn

EDevelop

ConceptualModel Basedon Local SiteConditions

A

AnalyzeData and

CommunicateResults

E

The AMProjectCycle

Lesson #3: Good M&E considers the context

Site A Site B

Which Site is Better Conserved?

Site A Site B

Lesson #3: Good M&E considers the context

Which Site is Better Conserved?

Site A Site B

Lesson #3: Good M&E considers the context

Which Site is Better Conserved?

LESSON #4: Good M&E tests assumptions

Inputs Activities Outputs Outcomes Impact

Humanorganization,financial, and

materialresources

Coordination,training, and

programmatictasks

Theimmediateproduct of

projectactions

Intermediateresult brought

about byprojectoutputs

Ultimateresult

attributed tocombinationof outcomes

achieved

Process Results

Generic impact chain

LESSON #4: Good M&E tests assumptions

Sustainableagriculturetechniques

Increaseyield/unit

labor

Reducedarea-planted

Reduceddeforest-

ation

Activity Output Outcome Impact

# of farmerswho adopttechniques

Outcome

Sample impact chain – sustainable agriculture

LESSON #5: Good M&E is diagnostic

Precisechain

Well-executedproject

Desiredresults

Used todesign

Leads to

Precisechain

Poorly-executedproject

Desiredresults

Does notlead to

Imprecisechain

Well-executedproject

Desiredresults

Does notlead to

ImpreciseChain

Poorly-executedproject

Desiredresults

Does notlead to

SUCCESS!

THEORY FAILURE

IMPLEMENTATION FAILURE

TOTAL FAILURE!

Used todesign

Used todesign

Used todesign

1. Different M&E needs require different M&E approachesa. Status assessmentb. Effectiveness measurementc. Accounting and certificationd. Basic research

2. For project management, evaluation should be integrally situated within the project cycle

Where do we go from here? Review of some lessons from the MCI project

Where do we go from here? Review of some lessons from the MCI project

3. Good M&E considers the context

4. Good M&E is predictive- Considers causal links

5. Good M&E is diagnostic – Tells why something did or did not work

Products From MCI Work

“Evolutionary” trees and synthesis of approaches in different fields

Archival database of M&E resources from all fields, including conservation

Synthesized framework (problem statement, lessons, and principles) for conservation

Interactive indicator decision tree and database

Available at…

www.FOSonline.org