ADVISORY BOARD MEETING 2013 Doing a Forest Governance Assessment: Practical Tips and Tricks Ken...

ADVISORY BOARD MEETING 2013

Doing a Forest Governance Assessment:

Practical Tips and Tricks

Ken Rosenbaum, Sylvan Environmental Consultants

Today we will look at three topics

1. The simplest way to carry out an assessment: take a data collection tool that someone else has designed and adapt it to your needs.

2. How to design your own data collection tool (or modify an existing tool) if you can’t find one that suits your needs.

3. What are good practices in collecting forest governance data?

But first a brief recap from Webinar 1: What is a forest governance assessment?

• An assessment is an attempt to measure forest governance.

• Assessments can be used to diagnose problems, compare conditions, or monitor efforts to change.

FOUR WELL-DOCUMENTED TOOLS AIMED AT FORESTS OR NATURAL RESOURCES

• The PROFOR Diagnostic. • The USAID Sustainable Conservation Approaches in

Priority Ecosystems (SCAPES) governance assessment tool.

• The World Resources Institute’s (WRI’s) Governance of Forests Initiative Indicators

• The Indonesia Participatory Governance Assessment (PGA) for REDD+.

The PROFOR Diagnostic Tool

The PROFOR Diagnostic Tool

•Useful to diagnose problems, to identify priority areas for reform, or to set a baseline for future monitoring•Based on the FAO–PROFOR Framework•Uses a set of up to 130 indicators (English, French, and Russian versions available)•Scored by stakeholders in workshop(s)•Validated by key informants•Complete “How to” guide available on PROFOR.info website

The USAID SCAPES Tool

The USAID SCAPES Tool

•Developed by Wildlife Conservation Society for USAID•Designed to work on landscape level, looking at natural resource management generally•Based on its own framework for analyzing governance, around pillars of Legitimacy, Capacity, and Power•A workshop, bringing together representatives of stakeholders or other key informants or experts, identifies key groups and their influence, strengths, and weaknesses•Full how-to guide available on frameweb.org

The Indonesia PGA

The Indonesia PGA

•Developed in Indonesia with support of UNDP/UN-REDD as part of REDD+ readiness•Not presented as a tool for general use, but as a report on what Indonesia did•Highly participatory, steered by an advisory group of experts in close contact with stakeholders•Used 117 indicators•Gathered data locally and regionally, via combination of methods. including document reviews, key informant interviews, and focus groups.•Report available on undp.org site

WRI Governance of Forests Initiative

WRI Governance of Forests Initiative

•An approach piloted in Brazil, Indonesia, and Cameroon•Uses 122 indicators•WRI has published separate publications on (1) the indicators and (2) methods to score them•Can score via desk reviews, key informant interviews, focus groups, participant observation, testing of systems, etc.

I’d Rather Design It Myself!

1. Refine your Scope: Exactly What to Measure2. Identify Sources of Data3. Select Data Collection Methods4. Develop “Tools”: Interview Protocols,

Questionnaires, Sampling Plans, etc.5. Write out a Data Collection Plan or Manual

Refine your scope: What to Measure

•Didn’t we set the scope in our Work Plan? • Yes, and we’ll start with that, but we need more

detail now•Look at what other assessments have done•Decide how detailed you need to be•Decide whether to use a narrative description of scope or to use criteria and indicators•Set out your scope in writing

Identify Sources of Data

•Written materials: past assessments, official publications, unofficial government documents, laws, budgets, media reports, academic studies, etc. •People: officials, academics, experts, stakeholders, etc. that can be reached through interviews, focus groups, workshops, surveys, etc. •Physical evidence: Less commonly used in governance assessments, these can include forest site evaluations, testing of government functions, inspection of boundary markers.

Identify Methods

•Now you have a sense of where the information might be; how will you collect & validate it? •Desk reviews: a good way to tap existing documents; low cost; but no “new” information•Expert analysis: Can use multiple experts to add depth to desk reviews, provide opinion, validate other methods; but potential for biased experts. •Surveys: Produce lots of data; repeatable; but costly; may not be in-depth; may have bias•Key informants: may tap rich sources of information at low cost; but may be biased, not easily replicated

More methods

•Focus groups: More people=broader perspective than one-on-one interviews or surveys, but some less assertive people may not be heard.•Workshops: Even broader perspectives, promotes communication among stakeholders, but can be expensive; hard to get balance of stakeholders; requires planning and skill to extract information from participants.

Develop tools

•Desk reviews: will you gather existing data or perform new analysis?•Experts: What kinds of experts? What terms of reference? •Key Informants: How to select? Protocol for interviews? •Surveys: Sampling issues. Question design. •Focus Groups & Workshops: Sampling issues. How to convene? How to capture (code) responses

Get help from social scientists!

Write it down

•A data collection plan: for use of the managers of the data collection effort

• Maybe just a refinement of the work plan• If it is more, check against the work plan, including

the timeline and budget, and adjust as needed•A data collection manual: for use of the people in the field who are actually collecting data

• Not always necessary, but in a large effort, helps with consistency

Vet & validate your methods

Consult experts and stakeholders about your design effort.

Do it formally or informally.

Be transparent! Transparency now leads to greater credibility later.

Good Practices in Data Collection

•Assembling a data collection team•Going out an getting your data•Assuring data quality

Assembling a team

•This step varies, depending on methods and size of effort•May need just a few people, who take on multiple roles•May need managers, logistics coordinators, researchers/field people, data managers, etc. •Larger teams need more documentation, training•Team members should be trustworthy, capable, and unbiased

Collecting data

•Interviewing, facilitation, and survey administration are all teachable skills. Be sure your team is using good and consistent techniques. •Give careful thought to coding: your team should capture data in a complete and consistent manner. •Be aware of ethical issues:

• be transparent and truthful; • get informed consent; • respect privacy and confidentiality; • keep people safe from harm; • guard the integrity of the data.

Quality assurance

When data come in from the field:

•Edit: Be sure data entries or notes are complete and readable; get back to collectors if not. •Clean: Flag data that stands out or suggests an error in entry; look for missing or duplicated entries; etc. •Verify: Spot check that data were actually collected; also that data were not miscopied or garbled in transmission.•Triangulate: Confirm against other sources.

WHERE WE HAVE BEEN & WILL HEAD NEXT:

• Webinar 1 (January 20) covered the

definition of forest governance and how to plan an assessment. A recording is available online.

• Webinar 3 (26 March) will cover analyzing your data, making recommendations, and getting your results out to the right audiences. Also, we will talk about setting the stage for any assessments that might follow.

For More Guidance:

The FAO–PROFOR Framework: Search on the internet for “Framework for Assessing and Monitoring Forest Governance”

The PROFOR–FAO Guide to Good Practices: Search on the Internet for “Forest Governance Data Collection and Analysis”

THE REFORM CHALLENGE

“If you want to make enemies, try to change something.”

— Woodrow Wilson, President of the Untied States, speaking in 1916.

ADVISORY BOARD MEETING 2013

THANKS FOR LISTENING

For more information contact:

Nalin Kishor, Ph. DSr. Natural Resources EconomistPROFOR Forests Team, GENDRnkishor@worldbank.orgThe World Bank1818 H St., N.W. Washington, D.C. 20433 ' +1-202-473-8672www.profor.info