Resources - Draft Evaluation Resources · Resources - Draft How to use the resources in this book Evaluation Resources serves as a toolbox of information and techniques to assist

Resources - Draft

Evaluation Resources

A toolbox for the design and implementation of an evaluation plan

A publication of

Ecosystem Management Initiative School of Natural Resources and Environment

The University of Michigan Ann Arbor MI 48109-1115

Page 2 Draft: Please do not copy or cite

Resources - Draft

11/17/2003 Draft sample pages

The following is a sample of pages from a preliminary draft of Evaluation Resources – a supplement to the Guide to the Design and Implementation an Evaluation Plan. It is a work in progress, as we are still developing accurate and useful frameworks of evaluation questions, indicators, tools and references. We welcome your feedback on this project.

Sheila Schueller [email protected]

The University of Michigan RESOURCES SAMPLE: Page 3 Ecosystem Management Initiative

mailto:[email protected]


Resources - Draft

How to use the resources in this book Evaluation Resources serves as a toolbox of information and techniques to assist you in planning and implementing an evaluation of your project. It is to be used together with the Guide, but can also serve as a reference to enhance many project activities. It includes:

1. Sample questions and indicators for key ecological, socioeconomic and process components of a project. These can be used to • identify and frame project objectives • identify threats and assets that affect a project’s ability to achieve desired outcomes • prompt ideas for evaluation questions related to outcomes, threats, or assets • choose indicators or measures to assess change over time or differences among

places and thereby answer evaluation questions

2. A general framework of strategies used by projects to achieve ecological, socioeconomic and process outcomes. This can be used to • identify project strategies • prompt ideas for evaluation questions related to strategy effectiveness (in

combination with questions about system components) or strategy implementation

3. Tools to help you through the evaluation process, including • a technique to prioritize objectives, threats, assets, or strategies for evaluation • a list of sources of available information or data • a list of useful references and web links by subject • a primer to statistical analysis • a glossary of terms used in the guide


Brief Table of Contents Note: Only some of the following sections are included – see Complete Table of Contents for details

How to use the resources in this book 5

EVALUATION QUESTIONS AND INDICATORS 11

How to use this section 11

ECOLOGICAL Evaluation Questions and Indicators 12

Overview 12

Key features of ecosystems 18

Threats to ecological systems 48

Assets that facilitate progress toward ecological goals 59

SOCIOECONOMIC Evaluation Questions and Indicators 62

Overview 62

Data collection tools for social outcome monitoring 62

Key features of social systems 63

PROCESS Evaluation Questions and Indicators 70

Overview 70

Key process issues 73

STRATEGY Evaluation Questions and Indicators 77

Overview 77

Strategies by category 77

Prioritization Tools 80

Sources of available data useful for evaluation 83

Additional resources: web sites and related materials 84

Glossary 85


Resources - Draft

Highlighted sections are those that are included as samples.

Complete Table of Contents

How to use the resources in this book 5

EVALUATION QUESTIONS AND INDICATORS 11

How to use this section 11

ECOLOGICAL Evaluation Questions and Indicators 12

Overview 12 Framing measurable objectives 13

Choosing ecosystem features (GUIDE A-1) 13 Choosing desired ‘levels’ of ecosystem features (GUIDE A-1 & D-1) 14

Evaluating progress towards objectives 16 Asking ecological evaluation questions (GUIDE B-1) 16 Selecting measures and indictors (GUIDE B-3) 16

Key features of ecosystems 18 LANDSCAPE 21

Composition 23 Extent 23 Representation 23 Arrangement 24

COMMUNITY 26 Composition 28 Representation 29 Structure 30 Dynamics 32

SPECIES 34 Population Demography and Dynamics 36 Genetics 38 Physiology 38

WATER 39 Physical Structure and Dynamics 40 Composition 41

AIR 42 SOIL 43

Composition 44 Structure 44 Dynamics 44

ENERGY AND NUTRIENTS 46 Energy Flow 47 Nutrient Cycling 47


Threats to ecological systems 48 Habitat loss, alteration or fragmentation 49 Local scale land alteration 51 Hydrology alteration 51 Disturbance regime alteration 52 Disruption of nutrient cycles 52 Incompatible resource use 53 Introduced species 54 Pollution 56 Sediment loading 57

Assets that facilitate progress toward ecological goals 59 Existing high quality/quantity ecosystem features 60 Existing programs or legislation compatible with ecological objectives 60 Economic incentives compatible with ecological objectives 60 Ecosystem features with public appeal 60 Existing human capital to implement strategies and/or monitor 61

SOCIOECONOMIC Evaluation Questions and Indicators 62

Overview 62

Data collection tools for social outcome monitoring 62

Key features of social systems 63 Local economies 63

Diverse and stable industries 63 Local investment, development and growth 63 Living-wage jobs 63

Quality of life 635 Social capital and community capacity 635 Community character 637 Opportunities for residents 63 Services and safety net 63 Social well-being 63

PROCESS Evaluation Questions and Indicators 70

Overview 70 Evaluating process can be simple 70 Identifying process issues for evaluation 70 How the resources section can be used 72 The role of process indicators 72 Selecting and answering your evaluation questions 72

Key process issues 73 Motivation 73

Shared sense of ownership or responsibility 73


Resources - Draft

Perception of interdependence 74 Shared sense of ownership or responsibility 74 Incentives created by alternatives to collaboration 74 Financial opportunities 74 Perception of possibilities 74

Organization 74 Outreach 74 Coordinator 74 Well-managed and open process 74 Creation and use of a management plan/framework 74

Resources 74 Facilitation & process management 74 Scientific expertise and information 74 Funding, staff and equipment 74

Adaptability 75 Joint learning grounded in credible science/information 75 Monitoring and assessment connected to decision making 76

Legitimacy 76 Effective representation of all key affected interests 76 Declining opposition; positive external perceptions 76 Accountability and ties to statutory decision making processes 76 Commitment of agency & political leaders to process and follow through 76

Energy 76 Dedicated energetic individuals 76 Process champions; social networks/relationships 76 Small successes/recognition 76 Dedicated resources 76

STRATEGY Evaluation Questions and Indicators 77

Overview 77

Strategies by category 77 Resource Management and Restoration 77

Land Acquisition/Protection 77 Species Reintroduction 77 Species Removal 77 Restoration/simulation of processes (disturbance, hydrology) 77 Alter/maintain Resource use practices 77 Leave-alone 77

Socioeconomic development 77 Marketing/Attract investment 77 Development of sustainable industries 77 Certification 77 Community events 77 Educational/vocational training opportunities 77

Law and Policy 77 Enforcement and watchdog 77


Litigation 77 Legislation and treaties 77

Education and Outreach 77 Curriculum development 77 Public Relations 77 Interpretation centers 77 Media 77

Information and Planning 77 Management Plan 78 Research 78 Inventorying/Identification 78 Monitoring 78 Review/Evaluation 78

Capacity Building 78 Partnerships/collaboration, trust-building 78 Skills development 78 Fundraising 78 Reorganization 78

Prioritization Tools 80

Sources of available data useful for evaluation 83

Additional resources: web sites and related materials 84

Glossary 85


Resources - Draft

EVALUATION QUESTIONS AND INDICATORS

How to use this section This section is split into Ecological, Socioeconomic and Process-related issues. Each section has an introduction and then a series of tables such as the one below.

Landscape attributes Sample Evaluation Questions

Sample Measures Sample Indicators (indirect measures)

COMPOSITION The number and type of different community, habitat, or vegetation types and topographic features of the landscape

What are the physical characteristics of the landscape? Are they changing in areas of land use?

Topographic relief, slope, aspect, elevation Shorelines, banks and channel attributes, see Water

Vegetation patterns

Specific topic - Feature or attribute for framing objectives (GUIDE A1)

Sample evaluation questions (GUIDE B1)

Sample measures and indicators (GUIDE B3)

Is the number of different community types increasing in the landscape?

# of different types presence/absence of certain types

Landscape-level biodiversity1


Resources - Draft

ECOLOGICAL Evaluation Questions and Indicators1

Overview Ecological outcomes such as improved ecosystem health, integrity or biodiversity are often the primary measures of success of resource management or conservation projects. These goals, while providing a vision or mission, are too broad to be directly measurable. Instead, ecological outcomes must be framed and measured in terms of several more specific features of an ecosystem. In this section we provide a framework of evaluation questions and indicators organized according to the parts and processes of ecosystems. The content of these tables can be used to frame measurable objectives and to evaluate progress towards goals.

1 For more information on ecological goals and measures of success, see the EMI literature review on which this section is based: Schueller, S.K. and E. McCance. 2003. Ecological goals for ecosystem management: What are we trying to achieve and how will we know if we are getting there? url for pdf on emi eval website

Resources - Draft

Framing measurable objectives To frame your project’s objectives in such a way that you can measure progress towards them, you must consider two components of an objective: 1) the part or feature of the ecosystem that you are interested in, and 2) the desired amount, level, range, or quality of that ecosystem feature. Choosing ecosystem features (GUIDE A-1) As you can see in the series of tables on ecosystem attributes that follow, there are many parts and processes of ecosystems, and making separate objectives for each of them can quickly become overwhelming. Overarching goals, such as increase biodiversity or maintain ecosystem integrity, are generally associated with all or almost all ecosystem features (see below). Therefore, your project’s focal ecosystem features should reflect a mixture of overarching goals and local issues. Overarching goal

Conceptual meaning or associated ecosystem properties

Associated ecosystem features

Biodiversity The variety and variability of living organisms at difference levels (from genes to species to communities); also the ways in which they interact with each other and with the physical environment

Landscape and community composition and representation; but also processes that create and maintain biodiversity, such as: landscape arrangement, community structure and dynamics, and water and soil condition

Health Free from ecosystem illness. Relatively stable (maintains some sort of organized or structured state) and resilient (able to recover to its original state after disturbance).

A mix of ecosystem features chosen to reflect influential ecosystem parts and processes, plus threats to ecosystem health

Integrity Whole or complete; having all the parts and processes that are characteristic or natural for that region.

All ecosystem features

Sustainability A balance between ecosystem use and protection; ecosystems need to be used in such a way that health, integrity or biodiversity are maintained over long periods of time.

All ecosystem features, plus threats related to destructive or excessive use of ecosystems


Ask yourself these questions to select ecosystem features for goals and measuring progress:

• Which of the overarching goals above matches your project mission? • What is of public interest – what do they want or need to know in order to

understand and/or support project activities? • Which ecosystem elements are locally critical and/or are unique features of your

project area (e.g. wetlands, spawning habitat, old growth forest, rare plants)? • Are there particularly influential ecosystem functions or processes (e.g. fire,

flooding, predation, stream productivity, etc.)? • Which ecosystem features are most vulnerable or most affected by stressors to the

system (e.g. connectivity of natural areas within the landscape, the chemical composition of the water, the health of individual species)?

You may be concerned not only about desired states or conditions of ecosystem features, but also about reducing threats or stressors to your system. For common threats and associated evaluation questions and indicators, see the next section on ecosystem THREATS. Choosing desired ‘levels’ of ecosystem features (GUIDE A-1 & D-1) To make your objective measurable, it must contain some description of desired state (quality) or amount (level or quantity) of the ecosystem feature of interest. To help you determine this, consider two key issues: a. The range of natural variability

Ecosystem features such as the rate of nutrient cycling or number of species will depend on the system. More is not always better. For example, nutrients in a grassland are generally cycled more quickly than in a forest, and bog species diversity is lower than fen diversity. To determine what is ‘right’ for a particular system in way that considers site-specific characteristics, many resource managers are increasingly relying on the “range of natural variability.” This approach, which was developed through the collaboration of scientists and practitioners, is useful not only for setting ecologically realistic goals, but also having meaningful benchmarks against which to measure progress.

Natural variability encompasses two principles: a) past or historical features of an ecosystem give an idea of what is appropriate or “natural” for that system, and b) variability, which includes both spatial heterogeneity and change over time, is an integral feature of all ecosystems. The key to using natural variation as guiding management principle is that the goal may not always be to increase the amount, rate or level of an ecosystem component (as goals are often stated, e.g. increase biodiversity, increase productivity), but to bring system components within the range of what is ecologically appropriate, based on historical information, for that area.

A variety of data types and sources of information can be used to determine the range of natural variability, including:


Resources - Draft

• Long-term data sets from permanent plots, forest inventories, or stand records. (Especially available if your site is on or near a Long-Term Ecological Research program sites – see http://lternet.edu/sites/)

• Historical records or paleo-ecological information (ask local universities) such as palynology, packrat middens, climate records, or dendroclimatology.

• Expert opinion of ecologists or naturalists familiar with that system • Photographs, writings, observations or experiences of long-term or previous residents

b. Realism

Just as social goals of management cannot fall outside the range of what is ecologically possible, ecological goals cannot be set without considering societal needs and land use expectations. A purely ecological approach to management or one that attempts to eliminate human influences, from the system, will not be successful because it is unrealistic. Also consider the time frame in which you aim to achieve objectives. What level or amount of change from current conditions is feasible for your project to accomplish? Example of development of measurable objectives using tables of ecosystem features Goal Ecosystem feature Measurable Objective Maintain biodiversity Representation of communities

across the landscape

At least 50 acres of each of the 3 major community types in the area

Community composition Increased native species richness relative to 1990

Restore viable populations of salmon

Landscape representation of critical salmon habitat

10% of project area is salmon habitat

Population size 500 salmon adults per spawning event


http://lternet.edu/sites/

Evaluating progress towards objectives Asking ecological evaluation questions (GUIDE B-1) As you are developing your evaluation plan you will need to identify what it is you need to know about your system. Some of these evaluation questions will relate to your goals. You will also want to track certain ecosystem features that are not an explicit part of your goal, but relate to project strategies or to the reduction of threats to the system. Skim through the “Sample Evaluation Questions” to get ideas. Questions can and should be combined with other questions – e.g. “Is the population viable?” and “How has land use changed?” can be combined to ask “Are populations more or less viable in areas of land use change?” Selecting measures and indictors (GUIDE B-3) Because of the nature of ecosystems, all of the attributes listed below are directly or indirectly related, and yet no single measure is a sufficient indicator of the system as a whole. When choosing a set of ecological indicators for evaluation select measures that:

1. are from both upper (ecosystem/community) and lower levels (individual/population) of ecological organization,

2. reveal something about what is there (elements or composition) as well as how it is working (dynamics, processes or function), and

3. include indicators (i.e. indirect measures or surrogates) and direct measures of an ecosystem.

By using a combination of direct and indirect measures you can achieve a better understanding of the system and determine why features are not moving in the direction desired. Indicators are actually indirect measures of ecosystem features. They provide information about multiple aspects of a system without requiring their direct measurement, thus saving time and money. However, interpreting changes in the value of an indicator may not be possible without knowing something about the underlying processes or elements. For example, density of vegetation may provide an indication of soil quality, but the cause of a decline in vegetation could not be determined unless direct measures of soil fertility – e.g. available nitrogen – were also made. If both direct and indirect measures are made then it can be deduced that declines in vegetation are not due to a decline in soil fertility, but something else, such as an increase in herbivore populations. Consider also that biological conditions alone are insufficient indicators of physical or chemical changes, because organisms may not respond until years after changes in environmental conditions.


Resources - Draft

Measures of ecosystem features can also be used simply to monitor change in ecosystem features of interest (without a desired target level or objective for that feature). In this case, measurements should still be associated with a question – one that relates to increased understanding of the system. For example, how are different species and processes related or how quickly and how much does an ecosystem feature change over time? Answering questions like can help determine the range of natural variability. If data collection is not associated with specific questions, it is likely to simply accumulate in file cabinets, without efforts paying off by finding needed answers to questions.


Key features of ecosystems There are many possible ways of dividing up an ecosystem into measurable chunks. We could split ecosystems into:

1. biological components (all living things), and 2. abiotic environment (all nonliving things, from light and water to concrete).

However, these two are closely linked by the exchange of energy and matter, and so it does not make sense to consider them separately. Alternatively, we can focus on different levels of ecological organization:

1. individuals, 2. populations, 3. species, 4. communities, and 5. landscapes.

Again, these are linked to their physical environment, but they are also linked to each through various ecological and behavioral interactions. To account for the overlap among the parts of ecosystems, some research-practitioners instead consider ecosystems in terms of three primary attributes:

1. composition – the identity and variety of elements (e.g. # and type of species in a community),

2. structure – the physical organization or pattern of those elements (e.g. proximity of habitat fragments in the landscape), and

3. function – the ecological and evolutionary processes linking those elements (e.g. nutrient cycling).

These terms are frequently used on different levels or scales (e.g. community composition, landscape structure, ecosystem function), but are not always easily communicated to the public. How are ecosystem features organized here? We take an approach that combines the useful features of the above schemes with the way practitioners commonly organize their goals and management plans on the ground – by measurable aspects of ecosystems:


Resources - Draft

• Landscape

o Composition - how many and which communities make up the landscape o Extent – area of the landscape and the communities within it o Representation – relative abundance of different community types o Arrangement – physical layout of communities in the landscape

• Community o Composition – how many and which organisms make up the community o Representation – relative abundance of different species or species groups o Structure – arrangement and physical features (layers, non-living habitats, etc.)

of a community’s biota o Dynamics – activity and change over time within a community (species

interactions, succession, disturbance) • Species

o Population demography & dynamics – vital statistics (population size, number of females, juveniles, etc.) and their change over time

o Genetics – the presence of certain genes or the level of genetic variation o Physiology – individual health of organisms

• Water o Physical structure & dynamics – water flow and storage and its relation to

habitat creation o Composition – chemical and biological aspects of water

• Soil o Composition – its biological and physical makeup o Arrangement – the structure of soil components o Dynamics – soil fertility and movement

• Air2 • Energy and Nutrients

o Energy flow – ecosystem productivity o Nutrient cycling – input and outputs from system and their link to fertility

NOTE: There are endless lists of possible measures of a given ecosystem feature. These tables are not meant to be exhaustive, but representative. They present a sample of the wide array of existing indicators and measures, especially those that may be more practical and relevant with respect to evaluation of ecological progress. Technical terminology is provided to facilitate further research or inquiry into a subject.

2 Measures of air condition are not included in this section, because generally we are concerned not with the condition of the air (i.e. we do not measure or manage oxygen levels), but in preventing threats to its condition – pollutants – so see the next section on ecosystem threats.



Resources - Draft

ECOLOGICAL questions and indicators sample

LANDSCAPE A landscape is a typically large area that encompasses a collection of communities or ecosystems (e.g. jack pine forest, tall grass prairie, and serpentine barrens), or land cover types (e.g. agriculture, urban development, and forest). These types can be classified broadly or more finely depending on the evaluation question (various classification schemes exist3). Changes in the type, size, and arrangement of patches in the landscape can determine the ability of species to persist in the area by affecting processes such as the spread of fire, the movement of individuals, and the availability of required resources. Because the integrity of individual communities (covered in the next section) will depend to a large degree on their context, it is important to evaluate landscape-level changes, even if this means obtaining information about land or waters outside of the project’s jurisdiction. The framework of indicators below includes four inter-related attributes of a landscape:

1. Composition 2. Extent 3. Representation 4. Arrangement

3 See existing standard systems for classifying community type ( http://www.fgdc.gov/ and http://www.esa.org/vegweb/)


http://www.fgdc.gov/

http://www.esa.org/vegweb/

Composition is simply what kinds of communities are in the landscape. This can be important for assessing changes in the basic make-up of the landscape (identifying whether desired types are absent or new types arise from land use or succession), for measuring landscape-level diversity, and as a surrogate measure of species richness. The Extent of the landscape’s communities is important because species richness is directly related to the available habitat area (but shape gives meaning to area – see Arrangement). This informative attribute is also relatively easy to measure. It is, however, not a sufficient measure of ecological success, because of the large effects of shape and arrangement of the viability of any area (see Arrangement). Combining information on composition and area, you can assess Representation, or the relative contribution of different community types to the total landscape area. A key indicator of progress towards goals may be the relative area of, for example, forest vs. pasture in the landscape. You may be particularly concerned about the relative area of certain types of communities, such as:

• Critical habitat for sensitive species or species of interest • Ecologically sensitive or critical zones (e.g. riparian buffer zones, floodplains, vernal

pools, breeding grounds) • Hotspots (areas with high species richness) • Special geographic features (bedrock glades, interior beaches, open cliffs) • Rare/imperiled, endangered types • Important habitat/community types for regional or global biodiversity

Arrangement describes how patches are distributed in the landscape – what is their shape, how close together are they, what is at their boundaries? Knowing the shape and proximity of patches gives meaning to the measure of area. For example, 10 acres of fen meadow may be distributed as 10 1-acre patches separated by 10 miles each or as 2 adjacent 5-acre patches. Similarly, a patch that is 10 miles long by 2 miles wide has the same area as one that is 4 by 5 miles, but the latter has more core area. DATA SOURCE: Common sources for this large-scale assessment of systems include: GIS data (see the Federal Geographic Data Committee http://www.fgdc.gov/ ), vegetation maps, field surveys (such as those of the Natural Heritage Program), satellite images, aerial videography, etc.


http://www.fgdc.gov/

Resources - Draft

LANDSCAPE attributes

Sample Evaluation Questions


Is the number of different community types increasing in the landscape?

# of different vegetation types presence/absence of certain community or ecosystem types

Landscape-level biodiversity4

Composition The number and type of different community, habitat, or vegetation types and topographic features of the landscape What are the

physical characteristics of the landscape? Are they changing in areas of land use?

Topographic relief, slope, aspect, elevation Soil texture and variability Shorelines, banks and channel attributes; see also Water

Vegetation patterns

Extent The area of the landscape and communities within landscape

What is the area of the total landscape? of each community type within the landscape? Are we increasing the protected area?

# of acres included in project # acres of different community types area of habitat for selected species

Representation Relative area of different community or ecosystem types within the landscape

How well are we representing different community types? What is the relative area of natural communities?5

# of patches of each habitat type Size of the largest patch of each type Relative area of different types (e.g. % open space) Natural cover type Index6 Dominance Index

Presence of indicator species characteristic of certain community or ecosystem types

4 TNC estimates that by representing a variety of natural communities, 85-90% of species can be protected without having to manage species individually – a “coarse-filter” approach (Noss 1987 in Christensen et al 96). 5 For a tool to assess representation of vegetation classes on the landscape level see: http://www.wec.ufl.edu/coop/exceltnc/


http://www.wec.ufl.edu/coop/exceltnc/




Arrangement Structure or layout of communities within the landscape

Have resource management practices increased the connectivity of the landscape?7 Are patches of the same type and/or different types well-linked?

Number, area, and length of corridors between patches Total area vs. number of patches Distance between patches Barriers to dispersal (see fragmentation in “Threats” framework) Length of habitat adjacent to non-natural habitat Contagion index (measure of clumping or aggregation of similar land types) Mean, minimum, maximum distance between patches

Persistence of populations in patches for indicator species sensitive to fragmentation - amphibians - migratory birds Presence of wide-ranging species Presence of specific gene stocks among populations (indicates gene flow) – see Genetics Loss (export) of nutrients from patches Water flow among patches

6 EMAP Western Pilot 7 It is important to consider the connectivity of aquatic as well as terrestrial habitats. Aquatic biota depend on the wetlands and floodplains that link rivers, lakes, or estuaries.


Resources - Draft




Are acquisitions improving the shape of the area by increasing core area relative to edge? Does the shape of areas more closely match natural patterns?

Area to perimeter ratio Elongation Core area Fractal dimension (measure of complexity of patch shape based on perimeter-area measures)

Abundance of characteristic edge species

Arrangement, cont. Is the quality of the intervening habitat (matrix) increasing?8 Are created corridors more effective than the previous matrix?

Degree of ecological dissimilarity between matrix and patches

Observation of different types of species moving between patches; using corridors

Overall landscape condition

Has the overall quality of the landscape increased?

(see above)

# of self-sustaining populations of indicator species that are representative of that area and require a large continuous range

8 There is a good example of on-the-ground efforts to consider and improve the quality of the matrix in the Interagency Committee plan for conserving the Northern Spotted Owl. (Thomas, J.W., Forsman, E.D., Lint, J.B., Meslow, E.C., Noon, B.R., and Verner, J. 1990. A conservation strategy for the northern spotted owl. Report of the Interagency Scientific Committee to address the conservation of the northern spotted owl. Portland, OR.


ECOLOGICAL questions and indicators sample COMMUNITY Ecological communities are characteristic collections of species that are closely linked by their interactions (e.g. competition, pollination, predation) and exchange of energy and nutrients. In the landscape-level assessment (above section) you may have chosen indicators that relate to how much there is of any one community type (extent and representation), and one aspect of community quality: arrangement in the landscape (i.e. does it have a large core area, is it adjacent to a ‘harsh’ environment). In this section we provide additional indicators to assess the quality of a habitat or community.

How to prioritize ecological communities for evaluation

If there are many different communities within your project area and your evaluation resources are limited, consider which of your communities may merit closer evaluation because they are:

• Areas containing one or more imperiled, threatened, and/or endangered species • Important breeding grounds for wildlife • Habitat for economically important species • Critical habitat for sensitive species or species of interest • Ecologically sensitive zones (e.g. riparian buffer zones, floodplains, vernal pools) • Communities that are regionally and/or globally significant to biodiversity • Special geographic features (bedrock glades, interior beaches, open cliffs) • Rare/imperiled, endangered types (see list in Christensen et al. 96 Table A1) • Biodiversity hotspots (with especially high species richness) • Rare plant communities (e.g. plant communities covering less than 3% of the land area)

The framework of indicators below includes four inter-related attributes of a community:

1. Composition 2. Representation 3. Structure 4. Dynamics


Resources - Draft

Composition is a measure of how many different species (species richness) and what kinds of species make up the community. Often there are too many species to inventory all of them and assessment of composition may instead involve measuring the number and type of species within certain groups, or treating whole groups as the unit of measurement. Overall community composition can indicate the impact of resource use and management and can provide a tangible measure of success – for example, an increase in the occurrence of certain species types. NOTE: Whereas species number is a common measure of community condition, knowing which species are present is critical to evaluating success. An increase in species number may be due to the arrival and spread of new exotic species or an undesired change in disturbance levels. Representation captures whether certain species are more dominant than others in the community. Measures of representation – including relative abundance and species evenness – require some knowledge of how many individuals there are (or % area covered) of each species. Again, this may be feasible only for certain species groups.

DIVERSITY includes both an assessment of how many different species here are (species richness) and how many individuals there are of each species (species evenness).

The Structure of community describes the arrangement and physical features of a community’s biota. This includes not only their physical layout (e.g. the presence dead trees, the layering of vegetation), but also their distribution within the food web (e.g. the abundance of predators vs. plants). Structure can be just as critical to the condition of the community as its species. Community Dynamics captures species interactions and processes such as succession and disturbance) that characterize a functioning or viable community.

How to chose focal species groups for measures of community condition

Consider monitoring the composition and representation of the following species groups that indicate aspects of community condition:

• Representative or characteristic species (e.g. tall-grass prairie plants, breeding birds) • Non-native (alien, exotic) species (see also Threats section) • Habitat generalists vs. specialists • Species with particular functional roles (e.g. pollinators, decomposers, soil-stabilizing

vegetation) • Species with particular habitat requirements (e.g. breeding birds, amphibians, shade-

tolerant plants)


COMMUNITY attributes

Questions Measure Indicator (indirect measure)

Composition How many and which species

Has the number of species of this community increased?9 The number of native species? Are characteristic fen plants present?

- Species inventory - Total # of species

(species richness)

Diversity estimate based on species-area curves # of species within certain guilds or taxonomic groups (e.g. bird, mammal, overall vegetation10, or understory plant diversity) Index of biotic integrity for freshwater; benthic index for coastal waters

9 It is key to consider native species here and not just totals, because exotic species may contribute to total numbers but not to the goal of maintaining native ecosystems. For measures of non-native invasion see “Threats” Framework. 10 Vegetation usually provides a good surrogate for overall biodiversity (Scott et al. 1993 in Christensen et al. 96)


Resources - Draft



Representation Relative abundance of different species

Relative to the range of natural variability how appropriately are different types of species represented in the community? Do native species make up a greater proportion of the community than before? Are invasive species more dominant or rare?

Relative # of individuals (or percent cover or biomass for vegetation) per species (species evenness) Dominance index

Floristic Quality Index Relative abundance (or dominance/rarity estimate) of certain species or species groups (see box).

Is the native diversity of this community changing?

Diversity index11 (that combines richness and evenness information), such as Simpson’s Diversity index (D), Shannon index (H), the Berger-Parker index (d), Hill's N1, and Q-statistics.

Diversity index of a certain guild

11 Each of these has strengths and weaknesses so often it is best to use a combination of indexes to measure diversity. For more info on estimation of biodiversity see the series of papers in Philosophical Transactions: Biological Sciences, Vol. 345, No. 1311, Biodiversity: Measurement and Estimation, Jul. 29, 1994




Structure Arrangement and physical features of biota

Is the structure of the food web12 closer to what is considered natural for this area?

Food web complexity: # of trophic (feeding) levels or guilds

Presence/absence of top predators and dominant herbivores Relative proportions of different functional feeding groups (useful especially for aquatic invertebrates)

12 Food web or trophic structure refers to how organisms are distributed across different feeding levels (producers, herbivores, primary carnivores, etc.). Measures of trophic structure also provide valuable information on ecosystem processes, such as productivity and energy flow, which are more difficult to measure directly and yet are very sensitive to resource use changes.


Resources - Draft



STRUCTURE, cont. Has the physical structural complexity of the community increased? Does the current structure fall within the range of natural variability? Is species richness higher in areas with more complex structure?

Presence of living or non-living structures important as species habitats or for nutrient cycling (e.g. in forests: logs, woody debris, snags13; in streams: large woody debris and presence of macrophytes such as eel grass beds, marsh grasses Size and arrangement of vegetation gaps (light map) % cover or % openness Ratio of water to emergent vegetation (wetlands) Index of canopy closure14 Number and relative abundance of different life forms (e.g. shrubs, trees, herbs) Number of strata or vegetation layers % of trees in different height classes For more on aquatic structure see Water

Presence of species that rely on logs, snags, woody debris, etc. as habitats N-fixation by lichens Nesting success of ground-nesting birds

13 Snags are dead standing trees. These structures provide habitat for many forest species and contribute to soil fertility after decomposition. 14 Lieberman, et al. 95. Journal of Tropical Ecology 11:161-178.


Community attributes

Sample Questions Sample Measures Sample Indicators (indirect measure)

Dynamics Interactions or processes that characterize a functioning or viable community15

Are species interactions16 occurring appropriately? Has herbivore pressure on native plants decreased? Is pollination / seed dispersal adequate for the reproduction of native plants?

Herbivory: browse damage Predation and parasitism rates: % loss to predation per year; average parasite load Pollination: Fruit set (#fruits per flower) Seed dispersal: Seed rain at different distances from parent plant

Competition: Outcome of competitive interactions (see dominance, above) Predation: Relative abundance of predator and prey species (see dominance, above) Pollen flow distance and amount Persistence of indicator species that requires certain species interactions (e.g. plant that cannot self-pollinate)

Has the rate of succession changed? How have management practices affected the percent of land in a particular successional stage?

Measures of community composition, representation and structure over time

% of community or landscape in different successional stages Nitrogen export vs. retention Presence of species characteristic of a successional stage (e.g. Nematode maturity index for soil; climax vs. pioneer species for vegetation)

15 Functions such as energy and nutrient flow are covered in their own section, because these are very dependent on the non-living physical environment as well as the living community, making them ecosystem-level functions instead of community-level functions. 16 Species interactions include competition, predation, parasitism, disease, herbivory, pollination, seed dispersal, protection mutualisms, etc.


Resources - Draft

Community attributes

Sample Questions Sample Measures Sample Indicators (indirect measure)

How closely do current conditions match natural or historical patterns of disturbance?

Frequency, intensity, duration, and extent of disturbance such as: fire, tree falls, insect outbreaks, drought, ice damage, wind throw, trampling, etc.

Community composition (see above) Relative abundance of disturbance-tolerant species (e.g. fire-tolerant trees) For tree falls: Size and arrangement of vegetation gaps (light map)

Overall community condition

Has the overall quality of the community increased?

Measures of community attributes (above) known to be critical habitat requirements for species of interest

Floristic Quality Index (terrestrial) Index of Biotic Integrity (aquatic) # of self-sustaining populations of several indicator species characteristic of that community type % of species that are classified as endangered, threatened, vulnerable, of concern, or at-risk % cover by non-native species % of acres falling within range of natural variability for community measures above self-sustaining populations of indicator species with specific habitat requirements


SPECIES It is impossible to assess the condition of all species in your project area, but the populations and even individuals of certain species can provide important information about changes in resource use and the impact of management efforts on the community and landscape. How can you prioritize which species you will monitor or manage more closely? Consider the various types of focal species in the table below. Focal Species Description Examples (site-dependent)17

Keystone Have an ecological impact larger than would be expected from their numbers alone (most evident when they are removed from the system).

Top predators, Dominant herbivores, Habitat creators

Ecosystem engineer

Have a large effect on other species by dramatically altering their environment.

Gopher tortoise, beaver, prairie dogs, coral, humans

Umbrella Either need large areas to persist or use multiple habitats and therefore, if protected, would also protect many other species.

Large carnivores, wolves, grizzly bears, northern spotted owl

Flagship Popular, charismatic species that can attract public support for conservation issues.

Panda, monarch butterfly

Representative Characteristic or even unique to the community type of interest

Karner blue butterfly of oak savannas

Exotic (alien, non-native)

Did not evolve in that location. Non-natives that spread rapidly and widely are also invasive.

Purple loosestrife, zebra mussels, argentine ant

Resource / game

Harvested and/or used for economic gain or subsistence.

White-tailed deer, salmon, timber pine

Special status Species that are monitored or of interest even if do not fit into any of the above categories.

Threatened, Endangered, Special concern, rare, vulnerable, or sensitive sppecies

Indicator Because of their physiology, habitat requirements, relationships with other species and/or sensitivity to certain conditions, their presence (or absence, abundance, distribution, reproductive success) is an indication of ecosystem conditions too difficult or expensive to measure directly.

Gopher tortoise (Gopherus polyphemus) is highly sensitive to soil disturbances; the cyanobacteria Oscillatoria rubescens is signifies impending eutrophication; Longleaf pine (Pinus palustris) is associated with frequent fires in Florida

17 See Schueller and McCance 2003 for references.


Resources - Draft

Measuring the presence/absence of a species may suffice if the species is being used as an indicator of an ecosystem attribute. Sometimes, however, you may need more information to assess the condition of species, especially if the species is a goal in itself, such as how many individuals there are and their physiological state. The framework of indicators below includes three inter-related attributes of a species:

1. Population demography & dynamics 2. Genetics 3. Physiology

Population demography and dynamics describes both the vital statistics of a species – how many there are, the relative number of males, females, juveniles, etc. – as well as how these statistics are changing over time. Is the population on the rise? Is it growing more or less rapidly than previous years? Static and dynamic features of a population reflect its overall condition and future prospects. Population viability assessment can in turn serve as an indirect measure of community and landscape viability. Genetic parameters of a population, such as the presence of certain genes or the level of genetic variation, are not always difficult to measure and can reveal aspects of the population that would be otherwise difficult to determine. Genetic variation – measured as genetic heterozygosity or, in a well-studied system, as the proportion of a visible color or morphological variants – can indicate whether small populations have high levels of inbreeding (and are therefore prone to poor breeding success), can bring to light evolutionary changes due to altered selection pressures or geographical isolation, and can give a picture of how much individuals are moving among populations. That is, genetics can serve as an indicator of corridor success. Consider genetic evaluation of species whose genetics are well known (historically well-studied and/or the focus of local university research) and therefore more easily measured. Physiology of individuals of a species may be an important evaluation measure for those species of particular concern, such as threatened or endangered species or species with high economic value. Organism physiology can also serve as valuable indicator of toxin levels or disease spread in an area.


SPECIES attributes


Population Demography and Dynamics Size (number of individuals)

Are populations of rare species growing? Is the population size closer to the minimum viable population size (if known)?

Numbers of localities and numbers of individuals per locality relative Number of individuals per area (density)

Observation of breeding/rearing behavior Density estimate from limited area sampling

Dispersion (arrangement)

Are individuals more able to find mates? Do some habitats support more individuals than other areas?

Distance between individuals (nearest neighbor) Degree of clumping

Structure (demographics)

How does the age structure of populations differ in managed areas? Are there enough young or reproducing individuals?

Sex ratio % of population that is of reproductive age Age class distribution (proportion of juveniles or fledglings) % of saplings made up by species of interest

Population dynamics (see next row) See genetic variation, below


Resources - Draft

SPECIES attributes


Dynamics (changes over time)

What is the population size likely to be next year? Are populations of rare species stable?

Birth rates; frequency of reproductive events; litter size Number of new recruits that make it to next year (recruitment rate; fledgling success) Death or mortality rates Growth rate18 Frequency of immigration and emigration

Population persistence Model estimates based on demography

18 A population does not have to be growing to be viable. Many natural populations fluctuate at around zero net growth. A decline in growth rate indicates the need to look at cause – recruitment and/or organism condition.


Species attributes Questions Measure Indicator

(indirect measure)Genetics What is the level of

variation in populations within fragments? Are these populations highly inbred? Is this species evolving? Are management practices changing gene frequencies?

Degree of variation (heterozygosity) Change in gene frequencies over time

Proportion of different morphs Frequency of deleterious mutations Change in color or other morph frequencies over time

Physiology Are individuals of this species healthy? Have signs of disease, trauma, poisoning decreased?

Animals: Glycogen stores, blood chemistry, sores, lesions Vegetation: carbohydrates stores, nutrients and polyamines

Animals: Behavior, responsiveness, mass to length ratio (fish), deformities Vegetation: abnormal defoliation, leaf color, crown die-back Either: growth rate, enzymatic activity (e.g. detoxification enzymes)

Overall species condition

Has species viability increased?

# of years populations persist (including new recruits) Repeated completion of life cycle without human intervention


Resources - Draft

WATER The composition and movement of water reveals the interactions of soil, water and living organisms over a large area. Measuring and reporting on water is also a key way to resonate with more people, even those outside your project boundaries, because it is directly related to their health. Even if your project area is not focused on water issues, the condition of water in your area can be a key bottom-line indicator of overall project progress. The framework of indicators below includes two inter-related attributes of water:

1. Physical structure & dynamics 2. Composition

The physical structure and dynamics of water in a stream, lake, pond or ocean determine the creation of habitats across the landscape and thus are directly related to the condition of biological communities. Physical attributes of water include how fast and where water is flowing, including flooding, and how much water is stored where. Water composition describes the chemical and biological aspects of water, such as its pH, temperature, oxygen content and the living organisms making up aquatic communities. These measures of water condition allow you to assess the ecological impact of local and large-scale land use. Changes in the attributes below can be measured in terms of miles of streams, percent of watershed, or number of lakes/ponds/bodies of water meeting certain conditions.


Water or Aquatic Habitat attributes


Physical Structure and Dynamics Water flow and storage, including floods

Are surface and groundwaters flowing within the range of natural variability? How have increased water removal rates affected the stream? Are we successfully returning this system to pre-dam condition? Are current water flow patterns increasing pollutant resident times?

Timing, magnitude, frequency, duration, velocity, variability, direction of water flow Frequency, intensity, extent and duration of flooding Water level fluctuations Water aquifer withdrawal and recharge rates Inflow/ Discharge

Sediment and debris transport rates Distribution and extent of aquatic and wetland habitats Structure of shoreline and/or bottom terrain of estuary, stream, or lake Channel morphology and complexity (e.g. presence of oxbows and backwaters) Salinity gradients Pool to riffle ratio Abundance of species sensitive to hydrology, e.g. flood-tolerant vegetation, submerged aquatic vegetation (SAV), or floodplain spawning fish


Resources - Draft

Water or Aquatic Habitat attributes


Composition Chemical and biological makeup

Dissolved oxygen Salinity Organic carbon Nitrate Phosphate pH Sediment loads (turbidity) Particle size and distribution Fallen leaves and large woody debris Pollutants or contaminants – see Threats

Indicator species of salinity (e.g. Siren spp.), pH (e.g. diatoms), eutrophication (e.g. dippers, Cinclus cinclus), pollution-tolerant macroinvertebrates fish, etc.

Overall Water Quality

Index of Biological Integrity


AIR Because air quality is usually managed in the context of reducing air contaminants or pollutants, measures for evaluating air condition are located in the section on ecosystem Threats.


Resources - Draft

SOIL Soil is often a key component of resource management goals because its condition is critical to all of the organisms that it directly or indirectly supports. Although many of the indicators of soil condition are community attributes, direct measures of soil condition can provide causal information on what might be preventing soil quality from improving. The framework of indicators below includes three inter-related attributes of soil:

1. Composition 2. Arrangement 3. Dynamics

Composition describes the biological and physical make up of the soil. This can be important for assessing recovery of the soil from previous use and indicating changes in function. Arrangement of soil and soil components can vary depending on management and ecosystem type and includes measures of the physical organization of soil components into layers, patches, and slopes. Dynamics of soil include its biological functioning, that is fertility and productivity, as well as how its arrangement is changing, such as soil erosion.


SOIL attributes Questions Measure Indicator (indirect measure)

Composition How has logging affected soil composition? Are we changing the biological components of soil? What is causing decreased soil fertility?

Presence of microbes and fungi Texture (sand, loam, silt, clay) Amount and type of organic matter Nutrient content Mineral content (parent material) Salinity pH

Vegetation pH buffering capacity soil structure (below) Biological activity

Structure Are soils less compacted? How has the amount of topsoil changed? How has the topography of an area changed? Are management practices changing the physical structure of the soil?

Water-holding capacity or (infiltration, porosity) Topsoil depth Bulk density (Weight/volume) Slope, aspect (position in the landscape) Patchiness (aggregation) Tilth

Vegetation Sediment export to stream drainage or groundwater

Dynamics Has soil fertility increased? (see also energy flow and nutrient cycling, below) Have soils recovered from previous use?

Biological activity (respiration, decomposition rates) Nutrient availability: Ca, P, Si, N Cation exchange capacity Leaching

Organic matter (amount, quality index) Vegetation Microbial community composition Microbial biomass


Resources - Draft

SOIL attributes Questions Measure Indicator (indirect measure)

Sediment export to stream drainage or groundwater

Leaf litter depth Nutrient release

Overall soil condition

How similar are the soil characteristics to the range of natural variability? Is soil condition supporting biodiversity?

Nematode maturity index Disease suppression Functions of healthy soil (Aber et al 2000): - Nutrient cycling - Maintains

productivity during period of shortage (e.g. drought)

- Allows for plant recovery after disturbance


ENERGY AND NUTRIENTS Although many of the above features can be used to assess whether an ecosystem is functioning properly, measures and indicators of energy flow and nutrient cycling will provide the most direct information on ecosystem performance19. For example, the leaching of nitrogen out of a system can be a key signal that it is either disturbed (decreased or disrupted vegetation will decrease nitrogen uptake) or over-loaded with N (because of excessive nitrogen loading). Energy flow and nutrient cycling fall within “ecosystem services” – ecosystem functions directly valued by humans. This makes them especially useful for goal setting and prioritizing monitoring efforts, because they can be assigned dollar values. In addition, productivity and fertility measures are frequently used in agriculture and so the facilities and infrastructure for measuring several aspects of energy flow and nutrient cycling are readily available! Every state, for example, has a soil analysis lab – housed within a state university – that accepts samples for testing.

19 Other key ecosystem processes that affect function, such as evolution, fire and other disturbances, and hydrological processes, are integrated into the tables above.


Resources - Draft


Energy Flow Have we restored natural levels of productivity?

Net primary production (new plant biomass per area per year)

Leaf area density Tree diameter increase Canopy or vegetation reflectance (remote-sensing)

Nutrient Cycling Are nutrients cycling at the appropriate rates? Can energy assimilation rates of the ecosystem support current resource use in the long-term?

Input/output budgets (landscape yield, run-off) of C and N Carbon storage (terrestrial) Trophic status (lakes) 14-CO2 fixation rate (aquatic) Diel changes in O2 and CO2 fluxes (aquatic) Nutrient concentrations in soil or water (Nitrogen, organic N, NOx, ratio of C to N, Phosphorous, ortho-P, particulate P, organic P, Calcium, Silicon, Potassium)

Fungi and soil microbial community Amount of nitrogen leaching to water system or below rooting zone (vs. retention) litter depth % cover or % openness (indicator of leaf litter) rate of soil organic matter increase and litter layer accumulation Green leaf nutrient content ratios


Threats to ecological systems Threats are circumstances or forces that are preventing the ecosystem from being in the state you desire. Direct and indirect reduction of threats to the ecosystem may be an integral part of your project’s goals. Tracking changes in threats and their impacts over time, or comparing threats in places under different management are useful ways to measure progress towards goals and better understanding the impact of your efforts.

Including threats in goals and evaluation

Ecological goals and measures of success should reflect the combined approach of enhancing (preserving, maintaining) certain ecosystem features and reducing (avoiding, minimizing) common threats to those features. For example, Dale et al. (2000) provide the following guidelines for land management based on ecological principles:

1) preserve rare landscape elements and associated species

2) avoid uses that deplete natural resources over a broad area

3) retain large contiguous or connected areas that contain critical habitats

4) minimize the introduction and spread of nonnative species

5) avoid or compensate for the effects of development on ecological processes

6) implement land-use and management practices that are compatible with the natural potential of the area

Below are several inter-related threats that apply to both aquatic and terrestrial systems. Add/integrate – from em95: (roughly in order of frequency said it was a stress)

1. hydrologic alteration 2. land conversion to urban uses; 3. exotic species 4. agricultural practices 5. roads or other infrastructure 6. disruption of fire regime 7. non-point source pollution 8. grazing and range management 9. timber/forest management 10. land conversion to agricultural uses 11. recreation 12. point source pollution 13. mining 14. overfishing, overhunting, or overcollecting

stripmining farming waterborne disease outbreaks pest outbreaks # of illegal dumps along the waterway Phosphorous loads

• in water column • accumulation in sediments • lake water total P • TN:TP ratio – indicator: phytoplankton community structure (rel. abundance of

cyanobacteria), macrobenthos Hydrology


http://www.esa.org/publications/landuseb.htm

Resources - Draft

indicator: submerged aquatic vegetation biomass (SAV), e.g. Chara – macroalagea, pioneer to vasucal plants Habitat loss, alteration or fragmentation Multiple causes and consequences Possible consequences: habitat destruction (direct loss of species), fragmentation (affects biodiversity – eliminates species with large range requirements, edge effects; inbreeding depression)…

General threat Types or examples


Sample Measures and Indicators

Development (commercial or residential) Conversion to rangeland, agriculture or aquaculture

To what extent are natural areas converted to other uses? What is the rate of loss of natural areas? The rate of land conversion from x to y? What is the rate of loss of specific habitats for species of concern? How many acres of natural areas are indirectly affected by land conversion? How do ecosystem attributes compare between natural and altered areas?

Change in land use/cover (using satellite imagery or land-planning statistics), including: # of acres of developed land # of acres grazed # of stream miles with vegetative cover % of land with impervious surface (e.g. paved) % loss of wetlands road density soil organic matter See indicators for ecosystem features (e.g. compare biodiversity, nutrient-cycling, etc. of converted and ‘natural’ areas)

Fragmentation:20 To what extent is the # and size of discrete areas

20 Fragmentation is “the alteration of previously continuous habitat into spatially separated and smaller patches.” (Dale et al 2000). Although it is often the direct result of human land-use





Alteration of parts of continuous habitat Introduction of infrastructure that acts as a barrier to species movement

area fragmented? Is fragmentation increasing or decreasing? How severe are the barriers? What is contributing to fragmentation? Loss or degradation of intervening habitat, infrastructure (such as roads, fences)?

(e.g. patch sizes of natural areas within urban or suburban regions) # and size of corridors Average patch size Number of patches in different size classes Ratio of edge to interior area Length or density and type of barriers to dispersal (roads21, fences, etc.) Road density, type (paved, lanes), use, location Abundance of indicator species: - with large area

requirements (e.g. large carnivore)

- characteristic of fragment edges (e.g. cowbird)

practices, such agriculture, forestry, and development, fragmentation can also be the result of wildfires, wind, or flooding. Patches may be separated not only because intervening land has been modified, but also by adding even small roads or fences into continuous areas. 21 The Wilderness Society is developing a software program that assess the ecological and economic impact of raods, called RoadNET. For more information search the Wilderness Society website: http://www.wilderness.org/


http://www.wilderness.org/

Resources - Draft

Local scale land alteration – CHANGE = result of something else which is the threat Vegetation

removal/loss (e.g. prior to construction, trampling)

To what extent has vegetative cover been reduced in this area?

% of ground covered with vegetation Rate of soil erosion22 Amount of sediment in run-off

Soil removal (e.g. prior to opencast mining)

Soil compaction What is the frequency of heavy machinery use along streambeds? What percent of logged areas has degraded soil quality?

Bulk density Degree of trampling/disturbance Puddling

Any physical change

Has this habitat been altered or disturbed (related to resource use and/or management practices)?

Abundance of indicator species: - with specific habitat

requirements - typically a colonizer of

disturbed habitat Microclimate23 (local temperature, moisture and light)

Hydrology alteration Alteration of water

course (dams, canals) or water table (drainage tiles, in-filling)

Have hydrology changes increased in the last ten years? How does the degree of hydrology alteration relate to… (e.g. the composition of the invertebrate aquatic community?)

# of stream miles affected by dam control # of acres drained changes in level of water table # of acres with drainage tiles in place

22 Cite mystic example as way to measure erosion 23 Microclimate changes under different management practices and can indicate subtle changes in ecological processes (see Chen et al. 1999), and it can be an indication of habitat viability


Disturbance regime alteration Fire suppression How much area is

affected by alteration of disturbance patterns?

# of acres not burned within last ten years

harvesting tree falls

Disruption of nutrient cycles Nutrient loading

Nitrogen deposition in soil

Nutrient run-off What is the severity of nitrogen run-off into streams? Is it lower in no-till agricultural areas?

N concentration Frequency of eutrophication or reported anoxia or hypoxia

Nutrient removal Amount of Nitrogen loss from harvesting

Population status of

harvested species Generation time of species relative to harvest rate

Severity of destruction Duration of impact Frequency of destruction Extent (area) affected by harvesting (# of acres)

vegetation trampling soil compaction # failed breeding seasons trail vs. interior plant species composition


Resources - Draft

Overall Change in number of

species with threatened and endangered status

Incompatible resource use Overharvesting of animals and plants can reduce populations to levels below critical size, preventing them from recovering and/or affecting the species with which they interact,…




Over-harvesting (fishing, grazing, hunting, collecting)

What is the hunting/ fishing pressure on targeted species? What is the sustainable harvest level of a species? How do current harvest levels compare to this? Have large predators been removed from the system? Is stream quality (see ecological features) improving even in grazed areas?

Destructive harvesting (clear-cut logging, cyanide bomb fishing, mining)

What are the indirect effects of our current harvest practices? Can we harvest in a way that is less destructive?





Are destructive harvesting practices increasing in frequency or area? How many acres have been selectively cut?

Disruptive or destructive recreation or human presence (off-road vehicle use, military training activities, tourism, etc.)

Is human presence affecting species interactions, including breeding behavior? Is it causing local land alteration (see above)? Are invasive species more abundant along frequently used trails?

Introduced species Can reduce biodiversity directly or indirectly – through predation, competition, transmitting disease, introgression (hybridizing with native species), or altering habitat features such as… (rooting, nitrogen fixation, trophic structure…


Resources - Draft




Non-native invasive species

What is the extent of invasive species success and impact? Are invasive species becoming less abundant in managed vs. unmanaged areas?

- Proportion of total species that are non-native

- Number of highly invasive species

- Percent cover by non-native or invasive species

- Average density of invasive species of interest

- % of sites where invasive density is greater then x

Have we reduced the source of new introduced species in the project area?

- Proportion of local nursery plant sales that are non-natives

- Number of new invasive species in project area

- Genetically

modified organisms

Are potential sources of GMO pollen, seeds, or propagules increasing in the area?

Surrounding agriculture area with gmo crops # of individuals escaped from gmo aquaculture











Pollution Immediate direct effects, as well as indirect and cumulative effects on species and habitats




Point and non-point (run-off)

How many acres are affected by a pollutant? What is the change in concentration of pollutant or in streams/air/soil/fish? Is the source of the pollutant decreasing?

Fossil fuel burning Recreation (boating) Soil and water pH


Resources - Draft




….. Fertilizers

Pesticides

Use of fertilizers/ pesticides Concentration of fertilizer/pesticide in streams and groundwater near farmland

Sediment loading

What is the change in amount of sedimentation in streams?

Tubidity, suspended sediment acres of agriculture on steep slopes # roads crossing streams

Salt Presence of salt dependent plants

Air pollutants Emissions of greenhouse gasses Ambient concentration of air pollutants / particulates Consumption of ozone-depleting substances Frequency of complaints about air quality Visibility Soil acidification Lichen community composition (indicator of exposure to N- and S-based pollutants)

Water pollutants or contaminants (oil, fertilizer, toxins, coliform

Bacteria concentration, type Toxins in fish / top predators (e.g. existence of





bacteria, etc.) state fish consumption advisories) Capitella capitata Mytilus edulis (universal biomonitor of water-column pollution) Fish health Presence of contaminated sediments

Litter


Resources - Draft

Assets that facilitate progress toward ecological goals Ecological assets are often overlooked as threats receive more attention. Assets that can facilitate progress towards ecological goals include a) ecosystem features that functionally support the ecosystem, moving it in the direction desired, and b) socioeconomic or political circumstances that are either compatible with these goals and/or support the strategies used to achieve ecological objectives.


Asset Examples Question Indicator or measure Existing high quality/quantity ecosystem features

- Land already in protection

- Large, continuous (unfragmented) areas of land

- Self-sustaining populations (especially of host plants, keystone species, etc.)

-

Existing programs or legislation compatible with ecological objectives

- Clean Water, Clean Air, Endangered Species Act

- Conservation Reserve Program

- Certification programs (give links)

-

Economic incentives compatible with ecological objectives

- Demonstrated financial advantage of best practices (e.g. no-till agriculture)

- Economic advantage arguments for various situations provided by Wilderness Society research

-

Ecosystem features with public appeal

- scenic, aesthetically pleasing areas

- beautiful, cute or popular animals (e.g. panda, prairie dog, butterfly24)

- plants or animals with special

Are we using public appreciation of scenic areas to rally their support for other project activities? Can we use a token popular species to act as a symbol for

24 For an example, see Oak Openings’ use of the Karner Blue Butterfly: http://www.oakopeningsregion.com/


http://www.fsa.usda.gov/dafp/cepd/default.htm

http://www.fsa.usda.gov/dafp/cepd/default.htm



http://www.oakopeningsregion.com/

Resources - Draft

Asset Examples Question Indicator or measure with special significance (state bird, state flower)

-

whole-habitat restoration?

Existing human capital to implement strategies and/or monitor

- volunteers - school groups - university

students and faculty

- nature observers (e.g. bird watchers)

Have we increased our use of available volunteers for invasive species removal? To monitor changes in bird populations?


SOCIOECONOMIC Evaluation Questions and Indicators

Overview [intro to be added here] Data collection tools for social outcome monitoring [to be explained]

A. Community surveys

B. Focus groups

C. Expert interviews

D. Constituent survey of outcomes

E. Rapid community assessments

F. Rapid cultural assessments

G. Community economic profile


Resources - Draft

Key features of social systems [draft outline] Local economies Diverse and stable industries

Diversity Stability

Local investment, development and growth Value-added manufacturing Community resident assets and land ownership

Living-wage jobs Job training Traditional livelihoods Employment and earnings

Quality of life Social capital and community capacity

Interaction and participation Networks, relationships, and trust Leadership

Community character History and traditional culture Identity and sense of place Physical and aesthetic qualities of landscape Smart growth and sprawl

Opportunities for residents Social opportunities Education Recreation, arts, and culture

Services and safety net Affordable housing Day care Accessible health care system

Social well-being Crime and safety Health Demographics Environmental amenities (e.g. clean air and water) Perceived quality of life


SOCIAL questions and indicators sample

Healthy and enduring local economies Appropriately diverse and stable industries What is it? Diversity and stability are two important and interrelated components of “healthy and enduring local economies.” Diversity relates to the number of businesses and industries in a community or region, as well as the relative importance of each, in terms of the number of people employed and wages earned, as well as contribution to the revenue stream. Does the local economy rely too heavily on a single industry, such as timber extraction or winter-only tourism? An economy with a greater diversity of industries may be more resilient and stable when faced with depressions in the price of a particular commodity, changes in global markets, and economic downturns than an economy with fewer industries. Stability is the absence of dramatic shifts in economic conditions: Some projects described stability as “moving beyond boom and bust.” It may be important to note that an economy that is consistently under-performing may be stable, but it is likely not desirable. Thus, measures of stability and diversity must be understood in the context of a suite of other economic indicators as well. What constitutes an “appropriate” level of diversity and stability will depend on popular consensus, but other communities can serve as a guide to determining feasible levels. How do you measure it? It is possible to measure local conditions or indicators that point to changes in diversity and stability. The indicators suggested in the chart below are drawn from existing projects in the United States. Data for a majority of these indicators may be drawn from the Census, REIS, and County Plus data sets, as described in the “Data Sources” page of the resources section. In analyzing the data, it is particularly important to look for trends (increasing, declining, or steady) and to measure the rate of change. Observe not only changes over time, but also changes in relation to other comparable locales. Finally, as suggested below, look for signals of future stability as well as current stability.


Resources - Draft

Aspect of Industries

Evaluation Questions Possible Indicators

Diversity How diverse is the economic base of the community? Is diversity increasing or decreasing?

Total number of employers in community

% employment by industry Jobs per industry

Employment by top five employers

Earned wages per industry

How reliant is the community on particular industrial sectors?

Tourism as % of payroll by sub-region

% of employment in service sector

Stability How consistent are jobs, revenue, and earnings?

Change in jobs per industry

Change in earned wages per industry

Fiscal stability and balanced revenue

What signs point to future stability/instability?

Office vacancy rate

Number of building permits issued Labor productivity

Net tree growth and removals

Percent of harvested forest successfully restocked

Total assets of residents


SOCIAL questions and indicators sample Improved quality of life in our communities Enhancing Social Capital and Community Capacity What is it? Social capital, like financial capital, is an asset. It is the "glue" that holds communities and societies together. Basically, social capital is the reservoir of trust and the web of relationships among people in a given community. In general, the more trust that exists and the stronger the networks among individuals, the more the community is able to accomplish its goals and the better off are its members. Communities high in social capital are healthier, with better schools, less crime, more tolerance, lower mortality rates, and greater economic equality than those with less social capital. Networks of relationships in a community may be formal, for example through membership in organizations, or informal, through ties based on family, geography (neighbors), or friendship. They may also be either "horizontal," as in relationships among equals, or "vertical," for example between members of a community and a powerful politician. A community that builds relationships in both the horizontal and vertical dimensions will become more successful in mobilizing community assets at the local level and influencing policy at a broader level. Social capital, then, can be viewed both as an end in itself, since communities with more trust and stronger bonds are better places to live, and as a means to an end, since social capital is a resource with which individuals and communities are better able to achieve other goals, such as improved health, stronger economies, and better educational opportunities. How do you measure it? It has long been argued that collaborative ecosystem management projects build trust, strengthen ties among members of the community and organizations, and enhance the problem-solving capacity of communities. The concept of social capital introduces a tangible way to discuss the strengthening of networks and trust. Furthermore, it firmly links these improvements to measurable advances in a community's economic, social, and even environmental well-being. If increasing social capital is the ultimate or even the intermediate goal of a project, how can changes in the amount of social capital be measured? Social capital measurement involves monitoring changes in the level of trust, the degree of civic involvement, and the strength and density of networks of relationships among citizens in the project area. A combination of qualitative and quantitative methods may be useful in assessing social capital. Tracking changes over time and comparing trends within one community to a demographically similar community may give more meaning to the numbers.


Resources - Draft

Aspect of Social

Capital Evaluation Questions Possible Indicators

Trust How much do residents in the community trust their fellow citizens?

Percent of citizens who agree in a survey that, "In general, people (in this community) are trustworthy."

How many local organizations and associations exist in the community? How involved are individuals in these associations?

Number of organizations and associations within the community

Percent of adults who belong to one or more association, club, or community organization

Percent of youth who belong to one or more school or community club

Percent of population volunteering more than 50 hrs./year

Membership / Volunteering

Do people belong to multiple organizations? How much does membership overlap?

Median number of organizations or associations to which citizens >18 belong

Patterns of board overlap, determined from board membership rosters

Informal networks

How strong are the links among people in the community outside of formal group membership/business settings?

Answers, in a telephone survey, to the following questions: "How do you define 'neighbor'? How many neighbors would you say you have? How many of them do you know by name? What kinds of interactions do you have with them?"

Answer to question, “In the past month, have people visited you in your home? How many times?” (% yes, median number of visits)

Information How informed are citizens about community affairs?

Percent of population with local newspaper subscription

Percent of citizens who hold a library card

Participation in government

How involved are citizens in government?

Percent of eligible (or registered) voters taking part in presidential election

Percent of citizens who say they "attended one or more town meetings in the last year"


SOCIAL questions and indicators sample Maintenance of community character What is it? When describing their goals, many projects identify desired outcomes that fall under the heading of “community character,” including “agricultural character,” “cultural heritage,” “aesthetic benefits,” and “rural lifestyle.” Community character is defined by history and traditional culture, identity and sense of place, and aesthetic qualities of the landscape, including patterns of growth. In describing character, we ask, “What makes a place unique and how do people relate to their local environment?” How do you measure it? Unlike more quantitative measures such as unemployment, community character is by its nature qualitative. Nonetheless, several projects have devised more readily measurable numerical proxies or indicators to suggest changes in character. The indicators below require the application of both quantitative measurements and qualitative techniques, such as interviewing and focus groups. Aspect of Community Character


History and traditional culture

To what degree do residents recognize and appreciate local history?

Number of registered historic sites

Local historical societies per 10,000 people

Number of people visiting historic and cultural sites

Number of and attendance at traditional community events/festivals

Identity and sense of place

What do local residents feel is important or special about their community? How do people in the community describe themselves in the context of the community?

Responses to survey, interview and/or focus group questions

How knowledgeable are residents about the local natural environment and human history of the community?

Responses to "sense of place" questionnaire


Resources - Draft

Aspect of Community Character


Physical and aesthetic qualities of landscape

How is the local landscape changing and how do residents perceive these changes?

Conversion of farmland

Open space near villages Farm acreage Number of community gardens Cleanliness of the community Smart growth and sprawl

What is the pattern of development on the landscape? Does it reflect "smart growth" principles or does it tend toward "sprawl"?

Percent of new residential lots within 1/4 mile of services

Vehicle ownership, fuel consumption and travel per capita

Impervious surfaces Privately owned acres placed under

conservation easement Ratio of urban open space to developed land


PROCESS Evaluation Questions and Indicators

Overview When you evaluate process you consider issues like group composition, meeting effectiveness, communication, trust and relationship building among parties, and how decisions are made and implemented. These issues represent the nuts and bolts, engine, and steering wheel of your project. Without a strong process in place, your group is never going to stay together, make headway, or adjust to new challenges over time. This section of the resources section will help you identify and resolve any weaknesses in your project’s process – weaknesses that may undermine your ability to achieve your project goals. Evaluating process can be simple Evaluating process can be done simply and with limited funds because, in many cases, you only need to ask participants’ how they feel about an issue of concern. For example, let us assume participants in your group are not communicating well – a common problem for many partnerships. One way to assess the extent of the problem and identify a solution is to conduct a short survey on the topic at different time intervals, solicit feedback on how to enhance communication, and implement changes to solve the problem. Admittedly, many procedural issues require a more extensive analysis than a simple participant survey. Nevertheless, you will likely be able to evaluate your project’s process with simple, readily available information. Identifying process issues for evaluation On the following page are six overarching characteristics that reflect the essential needs of an effective process. Next to each characteristic are twenty common procedural requirements or qualities of successful processes as they relate to each broad, umbrella-like characteristic.25 Note that the first letter of each of the main characteristic below together spells the word: MORALE. Maintaining a positive morale is one, if not the most important factor to collaborative success. As you study the following list, consider which characteristics are most relevant or important for the success of your effort and whether your project exhibits them. Note also that certain elements are more relevant at different points in a group’s process. The characteristics below are organized in rough chronological order according to the time in a project’s history when they are most important.

25 These characteristics are based on the work of practitioners and academicians with experience in collaborative problem solving and alternative dispute resolution. These authors’ works are summarized in EMI’s report Objectives & Approaches to Evaluate Process in Collaborative Initiatives: A Literature Review on Existing Frameworks, Methodologies & Recommendations to Measure Process Success.


Resources - Draft

Motivation

• sense of urgency; perception of crisis/threats • perception of interdependence; sense of place or shared goals • shared sense of ownership or responsibility • incentives created by alternatives to collaboration • financial opportunities (congressional or foundation support) • perception of possibilities (“there might be another way”)

Organization • outreach; opportunities to interact (government programs, etc) • coordinator/lead responsibility/structure • well-managed and open process

o interest-based problem solving; consensus decision making o involves the “right” people and organizations

• creation and use of a management plan/framework o mission or vision statement; clear objectives; creative strategies

Resources • facilitation & process management • scientific expertise and information • funding, staff and equipment

Adaptability • joint learning grounded in credible science/information • ongoing monitoring and assessment connected to decision making

Legitimacy • effective representation of all key affected interests • declining opposition; positive external perceptions • accountability and ties to statutory decision making processes • commitment of agency & political leaders to process and follow through

Energy • dedicated energetic individuals • process champions; social networks/relationships • small successes/recognition • dedicated resources (money, expertise, etc.)


How the resources section can be used While you studied the above list, two questions may have surfaced:

1) What do some of the above requirements mean? 2) How can I determine if my project meets them?

This section of the resources section will help you answer these two questions. Take a moment right now to flip to any page in this section so you can see how the section is laid out. You will notice that there is a page in the resource section dedicated to each procedural requirement listed above. On each page we provide a brief narrative describing the requirement and its relevance to process success. Below each narrative are 3-5 diagnostic questions that can help you determine if your process exhibits a particular requirement; each set of questions begins with a general question followed by increasingly specific questions. The questions themselves can be answered with a simple “yes” or “no,” however, you may find it useful to back-up your answers with a more normative response. Importantly, these questions are designed to help you evaluate procedural issues relevant to your project. Depending on the issues that interest you and the amount of resources you have, you may chose to focus on some or all of the procedural requirements and questions we provide. The role of process indicators In many cases, you will be able to answer a particular question with little difficulty. For example, when asked whether your group has a mission statement you will probably answer: “yes” or “no.” However, when asked whether this mission statement is clearly motivating the parties and consistent with their interests you may crinkle your brow and answer: “hmm, I’m not sure.” For these kinds of thought-provoking questions, we provide sample indicators. For some projects, using the indicators may add a layer of complexity you could do without. However, for many projects, selecting an indicator can help you answer a question more objectively, particularly when you need to solicit participant sentiments on a particular issue of concern. These indicators can also be helpful for those who will be evaluating their process over time and want to be able to track improvements in their project’s process. Sample methodologies are provided next to each indicator to illustrate how and where you can collect important information. Selecting and answering your evaluation questions The process-related questions you ask will depend in large part on your project’s complexity. If your project involves a small group of participants working on a single issue, your group may simply skim the resource questions and attempt to answer the most relevant questions for your project. Thus, in one or two settings, you can use the resource section as a quick diagnostic tool, a quasi-checklist, to identify weaknesses in your project’s process. However, because the answers to these questions may not be obvious, many groups will need to dedicate more time and energy to answer them. If this is your case, you will find it helpful to use Worksheets B and C to develop a game plan to evaluate your project’s process.


Resources - Draft

PROCESS questions and indicators sample Key process issues Motivation Shared sense of ownership or responsibility Successful collaborative initiatives require active participation and commitment from all parties involved in the project. In order for parties to participate actively and contribute substantively to an effort, they need to feel ownership over the project and the results of the process. Common ways to cultivate ownership include participatory exercises at the start of an effort such as crafting a joint problem statement or engaging in community visioning. Throughout the process, parties may experience greater ownership if they are given meaningful tasks such as leading a joint research effort, coordinating a group activity, or facilitating discussions. As ownership increases, parties may also feel more responsible for the management of the process and its substantive outcomes. Moreover, when parties experience greater ownership and responsibility, it is likely your project will be more enduring, effective, and enjoyable.

Questions Indicator Sample methodology Do parties feel in control of the process and responsible for its outcomes?

Extent to which parties feel they have ownership over the process

Ask parties whether they feel in control of the process, can participate in decision-making, or whether they feel manipulated by the process

Are mechanisms in place to strengthen ties between parties (e.g., regular meetings, community visioning sessions)?

Frequency of meetings Nominate one person to track the frequency in which participants meet formally and informally

Is the process driven by a purpose that is practical and shared by all parties (e.g., development of management plan, reintroduction of endangered species)?

Extent of staff time committed by each organization to the effort

Determine through surveys, meeting minutes, workplans, or related means how much time is spent on project activities


Perception of interdependence Shared sense of ownership or responsibility Incentives created by alternatives to collaboration Financial opportunities Perception of possibilities Organization Outreach Coordinator Well-managed and open process Creation and use of a management plan/framework Resources Facilitation & process management Scientific expertise and information Funding, staff and equipment


Resources - Draft

PROCESS questions and indicators sample Adaptability Joint learning grounded in credible science/information An important part of working in partnership with others is to collectively agree on the sources of information to be used to identify issues of concern and specify a course of action. The information participants must understand and base group decisions on can be quite variable and may range from data on wildlife populations of concern, potential water withdrawals from a new industry, or prospective economic growth from recreation-related business. Predictably, much of this information may be too limited to allow a group to decide on a particular course of action. As such, a collaborative process often entails some degree of collective learning. The structure in which this ‘new’ learning takes place has important implications for a project’s internal/external legitimacy and will also impact each party’s willingness to act on any new information acquired.

Questions Indicator Sample methodology Are parties gathering information and learning as a group?

Extent of group learning and interaction

Ask parties if they feel they are learning from one another

Are mechanisms in place to ensure learning does not outpace participants?

Extent to which parties feel comfortable with the volume and complexity of information used in decision-making

Review project reports and ask parties to determine if information is presented in a way suitable to all parties involved (e.g., condensed, jargon-free, etc.)

Do parties agree on a mutually credible process for gathering new information? Have parties specified who will be involved in obtaining new information and the methodologies that will be used?

Extent to which parties have specified guidelines for obtaining new information

Review protocols for collecting new information


Monitoring and assessment connected to decision making Legitimacy Effective representation of all key affected interests Declining opposition; positive external perceptions Accountability and ties to statutory decision making processes Commitment of agency & political leaders to process and follow through Energy Dedicated energetic individuals Process champions; social networks/relationships Small successes/recognition Dedicated resources


Resources - Draft

STRATEGY Evaluation Questions and Indicators

Overview

Strategies by category Sources to complete: taxonomy of objectives, emi team, see M&S table Resource Management and Restoration Land Acquisition/Protection e.g. Habitat gained as percentage of entire floodplain or watershed Species Reintroduction Species Removal Restoration/simulation of processes (disturbance, hydrology) Alter/maintain Resource use practices e.g. “acres in conservation tillage, acres brought out of cropland” Leave-alone Socioeconomic development Marketing/Attract investment Development of sustainable industries Certification Community events Educational/vocational training opportunities Law and Policy Enforcement and watchdog e.g. polluter accountability Litigation Legislation and treaties Policy development/reform Land use policy – e.g. land ordinances Education and Outreach Curriculum development Public Relations Interpretation centers Media Information and Planning


Are objectives and strategies based on recent and accurate information? Are species used as indicators demonstrated indicators of..? Management Plan Research Inventorying/Identification Identify conservation sites (e.g. areas containing one or more imperiled species) – source infor natural heritage program Monitoring Review/Evaluation Capacity Building Partnerships/collaboration, trust-building Skills development Fundraising Reorganization Specific examples to add: Strategies to reduce threats Nutrient-loading

• source control • constructed wetlands (stormwater treatment areas) • sediment dredging

altered hydrology • aquifer storage and retriever walls • modified regulation schedule

# of acres of restored hydrology (measures of ‘restored’: Flow, groundwater level, flooding, channel morphology temperature, etc. relative to reference site or time) cessation of cropping reduction of crop losses from flooding acres of x ecosystem restored acres of prescribed burning annually acres affected acres protected conservation of threatened ecological systems ration of property targeted for conservation:property presently in negotiation for acquisition number of trees planted amount of trash collected for proper disposal could say ‘miles of streambank’ or acres of prairie restored – but need criteria of what is restored extent and type of management


Resources - Draft

draft of framework?: Strategy Goal Evaluation

question Measure Indicator (indirect

measure) Simulate natural disturbance26

Maintain processes necessary for ecosystem integrity / natural range of biodiversity

How well do out actions simulate natural process?

Compare frequency, intensity, and area of strategy to range of natural variability for that disturbance

Species, community and/or physical features (especially soil nutrients) in natural vs. simulated areas

Miles of stream bank restored

# or stream miles with revegetated banks; # of acres in conservation tillage

26 Note that it is important to frame your goal as the desired outcome (ecosystem properties and processes such as biodiversity, nutrient cycling, etc. ) and not simply as achieving the management activity of reintroducing fire to the system. In this way, you will measure strategy effectiveness in terms of progress towards an ecologically meaningful goal (Christensen et al. 1989 in Christensen et al. 96), and assure that simulated disturbances have similar ecological effects as natural processes (DellaSala et al. 1995 in Landres et al 99)


Prioritization Tools The following prioritization exercises can help you select topics for evaluation. These exercises are meant to complement the process presented in Stage B, Step 2 of the guide. Exercise to prioritize threats for evaluation

Choosing the threats you would like to evaluate may be difficult because for each of your project objectives you will likely have multiple threats you could evaluate. The following threat ranking exercise enables your project team to prioritize threats for evaluation according to specific criteria or attributes. You can chose different criteria to rank the relative importance or saliency of your threats, but we suggest using the following ones at a minimum:

• Community Perceived Importance: How do community members or project participants rank the threats affecting a particular objective?

• Area: Does a particular threat affect your entire project area or a limited area? • Intensity: How severe is the impact of a threat on your target (e.g., a type of habitat or

endangered species)? • Urgency: Is the threat occurring now or only likely to occur ten years from now? How

critical is it that people take action to deal with the threat today? Steps to follow to rank your threats for evaluation:

1) Tear out the “Threat Ranking” Worksheet in the back of this report and at the top of the worksheet list the first objective you have prioritized for evaluation.

2) Write down a list of threats affecting the project objective in the left hand column as shown in Figure B-2.

3) Depending on the number of threats you have listed, rank each threat according to the above criteria. For example, as shown in Figure B-2, Oakwood’s community members perceive habitat fragmentation as the # 1 threat.

4) Once you have ranked each of your threats according to the above criteria and any additional criteria you would like to add, you should add the numbers horizontally to calculate a threat’s “total score.” The threat with the lowest “total score” indicates that it should be a priority for evaluation.


Resources - Draft

Figure B-2: Example threat Ranking Worksheet Objective: 200 acres of high quality savanna Team members present: ________________________________ Date:_______________

Criteria

Threat Community perceived

importance

Area Intensity Urgency Total Score

Evaluation Priority

Invasion of non-native

species

3 3 2 2 10 3

Habitat Fragmentation 1 2 3 3 9 2

Lack of host species 2 1 1 1 5 1

Exercise to prioritize strategies for evaluation

As you look at your situation map you may see that many of your strategies are designed to mitigate multiple threats or capitalize on various assets. The following strategy ranking exercise helps your team prioritize strategies for evaluation based on specific criteria or attributes. You can choose different criteria to rank the relative importance of your strategies, but we suggest using the following ones at a minimum:

• Staff time & organizational resources: How much staff or volunteer time is spent implementing a particular strategy? How much money has been invested in a particular strategy?

• Confidence: Which strategies are you the least confident in their effectiveness? Which strategies are the most controversial? Which strategies have you previously evaluated and are certain have significant pay offs?

• Breadth: Based on your situation map, are any strategies employed to mitigate multiple threats? If yes, which one(s) are designed to affect the most threats to your project objectives? Based on the “Threat Index for Evaluation,” which strategies are designed to mitigate your most serious threats?

• Financial support: Which strategies are supported by funders? Which strategies have the greatest chance of continued funding?

Steps to follow to rank your threats for evaluation:

1) Tear out the “Direct Strategy Ranking” Worksheet in the back of this report and at the top of the worksheet list the first objective you have prioritized for evaluation.

2) Write down a list of strategies you need to prioritize for evaluation in the left hand column as shown in Figure B-3.

3) Depending on the number of strategies you have listed, rank them according to the above criteria. For example, as shown in Figure B-3 below, Project Oakwood dedicates


the most staff time and organizational resources to reintroduce endangered species and therefore that strategy is ranked # 1 under that criterion.

4) Once you have ranked each of your strategies according to the above criteria and any additional criteria you would like to add, you should add the numbers horizontally to calculate a strategy’s “total score.” The strategy with the lowest “total score” indicates that it should be a priority for evaluation.

Figure B-3: Example Strategy Ranking for Evaluation Team members present: ________________________________ Date:_______________

Criteria

Strategy Staff Time & organizational

resources Confidence Breadth

Financial support

Total Score

Evaluation Priority

Prescribed fire 3 3 2 2 10 3

Reintroduction of

endangered species

1 2 1 1 5 1

Land acquisition 2 1 3 3 9 2


Glossary

Sources of available data useful for evaluation Name Type of data Contact See excel file in resources section folder


Additional resources: web sites and related materials

DRAFT – WILL BE ORGANIZED BY SUBJECT! Useful websites with information on evaluation:

IHMC Concept Map Software. http://cmap.coginst.uwf.edu/info/. Downloaded 1/6/03 (add list in “web links for resources” document)

Useful print materials on evaluation:

Margoluis, R. and N. Salafsky. 1998. Measures of Success: Designing, Managing, and Monitoring Conservation and Development Projects. Island Press, Washington, DC. Margoluis, R. and N. Salafsky. Date?. Is Our Project Succeeding? Biodiversity Support Program, Washington, DC. Salafsky, N. and R. Margoluis. 1999. Greater Than the Sum of Their Parts: Designing Conservation and Development Programs to Maximize Results and Learning. Biodiversity Support Program, Washington, DC Saterson, K, R. Margoluis, and N. Salafsky. 1999. Measuring Conservation Impact: An In Interdisciplinary Approach to Project Monitoring and Evaluation. Biodiversity Support Program, Washington, D.C


http://cmap.coginst.uwf.edu/info/

Glossary


Glossary Activity: Collection Procedure: The manner in which you will collect your data. Comparison: What you will compare the indicator against to detect change. Evaluation question: A specific question about an element of your situation that you hope the evaluation will answer. Goal = a statement of what you are trying to achieve, stated as a desired condition of the system or some part of the system Indicator: a measure that represents the target or threat

Keystone indicator: indicates lots of stuff Integrative indicator: integrate lots of things

Objective: a more specific statement detailing the desired outcomes of a project in measurable terms. Frequently there are several objectives within an overall goal. Quality: describes specific characteristics of the target. Quantity: in ecological terms refers to the amount of the natural resource that is desired (e.g. number of acres, size of population). Quantity in the case of social goals could refer to the extent or area to be covered (e.g. city versus county, or 50% of the population.) Strategy: the collection of actions you will take to mitigate the threats and capitalize on assets towards realization of your objectives and goals.

Documents

Resources - Draft Evaluation Resources · Resources - Draft How to use the resources in this book Evaluation Resources serves as a toolbox of information and techniques to assist