11
Module 4:Module 4:Designing Performance Indicators Designing Performance Indicators
for Environmental Compliance for Environmental Compliance and Enforcement Programsand Enforcement Programs
22
Basic ConceptsBasic Concepts
33
Indicators…Indicators…
Measure and manage environmental compliance Measure and manage environmental compliance and enforcement program performance and enforcement program performance Ensure accountability to legislative bodies, Ensure accountability to legislative bodies, budget authorities, constituent groups, and the budget authorities, constituent groups, and the publicpublic Evaluate and improve effectiveness in achieving Evaluate and improve effectiveness in achieving program missionprogram mission
44
Types of IndicatorsTypes of Indicators
InputInput:: personnel, funds, and other resources that personnel, funds, and other resources that contribute to an activity (contribute to an activity (exampleexample: number of inspectors): number of inspectors)
OutputOutput:: quantitative or qualitative measures of activities or quantitative or qualitative measures of activities or work products (work products (exampleexample: number of inspections): number of inspections)
Intermediate OutcomeIntermediate Outcome:: changes in knowledge, behavior, changes in knowledge, behavior, or conditions that result from program activities (or conditions that result from program activities (exampleexample: : pounds of illegal discharge reduced)pounds of illegal discharge reduced)
Final OutcomeFinal Outcome:: the change in environmental quality the change in environmental quality attributable to program activities (attributable to program activities (exampleexample: improvement in : improvement in water quality)water quality)
55
Audiences & UsersAudiences & Users National/state policy makers and legislatorsNational/state policy makers and legislators Regulatory agenciesRegulatory agencies Industry sectors Industry sectors Stakeholder and interest groupsStakeholder and interest groups General publicGeneral public International bodiesInternational bodies
66
Benefits of IndicatorsBenefits of Indicators Targeting of program operationsTargeting of program operations Ability to set goals and adjust strategiesAbility to set goals and adjust strategies Decision-making for resource allocationDecision-making for resource allocation Ability to identify problems and correct Ability to identify problems and correct
performanceperformance Ability to motivate employeesAbility to motivate employees Communications with the publicCommunications with the public
77
Best PracticesBest Practices
88
Identifying IndicatorsIdentifying Indicators
Define Scope of Indicators Define Scope of Indicators Conduct Meetings with External Stakeholders and Conduct Meetings with External Stakeholders and
Internal StaffInternal Staff Apply the “Logic Model”Apply the “Logic Model” Develop Guiding PrinciplesDevelop Guiding Principles Select Criteria for Evaluating Potential Indicators Select Criteria for Evaluating Potential Indicators Develop Common Definitions for Key TermsDevelop Common Definitions for Key Terms Inventory Existing Data SourcesInventory Existing Data Sources Select an Appropriate Combination of IndicatorsSelect an Appropriate Combination of Indicators
99
Criteria for Evaluating IndicatorsCriteria for Evaluating Indicators
RelevantRelevant: to goals, objectives, and priorities : to goals, objectives, and priorities TransparentTransparent: promote understanding of program: promote understanding of program CredibleCredible: based on data that is complete and : based on data that is complete and
accurateaccurate FunctionalFunctional: easy to use: easy to use FeasibleFeasible: value to program outweighs cost: value to program outweighs cost ComprehensiveComprehensive: cover important operational : cover important operational
aspectsaspects
1010
Developing IndicatorsDeveloping Indicators
Select final indicators using criteria Select final indicators using criteria Establish internal work teams to:Establish internal work teams to:
Define measures in more detailDefine measures in more detail Review existing data for relevancyReview existing data for relevancy Develop new information collection and reporting processesDevelop new information collection and reporting processes Establish schedule for testing and implementing indicatorsEstablish schedule for testing and implementing indicators
Conduct pilot projects to test indicators and correct Conduct pilot projects to test indicators and correct problemsproblems
Resolve technical and methodological issuesResolve technical and methodological issues Execute implementation in phasesExecute implementation in phases
1111
Using IndicatorsUsing Indicators
Monitor Implementation with Regular Reports Monitor Implementation with Regular Reports Use Indicators to Analyze Performance of Use Indicators to Analyze Performance of
Organizational UnitsOrganizational Units Use Indicators to Review Effectiveness of Specific Use Indicators to Review Effectiveness of Specific
ProgramsPrograms Use Indicators to Report to External AudiencesUse Indicators to Report to External Audiences When Analyzing Data from Indicators, Understand When Analyzing Data from Indicators, Understand
the Data Limitationsthe Data Limitations Assess and Adapt IndicatorsAssess and Adapt Indicators
1212
Applying Indicators to Applying Indicators to Water GovernanceWater Governance
1313
Examples of Indicators Examples of Indicators of Effective Water Governanceof Effective Water Governance
Number of discharge permits revokedNumber of discharge permits revoked Total value of fines levied against violatorsTotal value of fines levied against violators Number of facilities ordered to cease operation until Number of facilities ordered to cease operation until
compliant with water regulationscompliant with water regulations Number of permits granted or updatedNumber of permits granted or updated Number of inspections/patrols conducted per monthNumber of inspections/patrols conducted per month Amount that discharges from facilities have been Amount that discharges from facilities have been
reducedreduced Exceedances addressed or returned to complianceExceedances addressed or returned to compliance
1414
A “Logic Model” of A “Logic Model” of Performance Measurement IndicatorsPerformance Measurement Indicators
InputsInputs
resourcesresources
OutputsOutputs
activitiesactivities
Intermediate Intermediate OutcomeOutcome
behavior changebehavior change
Final OutcomeFinal Outcome
environmental environmental ImpactImpact
PersonnelPersonnel
Funds for Funds for salaries, salaries, contracts, IT, contracts, IT, etc.etc.
Inspections Inspections conductedconducted
Enforcement Enforcement actions takenactions taken
Fines assessedFines assessed
Improved facility Improved facility management management practicespractices
Reduced illegal Reduced illegal dischargesdischarges
Greater Greater understanding of understanding of how to complyhow to comply
Improved ambient Improved ambient water qualitywater quality
Improved human Improved human and ecological and ecological healthhealth
1515
Exercise: Applying the “Logic Model”Exercise: Applying the “Logic Model”InputsInputs
resourcesresources
OutputsOutputs
activitiesactivities
Intermediate Intermediate OutcomeOutcome
behavior changebehavior change
Final Final
OutcomeOutcome
environmental environmental impactimpact
1616
ConclusionConclusion