90
Strategic Prevention Framework State Incentive Grant Coordinator Training

Strategic Prevention Framework State Incentive Grant Coordinator Training

Embed Size (px)

Citation preview

  • Strategic Prevention Framework State Incentive Grant Coordinator Training

    History of the Strategic Prevention FrameworkCreated by the Substance Abuse and Mental Health Services Administration (SAMHSA)Similar to other planning processes but somewhat different

    Proven Effectiveness Cross-Site Evaluation at www.spfsig.net Results from SPF SIG Cohorts 1 and 2 SPF SIG Cohorts 3, 4, and 5 are currently providing data

    SPF SIG Coordinator FAQs ContractsSubcontractsMilitary InvolvementIowa SPF SIG WebsiteDue DatesOne Step at a Time

    Working With a Coalition or Collaboration CouncilCountywide CoalitionFocuses on a variety of issues and projectsCan utilize a subcommittee processIncludes required sectorsIncludes military representativeSPF SIG Coordinator does not lead

    Working With a Coalition or Collaboration CouncilCollaboration CouncilEstablished due to no countywide coalitionNot a coalition Provides direction to the county SPF SIG projectIncludes required sectorsIncludes military representativeSPF SIG Coordinator does not lead

    Budget GuidanceReview Budget Guidance DocumentFunding environmental strategies No marketing/promotional materialsNo conferences/trainings outside what IDPH provides

    Project TimelineAssessment and Capacity June - August 2011

    Planning September November 2011

    Implementation and Evaluation December 2011 June 2014

    Upcoming DatesContractor Meetings:November 3, 2011 and April 2, 2012

    Planning Regional Training:September 7 - MoraviaSeptember 8 Sioux CitySeptember 27 - DubuqueSeptember 29 Cedar RapidsRegister at www.trainingresources.org

    Implementation and Evaluation Regional Training: November 8 - MoraviaNovember 10 Sioux CityNovember 14 - DubuqueNovember 17 Cedar RapidsUpcoming Dates

  • Q & A Session

    AssessmentProfile population needs, resources, and readiness to address needs and gapsEvaluationMonitor, evaluate, sustain, and improve or replace those that failImplementationImplement evidence-based prevention programs and activitiesPlanningDevelop a Comprehensive Strategic PlanCapacityMobilize and/or build capacity to address needsThe ProcessTrust the Process

    ReviewAssessment Key conceptsHow does it inform other stepsCapacity Key conceptsHow does it inform other steps

    Step 1. Assessment

    Module OverviewUnder the SPF, counties assess:

    Population needs

    Available resources

    Community readiness

    Assessing Population NeedsLooks at the patterns and effects of substance abuse in particular populations.

    Begins at the State level

    Based on State-level EPI data

    Assessing Population NeedsA State may identify one problem for all communities to address.

    Or, counties may choose from among several options. Either way, counties need to collect additional data.

    Assessing Population NeedsWhy is local assessment critical?

    To change a problem locally, you must understand it. Only then can you identify the best way to solve it.

    Assessing Population NeedsConsequences

    Describe the effects of misuseConsumption Patterns

    Describe how people use or misuse substances

    Assessing Population NeedsConsequences and Consumption : Up Close and Personal

    Who uses alcohol?Where is it happening?What kinds of problems does underage drinking and binge drinking cause?

    Assessing Population NeedsXXX

    Assessing Population NeedsTypes of Data

    Quantitative

    Qualitative

    Taking StockSee what the State has collectedLook for existing survey dataCollect new data

    Setting Priorities

    Setting PrioritiesMagnitude

    AlcoholOxycontin

    Setting PrioritiesChangeability

    AlcoholOxycontin

    Setting PrioritiesImpact

    OxycontinAlcohol

    Setting PrioritiesConcentration

    GirlsBoys

    Setting PrioritiesTime Lapse

    HeroinSmoking

    Assessing Risk & Protective FactorsEach substance abuse problem has its own set of risk and protective factors.

    Factors driving a problem in one county may differ from factors driving it in another county.

    Assessing Risk & Protective FactorsEffective, prevention strategies must address the underlying factors driving a problem.

    Assessing ResourcesWhat resources are available?

    Where are the gaps?

    Assessing ResourcesIdentified NeedsAvailable Resources

    Assessing ResourcesResources include:

    StaffFinancial supportSound organizational structure . . .

    Assessing ResourcesResources also include:

    County efforts to address prevention issuesCounty awarenessPractical experienceKnowledge of local politics and policies

    Assessing Community ReadinessCommunity Readiness

    Willingness to engage in and support prevention efforts

    Cultural Competence and AssessmentInvolving representatives across sectors

    Collecting information in appropriate ways

    Looking in your own backyard

    Sustainability and AssessmentEngage key stakeholders

    Track data sources

    Share assessment findings

    Step 2.Building Capacity

    What is Capacity?Capacity refers to:

    Types and levels of resources available to maintain a prevention system

    Readiness to commit resources to address prevention needs

    What is Capacity?Capacity-building can include:

    Forming partnerships

    Strengthening county prevention systems

    Increasing county awareness

    Cultural Competency and CapacityBroad cultural representation is key to sustaining prevention efforts in the long-term

    Sustainability and CapacitySkills Required:Increase participation and membership

    Build leadership

    Enhance cultural competence

    Improve organizational management and development

    Step 3.Planning

    Planning TasksStart with the problem and data

    Prioritize intervening variables

    Select evidence-based approaches

    Planning TasksIdentify measurable outcomes

    Create an Action Plan and Logic Model

    Consider cultural competency and sustainability

    Prioritizing Intervening Variables

    Change Intervening Variables

    Change Substance Abuse Problems

    Prioritizing Intervening Variables

    Underage Drinking

    Retail AvailabilitySocialAvailabilityLaw Enforcement

    Prioritizing Intervening VariablesChangeabilityDo we have the capacity to change this intervening variable?

    ImportanceHow important is this intervening variable in reducing the problem?

    Selecting StrategiesFederal registries Research or experts in the field (NIAAA, NIDA)Evidence-Based Practice Workgroup, a subcommittee of the SPF SIG Advisory Council

    What is a Logic Model?Ensures that no step along the way will be overlookedExplains the work you are doingIt is the approachLets others know:Where you are goingHow you are getting thereHow you are measuring progressHow you will know when you arrived

    Logic Model

    Action PlanDescribes how the strategies will meet its objectives along with the action steps Delineates responsibilities for achieving the outcomes in your strategic planGives organization to the processPromotes accountability

    Cultural Competence and PlanningEngage various populations in the planning processConsider cultural perspectives when selecting evidence-based strategiesLook at cultural perspective on intervening variableshow they may be viewed in different cultures

    Sustainability and PlanningProvides a Comprehensive PlanHelps build a strong coalitionFocuses on population level changeAttracts funders for future work

    Step 4.Implementation

    Overview of ImplementationAction Plan ReviewFidelity and adaptationFactors that influence implementationCultural competence and sustainability

    Implementing Your Action Plan

    Action PlanWhat you expect to accomplish Your approachAction stepsPeople responsible for completing action stepsTimeline

    Planting Seeds for EBP ImplementationIncrease county awareness of problemExpandinternal andexternalpartnershipsneeded forEBPIntroduce strategies to partners and key stakeholders

    Fidelity refers to the degree to which a strategy is implemented. Strategies that are implemented with fidelity are most likely to achieve their expected outcomes. In other words, they are most likely to be effective.Fidelity

    AdaptationAdaptation - How much, and in what ways, a program is changed to meet local circumstances

    Guidelines for Adaptation

    Consult the experts

    Retain core components

    Adhere to evidence-based principles

    Select strategies with the best initial fit

    Change capacity before changing strategy

    Be consistent with evidence-based principles

    Add, rather than subtractGuidelines for Adaptation

    Barriers to ImplementationOngoing needs for training and informationLack of feedbackMiscommunication

    How can we break through?

    Breaking Down the BarriersOffer training and supportGive meaningful tasksKeep people informed Clear communicationRecognize coalition/ Collaboration Council members

    Cultural Competence and Implementation Effective cultural adaptation is important to implementation

    Sustainability and ImplementationGood implementation involves the communityWhat about sustainability?

  • Environmental Strategy Overview

    What Are Environmental Strategies?Environmental Strategies reduce problems associated with the use of alcohol through changes in the physical, social, legal, and economic environment

    Why Are They Effective?Once in place, many environmental strategies can have immediate effects on availability

    Policy changes do not depend on a constant influx of dollars to keep them going

    Why Are They Effective?

    Changes in the legal, economic and social structures that affect substance use foster shifts in both individual attitudes and community norms

    Local people solve local problems best

    People support what they help create

    Holding young people solely responsible for underage drinking is like holding fish responsible for dying in a polluted stream.

    Additional InformationThe Evidence-Based Practices Workgroup Identified environmental strategies

    IDPH will fund the implementation of these strategies by SPF SIG counties

    The list of approved strategies will be provided to counties during the regional training

    Step 5.Evaluation

    EvaluationMeasures the impact of the SPF and the implementation of strategies, programs, policies and practices

    Evaluation Helps CountiesDescribe what they plan to doMonitor what they are doingImprove servicesDetermine which efforts to sustain

    Five Functions of EvaluationImprovementCoordinationAccountabilityCelebrationSustainability

    Measuring Population-Level ChangeIndividual-Level vs. Population

    Process and Outcome Process EvaluationDocumenting strategy implementation

    Outcome EvaluationDocumenting effects that you expect to achieve after the strategy is implemented

    *Process Evaluation

    If-Then RelationshipsUnderlying a logic model is a series of if-then relationships that express the programs theory of change IF then IF then IF then IF then IF then IF thenSample: Watertown Talk It Up, Lock It Up Media Campaign

    *Outcome EvaluationOver the duration of the strategy, to what extent has:

    School attendance improved?

    Community-wide prevention awareness activities changed adult norms about substance use?

    Youth substance use decreased?

    Evaluation vs. Monitoring

    Cultural Competency and EvaluationInvolve cultural groups in choosing appropriate evaluation methodsKeep the county engaged after data has been collected by involving them in interpretation and dissemination of results

    Sustainability and EvaluationMonitors progress and provides regular feedbackHelps provide a system for continuous improvementTracks county trendsProvides a way for counties to share their stories and results with stakeholders

    How successful was the county in selecting and implementing appropriate strategies?Were these the right strategies?Were stakeholders involved? Were new partners identified? What was the quality of the data?Evaluate the Entire Planning Process

    Features of a SPF EvaluationMeasures population changeMeasures change across multiple strategies and settingsEvaluates the entire planning processIs collaborative

    ***Welcome to Step 1: Assessment.

    *To support prevention planning, the SPF asks communities to assess three things:

    Population needs, including levels of substance abuse and related problems;Available resources to support prevention efforts; and Community readiness to address identified prevention problems or needs.

    In this module, well discuss each of these items separately. Well also look at the relationship of cultural competence and sustainability to assessmenthow these principles can support assessment efforts and how assessment contributes to cultural competence and sustainability.

    Well wait until the next module, which focuses on capacity building, to discuss ways to mobilize and/or strengthen the resources you have identified.*So, lets start with assessing population needs. In the substance abuse prevention world, population-level needs assessment looks at the patterns and effects of substance abuse in particular populations.

    Assessing population needs begins at the State level. Under the SPF, each State is expected to identify one or more substance abuse prevention problems that it wants to address. These priority problems are selected following a careful review of epidemiological data collected at the State level.

    *In some cases, the State identifies one priority problem and expects all communities to address it through their local efforts. In other cases, communities may be asked to choose from among several priority problems.

    In either case, communities will need to collect additional data, either to better understand the problem identified by the State, or to help them decide which of several State-identified problems they should tackle.

    *But why is local assessment so important?

    To change a problem locally, you need to understand what it looks like in your community and the factors that drive it. A comprehensive understanding of the problem will help you identify the most effective strategies for addressing it.

    *As we described in the introduction, problems can be thought about in terms of consequences and consumption patterns.

    Consequences describe the effects of misuse. Some examples of consequences are alcohol poisoning, alcoholrelated traffic crashes, and illness or confusion in older adults.

    Consumption patterns describe how people within a specific group use or misuse substances. For example, binge drinking among 18-25 year olds describes a consumption pattern. So does older people who mix alcohol with their medications.

    *It may be helpful to think about consumption patterns and consequences in terms of underage drinking. This is a serious problem in many communitiesbut what does it look like in your community? Which subpopulations use alcohol? Pre-teens? High-school dropouts? Where is it happening? On school property? At the beach? In the woods? Has underage drinking led to high rates of alcohol-related fatalities? Alcohol poisoning? School-related alcohol problems?

    Who and where describe consumption patterns. The effects are the consequences.*Consider this example:

    A State has high rates of motor vehicle crashes. But, according to State data, these problems arent distributed evenly across the State. Three towns in the northern region have rates four times higher than the State average. If you look more closely at these towns, you will also notice that alcohol consumption rates for males 18-24 are much higher than for other groups. In addition, binge drinking rates for this group of males is twice the State rate. Based on these data, you may want to be targeting males ages 18-24 with your prevention programs.

    Understanding whats going on at the local level is key to developing an effective prevention plan. Looking at local-level data will help you better define your problem and may reveal consumption patterns. You will need this kind of information to select an appropriate prevention approach.*Determining consumption patterns and consequences requires data. There are two basic kinds of dataquantitative and qualitative. Often, communities use a combination of qualitative and quantitative data to get a good handle on their local substance abuse problems.

    Quantitative data are usually reported numerically. An example of quantitative data is the percentage of car crashes caused by teens who have been drinking.

    Sources of quantitative data include counting, checklists, surveys, and analysis of statistics.

    Qualitative data are usually reported in words. Sources of qualitative data include stories, case studies, testimonials, and focus groups.*But, before you embark on a major data collection undertaking, take stock of the information already available to you. There may be more than you think.

    First, see what kinds of data the State has collected on your community. For example, it may have created a community profile for your town, based on epidemiological data it has collected.

    Next, look for existing survey data that might provide the local information youre looking for. For example, information about teen drinking patterns is collected on the Youth Risk Behavior Survey, or YRBS. And many communities routinely administer this survey. Its likely that yours does, too.

    But what if your community doesnt administer the YRBS? Or, what if youre looking for information that isnt collected on this survey? If this is the case, then you may need to collect new data.

    To fill in holes, you may want to look at qualitative data, collected through focus groups or key informant interviews. Or, you may decide to develop a survey to collect quantitative data.

    There are pros and cons to different data collection methods. These are described in our online toolkit. But, as a general rule, keep in mind that while not all data are of the same quality, sometimes good enough is ok. And sometimes its all that you can afford.

    *Once you have collected and reviewed the consequence and consumption data, you will need a process for setting priorities. As nice as it would be to address all of your substance abuse problems, chances are you only have the resources to tackle some of them.

    When setting priorities, communities find it helpful to analyze their data according to four criteria. *The first is magnitude. Magnitude describes the number of people affected by a problem. So, for example, suppose that more youth in your community use alcohol than Oxycontin. If magnitude were your only assessment criteria, you might choose to focus your prevention efforts on alcohol use, rather than Oxycontin use. *But magnitude isnt your only criteria. You also want to look at changeability.

    Changeability describes how easily a problem can be changed. In assessing changeability, ask: Do you know that the problem has been successfully changed in the past? Or if there any evidence-based interventions available to address the problem?

    Since there are many more evidence-based interventions that address alcohol use than address oxycontin use, if changeability were your main criteria, you might still focus your efforts on alcohol use. *And then theres impact.

    Impact describes the depth of a problem across a variety of dimensions, such as health, economic or criminal. So, for example, comparing alcohol and oxycontin use again, oxycontin has more legal ramifications because users typically break the law to get it. In this sense, oxycontin use has a greater impact than alcohol use.

    *Another criteria to consider when setting priorities is concentration or, more specifically, how concentrated the problem is in a specific population. This is important because it may be easier to produce results if you target a specific population group with your efforts.

    For example, data may show that rates of underage drinking are decreasing for teenage boysbut increasing for teenage girls. You might decide, then, to target teenage girls. *Finally, you want to look at the time lapse between consumption behavior and consequence.

    Time lapse describes the amount of time that elapses between consumption and consequence. This can be very different depending on the substance. For example, heroin overdose may be less frequent than tobacco-related deaths, but the time lapse between consumption of heroin and heroin-related death is much shorter than the time between smoking and developing lung cancer.

    And why is this important? Well if you want to show that you are having an impact within a set timeframe, you want to select a problem that you can realistically change within that time frame. Trying to show a reduction in cancer rates within five years will be much more difficult than showing a reduction in heroin use.*Once youve selected your prevention priorities, you will need to assess the factors driving the problem. Each substance abuse prevention problem has its own set of risk and protective factors.

    However, the factors driving a problem in one community may differ from the factors driving it in another community. *One of the most important lessons learned from prevention research is that, in order to be effective, prevention strategies must address the underlying factors driving a problem. It doesnt matter how carefully you implement a program. If its not a good match for the problem, its not going to work.

    So, if your problem is underage drinking, and the underlying factor contributing to this problem in your community is easy access to alcohol, your may decide to develop an intervention that addresses youth access. The best mentoring program in the world, on its own, will not have an impact on your problem. *In addition to assessing population needs, youll also need to assess available resources. In other words, what resources are available to address your priority problems? And where are the gaps?

    Communities take a big step toward effective prevention when they conduct a systematic resource assessment. It gives you a chance to address any gaps, in advance. It also may alert you to cultural resourcessuch as clubs, faith-based organizations, and task forcesthat you will certainly want involved in future planning.

    *Assessing resources is a key step in making a realistic match between identified needs and available resources.

    Imagine that your local data show that enforcement of underage drinking laws is a central factor in youth alcohol use and abuse. You conduct a resource assessment. It reveals that, while your coalitions relationship with parents and teachers is positive, your relationship with the police force is tenuous, at best.

    So . . . how far do you think your coalition will get trying to advance an enforcement strategy without the support of the police? Establishing a good working relationship with the police is the first step in strengthening enforcement. It might not be so important for a mentoring program, but its key to enforcement! *When people hear the word resources, they often think of staff, financial support, and sound organizational structure. And theyre right.

    *But substance abuse prevention resources also include:

    community efforts to address prevention issues,community awareness of those efforts,specialized knowledge about prevention research, theory and practice,practical experience working with particular populations,and knowledge of the ways local politics and policies help or hinder prevention efforts.

    Clearly, you cant assess every resource in town. So youll want to focus your assessment by only looking at resources that are relevant. In other words, resources that are related to your priority problems. A well-planned and focused assessment will produce far more valuable information than one that casts too wide a net.

    At the same time, remember that useful and accessible resources may well be found beyond your coalitionand beyond your community.

    A range of tools can help. In our online guide, weve included tools that will help you assess resources at the individual, organizational, and group levels. There are also tools on finding sources of resource information and on compiling community resource information.

    *In addition to assessing population needs and community resources, you will also need to assess community readiness. Community readiness describes the communitys willingness to engage in and support prevention efforts.

    Data may reveal what problems to addressbut is the community ready to commit resources to address the problems? Readiness should be a key factor in guiding the selection of prevention strategies. To what extent does your community believe there is an underage drinking problem? What are peoples perceptions of the problem? How accurate are they? How do key leaders perceive the problem? How is alcohol portrayed in local media?*Its critical that your assessment represent and reflect all sectors of the community. To do this, you will need to engage in a culturally competent assessment process. This means involving representatives from across sectors in assessment planning and data collection. It also means collecting information, across sectors, in ways that are appropriate and respectful. Finally, it means taking a close look at your own coalition to ensure that the people around the table reflect the diversity of your population.

    Ultimately, your assessment should include information about the cultural and ethnic make-up of your community, how substance abuse problems are perceived among different sectors, who has been engaged in prevention planning and implementation of prevention interventions, and what barriers to participation in prevention efforts exist.*Engaging key stakeholders in the assessment process will contribute to the sustainability of your overall prevention initiative. It will help to ensure their buy-in and lay the foundation for ongoing participation and support. Consider the example we presented earlier. Police who are actively involved in assessment efforts are more likely to support prevention activities that stem from the assessment. They are also more likely to sustain these activities, over time.

    Also, make sure to share your assessment findings with key stakeholders and other community members. The better they understanding baseline issues, the more theyll appreciateand want to sustainall that your prevention efforts accomplish.In Step 1, we talked about assessing capacity. In Step 2 we will focus on building or mobilizing capacity.

    *If you remember, from Assessment, capacity in an organization, partnership, or community refers to two things.

    It refers to the various types and levels of resources available to maintain a prevention system that can respond to local problems.And, it refers to the organization or communitys readiness to commit resources to addressing identified prevention needs.

    Its critical that your community have the capacity to address the substance abuse problems its identified. More specifically, you need to be able to support the prevention programs, policies, and strategies you choose to address these problems. Why? Because program, policies, and strategies that are well-supported are more likely to succeed.

    *In completing Step 2 of the SPF, youll need to consider three kinds of capacity building:

    Youll need to form partnerships.Youll need to strengthen your collaborations.And youll need to increase community awareness.

    In this module, well talk about each of these topics separately. Well also look at how the principles of cultural competence and sustainability can supportand benefit fromyour capacity-building efforts.

    But before going on, lets take a minute to review what we mean by resources. We can divide resources into two main areas:

    *Sustainability encompasses much more thandollars. Your coalitions capacityits membership, relationships, leadershipand organizational strengthis like a goose that lays golden eggs. You haveto take care of the goose! Sadly, some coalitions fall into the routine ofworking to maintain collaboration at the expense of crucial components ofcapacity that led to their initial success. In facing new decisions, think abouthow they affect your coalitions ability to work collaboratively. Are therecreative opportunities for growth? Does a particular strategy or decision limit your capacity?

    *In Step 1, we talked about assessing capacity. In Step 2 we will focus on building or mobilizing capacity.

    *Step 4 is where the rubber hits the roadwhere you do what youve said youre gonna do. In this module, well address several important issues related to implementation.

    *First, well discuss the relationship of fidelity and adaptation to implementation of evidence-based programs.

    Next, well discuss factors that may influence implementation.

    Well talk about action plan development.

    And finally, well discuss ways that cultural competence and sustainability affect implementationand how implementation contributes to cultural competence and sustainability.

    *And finally, its important to consider you history implementing prevention interventions in the past. Whats your track record? Do you have past successes that you can point to proudly and build on? According to one school-based prevention coordinator, The most challenging part [of bringing in a new prevention program] is convincing veteran teachers that it is not just another program to place on the shelf.

    *Adaptation refers to how much, and in what ways, a program, policy, or practice is changed to meet local circumstances.

    Balancing fidelity and adaptation can be tricky because any time you change an intervention, you may be compromising outcomes.

    Even so, implementing a program that requires some adaptation may be more efficient, effective, and cost-effective than designing a program from scratch.

    Adaptations can be planned or un-planned.

    Planned adaptations are made ahead-of-time, to meet identified needs.

    Unplanned adaptations can catch you by surprise. For example, you might plan to conduct all ten sessions of a school-based prevention program, but due to a snow storm and tight scheduling, end up delivering only nine.

    The $10,000 question is: Will this change compromise expected outcomes, and if so, by how much?

    *Weve developed a set of general guidelines for adaptation. We mentioned the first guideline in Step 3: Select an approach that best fits local needs and conditions. Here are some others.

    Select programs that have a large effect size. Effect size refers to the magnitude of the effects of an intervention. Policy change interventions generally have larger effect sizes than classroom-based interventions. The smaller an interventions effect size, the more careful you want to be about changing anythingbecause you dont want to inadvertently compromise any good youre doing. In general, adaptations to programs with large effect sizes are less likely to affect relevant outcomes. You will also want to change capacity before program. It may seem easier to change the program, but increasing local capacity to deliver the program, as it was designed, is a safer choice.

    Consult the experts, including the program developer, an environmental strategies expert, or your evaluator. They may be able to tell you how the intervention has been adapted in the past and how well these adaptations have worked out. Retain core components. Programs, policies, and strategies are more likely to be effective when their core components are maintained. If youre not sure which elements are core, consult the program developer, an environmental strategies expert, or your evaluator. Adhere to evidence-based principles. Programs, policies, and strategies that adhere to evidence-based principles are more likely to be effective, so make sure your adaptations are consistent with the science. And finally, Add, rather than subtract. It is safer to add to a program than to modify or to subtract from it.

    *Multiple factors influence implementation. These include:

    Staff or practitioner selection. Beyond academic qualifications or experience factors, certain practitioner characteristics are difficult to teach in training sessions, and so must be a part of your selection criteria.

    Pre- and in-service training is also important. Trainings are efficient ways to communicate background information, theory, philosophy, and values; and to introduce the components and rationales for key practices. They also provide opportunities to practice new skills and receive feedback in a safe environment.

    Pre- and in-service training is also important. Trainings are efficient ways to communicate background information, theory, philosophy, and values; and to introduce the components and rationales for key practices. They also provide opportunities to practice new skills and receive feedback in a safe environment.

    *But, while training is important, most skills are really learned on the job, with the help of a consultant or coach. Implementation of evidence-based practices and programs requires behavior change at the practitioner, supervisor, and administrative support levels. Training and coaching are the principle ways in which behavior change is brought about.

    Staff and program evaluation also influence implementation. Assessments of practitioner performance and measures of fidelity can provide useful feedback to managers and implementers on the progress of implementation and the usefulness of training and coaching.

    Program evaluation can be used to ensure continued implementation of core intervention elements, over time.

    **Environmental strategies incorporate preventionefforts aimed at changing or influencing communityconditions, standards, institutions, structures,systems and policies. Coalitions should selectstrategies that lead to long-term outcomes. Increasingfines for underage drinking, moving tobaccoproducts behind the counter, not sellingcold, single-serving containers of beer in conveniencestores and increasing access to treatmentservices by providing Spanish-speaking counselors. Analogy of stoplight here.

    *In Step 1, we talked about assessing capacity. In Step 2 we will focus on building or mobilizing capacity.

    **What obstacles were encountered during program implementation?

    **Has youth substance use decreased?

    To learn more about how to develop evaluation questions, see the tips sheet Examples of Evaluation Questions in your toolkit.Evaluation essentially explains what happened (what kind of change; how much; for whom). Monitoring explains how it was operationalized (how did we create the conditions for the change; what did we have to do for how many for that change).

    **For example, you might ask questions like: How successful was the community in selecting and implementing appropriate strategies? Were these the right strategies, given the risk factors the community identified? Were representatives from across the community involved in program planning, selection, and implementation? In what ways were they involved? Were you able to identify potential new partners? What was the quality of the data used in decision making?

    *Another feature of SPF evaluation is that it isnt just about outcomes. Under the SPF, communities are asked to evaluate their entire, 5-step planning process. You will need to show how each step of the SPF connects to the steps around it.