DESIGNING SUSTAINABLE PRGORAMS FOR PUBLIC HEALTH IMPACTfamilymed.uthscsa.edu/research08/pcrmsc/21st_2007/presentations/24... · DESIGNING SUSTAINABLE PROGRAMS FOR PUBLIC HEALTH IMPACT

DESIGNING SUSTAINABLE DESIGNING SUSTAINABLE PROGRAMS FOR PROGRAMS FOR

PUBLIC HEALTH IMPACTPUBLIC HEALTH IMPACT

Paul Estabrooks, Ph.D.Paul Estabrooks, Ph.D.Russell E. Glasgow, Ph.D.Russell E. Glasgow, Ph.D.

22

OVERVIEWOVERVIEW

General Issues in Creating Practical, General Issues in Creating Practical, Sustainable ProgramsSustainable Programs

Producing Public Health ImpactProducing Public Health Impact——The The RERE--AIM ModelAIM Model

Creating Sustainable ProgramsCreating Sustainable Programs

33

BARRIERS TO IMPLEMENTATION BARRIERS TO IMPLEMENTATION AND DISSEMINATIONAND DISSEMINATION

Characteristics of the Intervention: Characteristics of the Intervention:

Context Adaptable?Context Adaptable?

High costHigh costIntensive time demandsIntensive time demands

High level of staff expertise requiredHigh level of staff expertise requiredDifficult to learn or understandDifficult to learn or understandNot packaged or Not packaged or ““manualizedmanualized””

Not developed considering user needsNot developed considering user needsNot designed to be selfNot designed to be self--sustainingsustainingHighly specific to particular settingHighly specific to particular settingNot modularized or customizableNot modularized or customizable

44

BARRIERS TO IMPLEMENTATION AND DISSEMINATIONBARRIERS TO IMPLEMENTATION AND DISSEMINATION(cont.)(cont.)

Characteristics of Potential Adopting Characteristics of Potential Adopting ““SettingsSettings””: :

Context IssuesContext Issues

Competing demands occurCompeting demands occurProgram imposed from outsideProgram imposed from outside

Finance or organizations are unstableFinance or organizations are unstableClients and setting have specific needsClients and setting have specific needs

Resources are limitedResources are limitedTime is limitedTime is limited

Organizational support is limitedOrganizational support is limitedPrevailing practices that work against innovationPrevailing practices that work against innovation

Perverse incentives or regulations that oppose changePerverse incentives or regulations that oppose change

55

BARRIERS TO IMPLEMENTATION AND DISSEMINATIONBARRIERS TO IMPLEMENTATION AND DISSEMINATION(cont.)(cont.)

Characteristics of Research Design Used: Characteristics of Research Design Used:

Context Informative?Context Informative?

Not relevant, diverse, or representative:Not relevant, diverse, or representative:Sample of patientsSample of patientsSample of settingsSample of settingsSample of cliniciansSample of clinicians

Failure to evaluate costFailure to evaluate costFailure to assess implementationFailure to assess implementationFailure to evaluate maintenanceFailure to evaluate maintenanceFailure to evaluate sustainability Failure to evaluate sustainability

Glasgow RE, Marcus AC, Bull SS. Disseminating effective cancer screening interventions. Cancer 2004;101:1239-1250

66

BARRIERS TO IMPLEMENTATION AND DISSEMINATIONBARRIERS TO IMPLEMENTATION AND DISSEMINATION (cont.)(cont.)

Interactions among the Three Other Barrier Types; FitInteractions among the Three Other Barrier Types; Fit

Because of barriers, the program reach or participation is lowBecause of barriers, the program reach or participation is low

Intervention is not flexibleIntervention is not flexible

Intervention is not appropriate for the target populationIntervention is not appropriate for the target population

Staffing pattern does not match intervention requirementsStaffing pattern does not match intervention requirements

Inconsistent organization and intervention philosophiesInconsistent organization and intervention philosophies

Inability to implement intervention adequatelyInability to implement intervention adequately

Glasgow et al. Translation Research in Diabetes…In: Evidence-based Endocrinology; VM Montori (Ed).Humana Press, Totowa, NJ. Pages 241-256, 2005.

77

As Tested

CriticalElements

Program or Policy

Simplified Systems Model for Translational Research

Organization

Clinic

Program Delivery

Staff

Delivery Site(s)

Fit

ResearchDesign

AppropriateFor Question

PartnershipBroader HealthPolicy and

CulturalContext

Estabrooks PA, Glasgow RE. Am J Prev Med 2006;31(4S):S45-S56

88

“The law of halves” … a story

99

ULTIMATE IMPACT OF MAGIC DIET PILLULTIMATE IMPACT OF MAGIC DIET PILL

50% of Clinics Use Adoption 50%

50% of Clinicians Prescribe Adoption 25%

50% of Patients Accept Medication Reach 12.5%

50% Follow Regimen Correctly Implementation 6.2%

50% of Those Taking Correctly Benefit Effectiveness 3.2%

50% Continue to Benefit After 6 Months Maintenance 1.6%

Dissemination Step Concept % Impacted

1010

MORAL OF THE STORY?MORAL OF THE STORY?

(Each step of the dissemination(Each step of the disseminationsequence, or each sequence, or each ““RERE--AIMAIM””

dimension is important)dimension is important)

““Focus on the DenominatorFocus on the Denominator””

1111

RERE--AIM FRAMEWORK FOR AIM FRAMEWORK FOR FOCUSING ON KEY ISSUES IN FOCUSING ON KEY ISSUES IN

INTEGRATING RESEARCH INTO INTEGRATING RESEARCH INTO PRACTICEPRACTICE

1212

HISTORY/BACKGROUND HISTORY/BACKGROUND FOR REFOR RE--AIMAIM

Late 1990s: Late 1990s: Increasingly clear that major Increasingly clear that major problems moving research into practice.problems moving research into practice.

Helpful models for understanding (e.g., Helpful models for understanding (e.g., Rogers) and planning (e.g., Green & Rogers) and planning (e.g., Green & KreuterKreuter) ) health care programs but no systematic health care programs but no systematic models for translation.models for translation.

Almost total focus on efficacy. Assumed that Almost total focus on efficacy. Assumed that linear linear ““automaticautomatic”” process of efficacy process of efficacy effectiveness effectiveness dissemination.dissemination.

1313

RERE--AIM TO HELP PLAN, EVALUATE, AIM TO HELP PLAN, EVALUATE, AND REPORT STUDIESAND REPORT STUDIES

RR IncreaseIncrease RReacheach

EE IncreaseIncrease EEffectivenessffectiveness

AA IncreaseIncrease AAdoptiondoption

I I IncreaseIncrease IImplementationmplementation

MM IncreaseIncrease MMaintenanceaintenance

Glasgow, et al. Ann Behav Med 2004;27(1):3-12

1414

PURPOSES OF REPURPOSES OF RE--AIMAIM

To broaden the criteria used to evaluate To broaden the criteria used to evaluate programs to include external validity and programs to include external validity and context.context.

To evaluate setting issues relevant to program To evaluate setting issues relevant to program adoption, implementation, and sustainability.adoption, implementation, and sustainability.

To help close the gap between research studies To help close the gap between research studies and practice by:and practice by:

Informing design of interventionsInforming design of interventions

Providing guides for Providing guides for adopteesadoptees

Suggesting standard reporting criteria to Suggesting standard reporting criteria to increase transparent reportingincrease transparent reporting

1515

DefinitionDefinition: The number, percent of target : The number, percent of target audience, and audience, and representativenessrepresentativeness of those of those who participate.who participate.

ExampleExample: 65% of chronic illness patients : 65% of chronic illness patients invited to group medical visit attended initial invited to group medical visit attended initial session; those declining more likely to be session; those declining more likely to be Latino.Latino.

RREE--AIM ELEMENTS: AIM ELEMENTS: REACHREACH

Key IssuesKey Issues: Does program reach those at : Does program reach those at highest risk? Are different promotional highest risk? Are different promotional approaches or visit options required?approaches or visit options required?

1616

DefinitionDefinition: Change in temporally appropriate : Change in temporally appropriate outcomes, and impact on quality of life and any outcomes, and impact on quality of life and any adverse outcomes.adverse outcomes.

ExampleExample: Mailed reminder/telephone outreach : Mailed reminder/telephone outreach program increased colon cancer and program increased colon cancer and mammography screening rates by 20%; with no mammography screening rates by 20%; with no adverse effects on quality of life or cardiovascular adverse effects on quality of life or cardiovascular screening rate.screening rate.

RREE--AIM ELEMENTS: AIM ELEMENTS: EFFECTIVENESSEFFECTIVENESS

Key IssuesKey Issues: Logic model helps to clarify anticipated : Logic model helps to clarify anticipated effects; quality of life provides common metric effects; quality of life provides common metric across conditions and interventions; anticipate across conditions and interventions; anticipate unintended consequences.unintended consequences.

1717

RERE--AAIM ELEMENTS: IM ELEMENTS: ADOPTIONADOPTION

DefinitionDefinition: Number, percent and : Number, percent and representativenessrepresentativeness of settings and clinicians of settings and clinicians who participate.who participate.

ExampleExample: Six months after CME introduction, 52% : Six months after CME introduction, 52% of primary care physicians have used panel of primary care physicians have used panel management toolmanagement tool——but only 30% of specialty but only 30% of specialty care providers.care providers.

Key IssuesKey Issues: Need to focus on : Need to focus on ““denominatordenominator”” and and barriers among nonbarriers among non--users. Do initial users. Do initial adopteesadopteesinclude peer opinion leaders?include peer opinion leaders?

1818

RERE--AAIIM ELEMENTS: M ELEMENTS: IMPLEMENTATIONIMPLEMENTATION

DefinitionDefinition: Extent to which a program or policy : Extent to which a program or policy is delivered consistently, and the time and is delivered consistently, and the time and costs of the program.costs of the program.

ExampleExample: Patients being asked about their : Patients being asked about their race/ethnicity 65% of the time; takes an race/ethnicity 65% of the time; takes an average 30 seconds of time, but not all average 30 seconds of time, but not all reception staff asking consistently. reception staff asking consistently.

Key IssuesKey Issues: Consistency across staff, program : Consistency across staff, program components, and time. Balance between components, and time. Balance between fidelity and local customization.fidelity and local customization.

1919

RERE--AIAIMM ELEMENTS: ELEMENTS: MAINTENANCEMAINTENANCEDefinitionDefinition: :

Individual/member target: LongIndividual/member target: Long--term effects and term effects and attrition.attrition.

Setting/clinician: Extent of discontinuation, Setting/clinician: Extent of discontinuation, modification, or sustainability of program. modification, or sustainability of program.

ExampleExample: At one: At one--year followyear follow--up, was 58% attrition from up, was 58% attrition from Internet weight loss program; those present Internet weight loss program; those present maintained weight loss. Only 40% of clinicians maintained weight loss. Only 40% of clinicians initially referring continued to do so.initially referring continued to do so.

Key IssuesKey Issues: Does attrition bias results; qualitative : Does attrition bias results; qualitative approaches to understanding program adaptation.approaches to understanding program adaptation.

2020

RECOMMENDED PURPOSE OF RECOMMENDED PURPOSE OF TRANSLATION/EFFECTIVENESS RESEARCHTRANSLATION/EFFECTIVENESS RESEARCH

To determine the characteristics of interventions To determine the characteristics of interventions that can:that can:

ReachReach large numbers of people, especially large numbers of people, especially those who can most benefit.those who can most benefit.

Be widely Be widely adoptedadopted by different settings.by different settings.

Be consistently Be consistently implementedimplemented by staff members by staff members with moderate levels of training and expertise.with moderate levels of training and expertise.

Produce Produce replicablereplicable and and longlong--lastinglasting effects (and effects (and minimal negative impacts) at reasonable cost.minimal negative impacts) at reasonable cost.

2121

RERE--AIM SELFAIM SELF--RATING EXERCISERATING EXERCISE

Choose a specific program youare considering for implementation

2222

RATE (RERATE (RE--AIM) YOUR PROGRAMAIM) YOUR PROGRAM

REACHREACH

1. What percent of your target population has received or participated in your program?

2. How confident are you that your program is successfully attracting all members of your target population regardless of race/ethnicity, gender, age, or socio-economic status?

1 2 3 4 5 6 7 8 9 10

____ %

Not at all Somewhat Completely

2323


EFFECTIVENESSEFFECTIVENESS3. To what extent is your program achieving the full key

targeted outcomes that you had hoped for?

4. How confident are you that your planned program is being implemented without producing unintended adverse consequences (e.g., staff doing less of other recommended activities)?

1 2 3 4 5 6 7 8 9 10Not at all Somewhat Completely


2424


ADOPTIONADOPTION

1 2 3 4 5 6 7 8 9 10

5. What percent of other clinics or organizations like yours do you estimate will be willing to participate in your innovation after you are done testing?

6. How confident are you that your program will be adopted by those settings and staff who provide services for people in your target population who have the highest need?


____ %

2525


IMPLEMENTATIONIMPLEMENTATION

1 2 3 4 5 6 7 8 9 10

7. What percent of the time are all the various program components of your intervention are being consistently delivered as intended?

8. How confident are you that the program can be delivered by a variety of staff representing a variety of positions and levels of the organization?


____ %

2626


MAINTENANCEMAINTENANCE


10. How confident are you that your program will be sustained in your setting 2 years after it is introduced?

9. How confident are you that the program will produce lasting benefits for participants (1-2 years or longer)?


1 2 3 4 5 6 7 8 9 10

2727

RERE--AIM SCORING AND PROFILE SHEETAIM SCORING AND PROFILE SHEETScoreScore

1) Reach = Percent in Item #1 plus (10 x Item 2 Score) =1) Reach = Percent in Item #1 plus (10 x Item 2 Score) = ____/2 =____/2 = ________

2) Effectiveness = (10 x Score on Item #3) plus (10 x Item 4 2) Effectiveness = (10 x Score on Item #3) plus (10 x Item 4 Score) =Score) =

____/2 =____/2 = ________

3) Adoption = Percent in Item #5 plus (10 x Item 6 Score) =3) Adoption = Percent in Item #5 plus (10 x Item 6 Score) = ____/2 =____/2 = ________

4) Implementation = Percent in Item #7 plus (10 x Item 8 4) Implementation = Percent in Item #7 plus (10 x Item 8 Score) =Score) =

____/2 =____/2 = ________

5) Maintenance = (10 x Item 9 Score) plus (10 x Item 10 5) Maintenance = (10 x Item 9 Score) plus (10 x Item 10 Score) =Score) =

____/2 =____/2 = ________

2828

RE-AIM PROFILE

2929

A couple of other programs

•Diabetes prevention class

•Environmental-level physical activity intervention

•Individual-level physical activity intervention

Application of Model to Diabetes Prevention

Tested in

Be a Fat DetectiveThree Ways to Eat Less Fat

Healthy Eating/Move Those MusclesBeing Active: A Way of Life

Tip the Calorie BalanceTake Charge of What’s Around You

Problem SolvingThe Four Keys to Healthy Eating Out

Talk Back to Negative ThoughtsThe Slippery Slope of Lifestyle Change

Jump Start Your Activity PlanMake Social Cues Work for You

You Can Manage Stress Ways to Stay Motivated

DPP Intervention Organization

Clinic

Dietitians

Delivery Sites

Fit

DemonstrationProject

AppropriateFor Question

Partnerships across organizationalstructure defined necessary results for decision to deliver broadly across multiple clinics

Broader Health Policy and Cultural Context

1. National Organizational CampaignTo Promote Healthy Lifestyles

2. Prevention Department Desire to Provide Diabetes Prevention Program

Diverse Samples

Multiple Settings

Freq

uent

Con

tact

Research Staff D

elivery

Spac

e Li

mits

Limited Staff Tim

e

Approval of PIC & MOA

Clinic Staff Engagement

Prevention Department

Primary C

are

Scheduling, Cost, & Co-pay

Elec

tron

ic M

edic

al R

ecor

d

Re-invention of intervention retainingcritical elements but drastically reducing

contact

Critical Elements

Note: PIC=Physician in Charge; MOA=Medical Office Administrator

Participant PathParticipant Path

5 weeks pass

1 week passes

7 weeks pass

1 week passes

1 week passes

RandomizationWeek 0

Mailing 2: WEEK 5

Mailing 3WEEK 13

Mailing 4WEEK 21

6 month follow-up visitTreadmill; Accelerometer;

Surveys; WeightWEEK 26

18 month follow-up visit

Treadmill; Accelerometer; Surveys; Weight18 MONTHS

1 week passes

2 weeks pass

Computer session; Choose rec facility

WEEK 1

RecruitmentDuring Stress

Test VisitTreadmill;

Accelerometer; Surveys; Weight;

Height

DAY 1

Mailing 1: WEEK 2

IVR 2WEEK 12

IVR 3WEEK 20

KEY:Clinic Visit IVR Call MAILING

12 month follow-up visit

Accelerometer; Surveys; Weight

WEEK 52

Begin contacting participants for 6-

mo f/u

7 weeks pass

IVR 1WEEK 4

Environmental Intervention ArmEnvironmental Intervention Arm

Interactive Mapping InterfaceInteractive Mapping Interface

EnvironmentEnvironment

What to do when you receive a voucher: 1) Establish a one year membership with the study participant

as you normally would for anyone interested in arranging a membership, effective immediately.

2) The study participant will give you his/her study voucher instead of paying for the one year membership.

3) Be sure that the study participant’s name is written on the voucher and fax it to the research study project manager (contact information listed below).

4) Kaiser Permanente will disperse payment for recreation center memberships once a week. The research project manager will phone the facility to make payment by credit card every Friday. Membership rights should be granted to the study participant immediately, despite the fact that you may not received payment on the day that membership begins. This payment method has been approved by the facility recreation manager.

As part of your partnership with the CardiACTION research study, you will receive vouchers for a free 6-month membership from some of our study participants.

Kaiser Permanente, Clinical Research Unit2550 S. Parker Rd., Aurora, CO 80014PHONE: 303-636-3117FAX: 303-636-3159

Name of Recipient:

One year Period Begin Date:

RECREATION FACILITY NAMEFACILITY ADDRESS

CITY, STATE ZIPPHONE

Please contact Jane Smith with questions: 303-636-3117

Instructions for Participating Recreation Facilities CardiACTION Research Study

This certificate is redeemable for a FREE one year membership at

Map #1. From Your Home

£¤470

§̈¦I 225

Arapahoe Rd

Peoria St

Parker Rd

Cherry Creek State Park

Your Home

Horizon Park

Hampden Ave

Cham

bers Rd

Quincy Ave

Tower R

d

Heartwood Athletic Club

Cherry Creek Equestrian Trail

Gun C

lub Rd

Arapahoe Rd

home work rec. center park lake

Nearby Parks: 1. Horizon Park: 3.00miles north from your home, features include fields and walking trails.

Walking/Driving direction : Start out going NORTHEAST on E. Powers Pl → Turn RIGHT onto E. Berry Dr. → Turn LEFT onto S. Danube St. → Turn RIGHT onto E. Smoky Hill Rd. → Turn LEFT onto S. Himalaya Rd. → End at Horizon Park.

2. Cherry Creek State Park: 2.50 miles west from your home, wildlife habitat, multi-user trails, pond and creek, camping, picnic sites (4201 South Parker Road, Aurora, CO 80014).

Walking/Driving direction: Start out going NORTHEAST on E. Powers Pl → Turn RIGHT onto E. Berry Dr. → Turn RIGHT onto S. Danube St. → Turn RIGHT onto E. Orchard St. → Turn RIGHT onto S. Parker Rd. → End at Cherry Creek State Park.

Parks, Trails and Recreation Centers

Map #2. From Your Work

§̈¦I 70

Colfax AveM

oline St

26th Ave Smith Rd

§̈¦I 225

Syracuse St

d

Ironton St.

Montview Blvd.

Colfax Ave

Your Work

Highline Canal Trail

Andrews Dr.

Rocky Mountain ArsenalHavana St.

Tower R

d.

Montbello Central Park

river major road trail

Nearby Parks: 1. Montbello Central Park: 2.00 miles northwest from your work, features include a baseball field, soccer field, lighted softball field and tennis courts.

Walking/Driving direction: Start out going SOUTH on Kittredge St. → Turn Right onto E. 40th Ave. → Turn RIGHT onto Andrews Dr. → End at Montbello Central Park.

2. Rocky Mountain Arsenal: 2.24 miles north from your work, 2400 acres, features include fishing, nature programs and tours, wildlife watching..

Walking/Driving direction: Start out going SOUTH on Kittredge St. → Turn Right onto E. 40th Ave. → Turn RIGHT onto Chambers Rd. → End at Rocky Mountain Arsenal.

Mail Example

Automated telephone messages

Automated telephone messages

• Example message…

• Public courts and recreation fields are great resources for getting in some physical activity. Basketball, tennis, lawn bowling, badminton, and soccer….are all activities that are fun and can be done at many places in Denver!

EnvironmentEnvironment

Env_OutsideResources_Options.wav

1.Local Parks

2.Paths

& Trails

3.Courts

& Fields

Env_OutsideResources_CourtsFields.wav

Env_OutsideResources_PathsTrails.wav

Env_OutsideResources_Parks.wav

Env_OutsideResources_ArrangeNewCall.wav

Call Number

1

Env_WorkResources_Options.wav

1.Fitness Options

2.Existing

3.Exercis

es

Env_WorkSimpleEX.wav

Env_WorkResources_Existing.wav

Env_WorkResources_Afford.wav

Env_WorkResources_ArrangeNewCall.wav

GoTo: Concluding Assessment Flow Chart

2

From: Individual Arm Entry Flow Chart:

Combo_OutsideResources_Options.wav

From: Environmental Arm Entry Flow

Chart:

4. Already know .

Pre_Env_OutsideResources_Options.wav Pre_Env_WorkResources_Options.wav

Env_HomeResources_Options.wav

1.Afford

2.Existing

3.Equipment

Env_HomeResources_Equipment.wav

Env_HomeResources_Existing.wav

Env_HomeResources_Afford.wav

Env_HomeResources_ArrangeNewCall.wav

4. Already know

Pre_Env_HomeResources_Options.wav

3

4. Already Know

Individual Intervention Arm Individual Intervention Arm

CardiACTIONplanCardiACTIONplan

IndividualIndividual




Automated telephone callsAutomated telephone calls

• Tailored based upon goal attainment

• Potential to complete new action plan• Change Barriers

• Change Strategies

• Change Goals

• Assess mediators used for mailing tailoring


4444

RE-AIM PROFILE

4545

RE-AIM PROFILE

4646

RE-AIM PROFILE

4747

RE-AIM PROFILE

4848

RE-AIM Questions to Ask and Ways to Enhance Overall Impact

RE-AIMDimension

Questions to Ask of Potential Programs Possible Ways to Enhance Dissemination

REACH(Individual Level)

What percent of the target population comes into contact?

Does program reach those most in need?

Will participants be representative of the targeted population?

Formative evaluation with potential users with those declining

Small scale recruitment studies to test methods

Identify and reduce barriers

Use multiple channels of recruitment

EFFECTIVENESS (Individual Level)

Does program achieve key targeted outcomes?

Does it produce unintended adverse consequences?

How will impact on quality of life (QOL) be assessed?

Incorporate more tailoring to individual

Reinforce via repetition, multiple modalities, social support and systems change

Use stepped care approach

Evaluate adverse outcomes and QOL for program revision and cost-to-benefit analyses

ADOPTION (Setting/Organizational Level)

What percent of target settings and organizations will use?

Will these organizations having underserved or high-risk populations use it?

Does program help the organization address its primary mission?

Conduct formative evaluation with adoptees and settings that decline

Recruit settings that have most contact with target audience

Provide different cost options and customization of intervention

Develop recruitment materials outlining program benefits and required resources

IMPLEMENTATION (Setting/Organizational Level)

How many staff within a setting will try this?

Can different levels of staff implement the program successfully?

Are different components delivered as intended?

Provide delivery staff with training and technical assistance

Provide clear intervention protocols

Consider automating all or part of the program

Monitor and provide staff feedback and recognition for implementation

MAINTENANCE (Individual and Setting Levels)

Does the program produce lasting effects at individual level?

Can organizations sustain the program over time?

Are those persons and settings that show maintenance those most in need?

Reduce level of resources required

Incorporate "natural environmental" and community supports

Conduct follow-up assessments and interviews to characterize success at both levelsIncorporate incentives and policy supports

4949

5050

A MIXED SUCCESS STORYA MIXED SUCCESS STORY……

Sure Sustainability Is Important, Sure Sustainability Is Important, But How Do You Do It? But How Do You Do It?

5252

OutlineOutline

Essential Elements for Sustainability Essential Elements for Sustainability

Developing a sustainability action planDeveloping a sustainability action plan

Active for Life Case StudyActive for Life Case Study

RecommendationsRecommendations

5353

Essential for SustainabilityEssential for Sustainability

Producing Evidence and Materials to Promote the Producing Evidence and Materials to Promote the Program. Program.

Funding Development Committee. Funding Development Committee.

Fostering & Utilizing Program Champions.Fostering & Utilizing Program Champions.

Keeping the Community Involved.Keeping the Community Involved.

Planning for a sustainable program delivery team. Planning for a sustainable program delivery team.

Integration into Existing Operations Integration into Existing Operations

Have markers to let you know when you are Have markers to let you know when you are ““donedone””..

5454

Sustainability Action PlanSustainability Action Plan

Purpose: To provide a contextually flexible process Purpose: To provide a contextually flexible process that will result in a higher likelihood of the programs that will result in a higher likelihood of the programs becoming institutionalized and maintained. becoming institutionalized and maintained.

Step 1: Address the factors that could influence the Step 1: Address the factors that could influence the sustainability of your program.sustainability of your program.

Step 2: Examine the current and future resources Step 2: Examine the current and future resources available for program delivery and identify changes available for program delivery and identify changes that could make the program more sustainable.that could make the program more sustainable.

Step 3: Detail specific strategies to ensure Step 3: Detail specific strategies to ensure sustainability. sustainability.

5555

Step 1 OverviewStep 1 Overview

Address the factors that could influence Address the factors that could influence the sustainability of your program. the sustainability of your program.

Sustainability Needs AssessmentSustainability Needs Assessment

Initial PlanInitial Plan

Enabling StrategiesEnabling Strategies

5656

Step 2 OverviewStep 2 Overview

Identify potential operational and program Identify potential operational and program adaptations that you think will make the adaptations that you think will make the program more sustainable while at the same program more sustainable while at the same time keeping the programtime keeping the program’’s essential elements s essential elements and examine the current and future resources and examine the current and future resources available to deliver the program. available to deliver the program.

Adaptation ActivityAdaptation Activity

Connections MapConnections Map

Funding Matrix WorksheetFunding Matrix Worksheet

5757

Step 3 OverviewStep 3 OverviewLongLong--term Sustainability: Putting it all term Sustainability: Putting it all

together with planning and periodic retogether with planning and periodic re--evaluation.evaluation.What are the action items?What are the action items?Who is responsible?Who is responsible?When will it be done?When will it be done?–– Producing Evidence and Materials to Promote the ProgramProducing Evidence and Materials to Promote the Program–– Funding Development Committee. Funding Development Committee. –– Fostering & Utilizing Program Champions. Fostering & Utilizing Program Champions. –– Keeping the Community Involved. Keeping the Community Involved. –– Planning for a sustainable program delivery team. Planning for a sustainable program delivery team.

Integration into Existing Operations. Integration into Existing Operations. –– After 6After 6--12 months review the evaluation criteria from each 12 months review the evaluation criteria from each

of the action plansof the action plans

5858

Active for LifeActive for Life

12 sites funded by RWJF to deliver one 12 sites funded by RWJF to deliver one of two evidenceof two evidence--based physical based physical activity programs for older adultsactivity programs for older adults

Completed the 3 step sustainability Completed the 3 step sustainability action planaction plan

Completed a final survey reflective of Completed a final survey reflective of perceptions of potential for the project perceptions of potential for the project to be sustained after grant award to be sustained after grant award periodperiod

5959

Likelihood of Sustainability

0

20

40

60

80

100

Definite Hope to Littlechance

No Chance

Perc

ent o

f Gra

ntee

s

6060

Program Adaptations

0

20

40

60

80

100

Original Minor Major Not

Perc

ent o

f Gra

ntee

s

6161

Projected AdaptationsProjected AdaptationsMajor:Major:

–– Use program as a basis for expansion (e.g., nutrition; Use program as a basis for expansion (e.g., nutrition; stress)stress)

–– Shift telephone counseling principles to current inShift telephone counseling principles to current in--person staff contacts. person staff contacts.

–– Offer through worksites to all adults over 18 years oldOffer through worksites to all adults over 18 years old–– Change program length Change program length –– Combine with or use as an adjunct to ongoing Combine with or use as an adjunct to ongoing

programsprograms

Minor:Minor:–– Fee for use programFee for use program–– Reduce handouts or replace with personal diariesReduce handouts or replace with personal diaries–– Use volunteer instructorsUse volunteer instructors–– Use program resources that are on lineUse program resources that are on line–– Reduce participant incentivesReduce participant incentives

62626262

Funding Proportions for Funding Proportions for SustainabilitySustainability

Internal Organizational Funds:Internal Organizational Funds:–– Average 45%; Range 0Average 45%; Range 0--100%100%

GrantsGrants–– Average 25%; Range 0Average 25%; Range 0--80%80%

External sourcesExternal sources–– Average 11%; Range 0Average 11%; Range 0--30%30%–– Includes sources such as state funds Includes sources such as state funds

or foundation supportor foundation supportFees for ServiceFees for Service

–– Average 19%; Range 0Average 19%; Range 0--80%80%

63636363

Funding Proportions for Funding Proportions for SustainabilitySustainability

Future Financial Support by Likelihood of Sustainability

0

2040

6080

100

Definite Hope to Definite Hope to Definite Hope to Definite Hope to

External Grant Internal Fees

Perc

ent o

f Fun

ding

6464

Board of Directors or ChampionsBoard of Directors or Champions

About 50% of About 50% of ““definitesdefinites”” compared to 25% of compared to 25% of ““hope hope toto’’ss”” have such a grouphave such a group

Examples of members of champion groups:Examples of members of champion groups:–– Advisory board of champions who provide Advisory board of champions who provide

support and advocacysupport and advocacy

–– Executive directors of organization who Executive directors of organization who provide assistance in grant seeking, hosting of provide assistance in grant seeking, hosting of program, and support marketing/recruitment program, and support marketing/recruitment effortsefforts

–– Regional health services board that is Regional health services board that is responsible for those who administer AFL.responsible for those who administer AFL.

6565

Infrastructure and Plans for Evaluation Infrastructure and Plans for Evaluation

Infrastructure For Program Delivery

0

20

40

60

80

100

Definite Hope to Definite Hope to Definite Hope to Definite Hope to

Manage Facilitators Evaluation Eval Plans

Perc

ent

* With one exception, all sites indicated the pilot evaluation data was important in the sustainability process

6666

Examples of InfrastructureExamples of InfrastructureCommitment from public health leadership to Commitment from public health leadership to continue the servicescontinue the servicesExisting staff that can continue to support the Existing staff that can continue to support the project beyond the Grant period. project beyond the Grant period. ALED program is in line with mission and vision, ALED program is in line with mission and vision, management in place to continue to directmanagement in place to continue to directCreated health manager position to oversee this Created health manager position to oversee this programprogramAFL coordinator integrated into regular staff with AFL coordinator integrated into regular staff with additional dutiesadditional dutiesProgram Management integrated into current Program Management integrated into current staff rolesstaff roles

6767

Mission, Vision, and PolicyMission, Vision, and PolicyThe majority of sites have components of the The majority of sites have components of the organizational mission that addressed physical organizational mission that addressed physical activity promotion for older adults.activity promotion for older adults.

Only 2 sites implemented policy directives Only 2 sites implemented policy directives related to AFLrelated to AFL

–– Board of Trustees recommendation for Board of Trustees recommendation for organizational administration to develop organizational administration to develop comprehensive plan to address obesity, poor comprehensive plan to address obesity, poor nutrition and sedentary lifestyles.nutrition and sedentary lifestyles.

–– Redirected Title III recreational funding to agencies Redirected Title III recreational funding to agencies offering evidence based programming. offering evidence based programming.

–– Health District added ALED facilitation and trainer Health District added ALED facilitation and trainer training to fitness focused Health Educators job training to fitness focused Health Educators job description. description.

6868

Changes in MissionChanges in Mission1/3 of grantee organizations had shifts in mission 1/3 of grantee organizations had shifts in mission or structureor structurePositives:Positives:

–– New focus on disease prevention/self managementNew focus on disease prevention/self management–– New focus on obesityNew focus on obesity

NegativesNegatives–– Change in focus from service delivery to broader Change in focus from service delivery to broader

supporting role for larger geographic regionssupporting role for larger geographic regions–– Culture of organization no longer supportive of Culture of organization no longer supportive of

psychosocial service offeringspsychosocial service offerings

6969

Grantee Perceptions of the Grantee Perceptions of the Sustainability Planning ProcessSustainability Planning Process

Organization related:Organization related:–– Proactive planning and increased sense of Proactive planning and increased sense of

accountabilityaccountability–– Template to generate a shared organizational Template to generate a shared organizational

vision on sustainabilityvision on sustainability–– Highlighted elements that may not have been Highlighted elements that may not have been

addressed with the time crunch when addressed with the time crunch when recruiting participantsrecruiting participants

Partnership related:Partnership related:–– Helped identify and coordinate key players, Helped identify and coordinate key players,

their strengths and roles, sharing our visionstheir strengths and roles, sharing our visions–– Formalized processFormalized process——gave some leveragegave some leverage–– Some felt partnerships were less involved with Some felt partnerships were less involved with

this than they expected this than they expected

70707070

What Worked Well for GranteesWhat Worked Well for GranteesWritten plans focused our strategiesWritten plans focused our strategiesFunding Matrix and OutcomesFunding Matrix and OutcomesValues Clarification ExerciseValues Clarification ExerciseIdentifying potential resources for supportIdentifying potential resources for supportBudget developmentBudget developmentTemplates in generalTemplates in generalPushed for collaborationPushed for collaborationIdeas around fostering program and Ideas around fostering program and organizational championsorganizational championsContinual review of action planContinual review of action planNeeds assessment tools & Connections mapNeeds assessment tools & Connections mapOpportunities to share with other granteesOpportunities to share with other granteesAdaptable processAdaptable process

71717171

What DidnWhat Didn’’t Work Well for Granteest Work Well for Grantees

Getting the partners involvedGetting the partners involvedFormality of reportingFormality of reportingSome components werenSome components weren’’t relevant for each t relevant for each granteegranteeFelt difficult to focus on sustainability when Felt difficult to focus on sustainability when recruitment was ongoingrecruitment was ongoingConnections map and funding matrixConnections map and funding matrixSometime asked to provide detail on things Sometime asked to provide detail on things that we didnthat we didn’’t have enough information ont have enough information onDidnDidn’’t come to a true consensus on the best t come to a true consensus on the best approach to sustainabilityapproach to sustainability

7272

SummarySummarySites that have a strong belief the future Sites that have a strong belief the future sustainability of their programs have:sustainability of their programs have:

–– A more balanced plan for financial support from A more balanced plan for financial support from internal, grant, external, and fee for service sourcesinternal, grant, external, and fee for service sources

–– A greater likelihood of having a champion groupA greater likelihood of having a champion group

–– A greater likelihood of infrastructure for AFL A greater likelihood of infrastructure for AFL program management and facilitatorsprogram management and facilitators

Coalitions were dynamic and, on occasion, Coalitions were dynamic and, on occasion, difficult to navigatedifficult to navigate

The sustainability plan was viewed positively by The sustainability plan was viewed positively by grantees, but there was some concern related to grantees, but there was some concern related to focusing on sustainability so early in the focusing on sustainability so early in the process.process.

7373

Questions, Comments, Questions, Comments, & Smart Remarks& Smart Remarks

Documents

DESIGNING SUSTAINABLE PRGORAMS FOR PUBLIC HEALTH IMPACTfamilymed.uthscsa.edu/research08/pcrmsc/21st_2007/presentations/24... · DESIGNING SUSTAINABLE PROGRAMS FOR PUBLIC HEALTH IMPACT