Copyright © 2014 by The University of Kansas Refining the Program Intervention Based on Research

Copyright © 2014 by The University of Kansas

Refining the Program Intervention Based on

Research


What do we mean by refining the

intervention?• Refining the intervention refers to the process

of making your work more effective by using the results of your evaluation to adjust and improve the initiative

• Key questions you should consider:– What are we seeing? – What does it mean? – What are the implications for improvement?


Why should you use your evaluation research to refine the intervention?The primary purpose of evaluation is to ensure that your intervention is having the intended effect, and make adjustments if something is not working


When should you refine the intervention?

• Regularly throughout the life of an intervention• Once a year is typical

• There are other specific times when an evaluation may be especially helpful• When it’s clear that what you’re doing isn’t working• When participants are dropping out at a high rate• Between sessions a time-limited or sequential program• When funders or participants ask you to adjust some aspect of

your program• When funding or other resources are reduced• When the goal or issue changes


Who should be involved in refining the intervention?• Participants

• Staff members

• People who are directly or indirectly involved in supporting the work• This might include educators, government officials, health

professionals, employers, funders, etc.• Since their contribution is needed to make changes successful, it

is important that they have input into the planning of those changes

• Those who led and participated inthe evaluation


• Evaluating an initiative should look at three main areas:• Process• Impact• Outcomes

• Each aspect of the intervention builds on what comes before. In order to have the impact you want, you have to put together and run your program well (process). If your process didn’t go properly, then you haven’t really conducted the program you planned for.

How do you refine an intervention based on

research?



research?• Process• An evaluation of your process compares what you planned

to do with what you actually did • Process encompasses logistics and program implementation

• Community participation• Community assessment• Program planning• Staff hiring and/or volunteer recruitment• Staff and/or volunteer training• Outreach to and recruitment of potential participants• Implementation strategy• Evaluation strategy• Timelines and benchmarks


• Impact• The impact is the effect it had on the environmental

conditions, events, or behaviors that it aimed to change• Generally, a program aims only to influence one or more

particular behaviors or conditions – risk or protective factors

• An evaluation of the risk and protective factors you targeted, will tell you if you made the desired impact


research?



research?• Outcomes• The outcomes of a program are the changes that actually took

place because of it• The goal of an intervention is usually not just a change in behavior

(risk/protective factors), but the changes in community health that occur as a result of that change• For instance, the impact of a tobacco control program is that people stop

smoking. However, the intended outcomes are reduced rates of heart disease, lung cancer, and other smoking-related diseases in the population

• If your intervention is having the desired outcome, you should still strive to make the intervention better and more effective• Ask yourself if there are additional risk and protective factors you could target,

or areas of the program that could be expanded or strengthened?


• Outcomes• If your intervention did not result in the outcomes you intended,

consider what you might need to change• Are there risk and protective factors that would be more relevant?• Are there better ways to communicate your program’s message?• Were there parts of the program that worked less effectively than others?• Were there problems in how the program was delivered? Was it too formal?

Were there unaddressed language barriers?• Was the program culturally and contextually adapted for your situation?

• In evaluating the results of the program, it helps to work backwards to determine where the intervention went off track, starting with outcomes and working back to process to determine what area of the intervention needs to be adjusted for better results


research?

Documents

Copyright © 2014 by The University of Kansas Refining the Program Intervention Based on Research