Upload
core-group
View
46
Download
0
Tags:
Embed Size (px)
Citation preview
Implementation Research: What is it? Am I doing it?
How can I do it better?
Olakunle Alonge, Assistant Scientist, Health Systems Program, JHBSPH Jim Ricca, MCSP Learning and Implementation Science Team Leader Stefan Baral, JHBSPH
Learning Objectives
By the end of this session, participants will be able to: • Explain what Implementation Research (IR) is and how it
related to other forms of research • Explain three commonly studied Implementation Research
outcomes • Design answerable Implementation Research questions of
importance to key stakeholders
2
Implementation Practice and Research
• “Implementation [practice] is the act of carrying an intention into effect, which in health can be policies, programmes, or individual practices (collectively called interventions)” e.g. the act of administering misoprostol for AMTSL
• “Implementation research is the scientific inquiry into questions concerning implementation” e.g. study on the acceptability of misoprostol among pregnant women in an area
3 Source: Peters, Adam, Alonge, Agyepong and Nhan, BMJ 2013; Gilbert, JRScT 1991
Implementation Research
Translational Research
Implementation Research and Other Common Studies in Public Health
4
T4 studies
T3 (D&I)
studies
Intervention studies
T2 studies
T1 studies
QI studies
Operation Research
M&E
Source: Lobb and Colditz, Annual Rev Pub Health 2013; IRDS Consultative Meeting Report, 2014
Implementation Outcomes
• Implementation variables: – Constructs whose vector properties (magnitude and direction)
changes with an implementation process
• Implementation outcomes: – Implementation variables that describe effects (results) of a deliberate
and purposive action to implement an intervention – Useful for measuring success or failure of an implementation process – Intermediate outcomes of the effectiveness of an intervention (on
individual/population health outcomes and quality of life) – Underdeveloped constructs; operates across socio-ecological levels
6 Source: Peters, Adams, Alonge et al 2013; Proctol et al. 2011; Fixsen et al. 2005
Conceptual Framework for Implementation Outcomes
Clients Outcome Satisfaction
Symptomatology Function
Population-
Based Incidence of
diseases Morbidity Mortality
DALYs
Health Outcomes
Efficiency Coverage
Equity Responsiveness
Services Outcomes**
Acceptability Adoption
Appropriateness Costs
Feasibility Fidelity
Penetration Sustainability
Implementation Outcomes I*
7
• *measurable both at the individual and aggregated level • **measurable mostly at an aggregated level
Source: Proctor et al 2011
Implementation Outcomes - Definitions
Implementation Outcome
Working Definition* Related terms**
Acceptability The perception among stakeholders (e.g. consumers, providers, managers, policy-makers) that an intervention is agreeable
Related factors: (e.g. Comfort, Relative advantage, Credibility)
Adoption The intention, initial decision, or action to try to employ a new intervention
Uptake, Utilization, Intention to try,
Appropriateness The perceived fit or relevance of the intervention in a particular setting or for a particular target audience (e.g. provider or consumer) or issue
Relevance, Perceived fit, Compatibility, Perceived usefulness or suitability
Feasibility The extent to which an intervention can be carried out in a particular setting or organization
Practicality, Actual fit, Utility, Trialability
8 Source: Proctor et al 2011; Peters, Adams, Alonge et al 2013
Implementation Outcomes - Definitions Implementation Outcome
Working Definition* Related terms**
Fidelity The degree to which an intervention was implemented as it was designed in an original protocol, plan, or policy
Adherence, Delivery as intended, Integrity, Quality of delivery, Intensity or dosage of delivery
Implementation cost
The incremental cost of the implementation strategy (i.e. how the services are delivered in a particular setting); including the cost of the intervention itself.
Marginal cost, Total cost***
Penetration/ Coverage
Extent to which a practice or program is integrated within a service settings and system (niche saturation). The degree to which the population that is eligible to benefit from an intervention actually receives it.
Reach, Access, Service Spread or Effective Coverage, Penetration
Sustainability The extent to which an intervention is maintained or institutionalized in a given setting.
Maintenance, Continuation, Routinization, Institutionalization,
9 Source: Proctor et al 2011; Peters, Adams, Alonge et al 2013
Fidelity - Definition
• Degree to which an intervention was implemented as prescribed by the original protocol or program developers
• Adherence to the core program components – Core components: elements without which the intervention will not
have its intended effect – Balancing fidelity and adaptation: 100% fidelity may be unrealistic
13 Source: Dusenbury et al 2003, Rabin et al. 2008
Fidelity – Domains of Interest
• May consist of the following domains:
14 Source: Dane & Schneider, 1998; Dusenbury et al 2003; Carroll et al. 2007; Durlak and Dupre 2008; Proctor et al 2011
Dane & Schneider, 1998
Carroll et al. 2007 Proctor et al. 2011 Durlak and DuPre 2008
Adherence Adherence Adherence Adherence
Dose Exposure or dose Dose
Quality of delivery Quality of delivery Quality of delivery
Participant responsiveness
Participant responsiveness
Program differentiation
Program differentiation
Intervention complexity
Facilitation strategies
Fidelity – Domains of Interest
• Domains of interest: – Adherence – an intervention is being delivered as it was designed or written – Dosage or exposure - frequency and duration of the intervention is as full as
prescribed by its designers. Could mean amount of intervention that the recipient received
– Quality of delivery - the manner in which an intervention worker (e.g. teacher, volunteer, or staff member) delivers a program
• Based on techniques prescribed by developer • Based on a theoretical ideal
– Participant responsiveness - how far participants respond to, or are engaged by, an intervention
– Program differentiation - identifying unique features of different components of an intervention, and identifying the core components
• Perspectives in measurement: – Representative (commonest approach) – Comprehensive – Complex
15 Source: Mihalic 2004, Carroll 2007
Acceptability
• Perception among stakeholders (e.g. consumers, providers, managers, policy-makers) that an intervention is agreeable
– Based on direct knowledge of or experience with specific aspects of the intervention, e.g. content, comprehensiveness, ease of use, comfort, etc.
– Distinguished from service satisfaction which often focuses on general service experience
– Dynamic and changes with experience
• Often conceptualized as consisting a single domain
17
Feasibility
• The extent to which an intervention can be carried out in a particular setting or organization
– Typically assessed retrospectively to explain success or failure of an implementation process (actual fit)
– Closely related to appropriateness which looks at the perceived fit of an intervention for a setting (i.e. prospectively assessed)
• Has multiple domains and similar measurement perspectives as in fidelity (i.e. representative, comprehensive, complex)
19
Summary – What is IR? / IR Outcomes
• Implementation research is the scientific inquiry into questions concerning implementation
• Such research could be conducted by observing changes in implementation research outcomes (IRO) during the implementation process
• IRO describe effects (results) of a deliberate and purposive action to implement an intervention
• IRO are mediators of individual/population health outcomes • IRO could comprise multiple domains or a single domain, are
defined and measured in context across single or multiple socioecological levels
• They could be measured using quantitative, qualitative and mixed method approach
• Some common examples of IRO are fidelity, acceptability and feasibility 22
24
Addressing Public Health Problems
Define the problem Measure the magnitude of the problem
Develop a conceptual framework for the understanding of the key
determinants of the problem
Identify and develop strategies for intervention (program)
Set priorities and recommend interventions (policy)
Implement interventions and evaluate outcomes
Communication strategy
25
Problem Definition Creating a problem definition is an iterative
process • Non-linear process • Changes in the definition are expected • Requires informed discussion • Tests team skills
– Key difference with IR is the involvement of end users in defining the public health problem
Defining a problem is an iterative process and is often the
most difficult step in the problem solving methodology, but a good problem definition goes a long way towards successfully solving the problem
http://www.notablebiographies.com/De-Du/Dewey-John.html
Prioritization of Health Issues for IR
• Priority Rating Formula (Hanlon) – Component A = Size of the problem – Component B = Seriousness of the problem – Component C = Estimated effectiveness of the solution
– Basic Priority Rating
• BPR = (A + 2B) * C
– +/- Component D = PEARL factors (propriety, economic, acceptability, resource availability, legality)
– Overall Priority Rating
• OPR = [(A + 2B) * C]D
27
Specificity of the Problem
• What is the specific dimension or aspect of interest?
• If you think you are being too specific, you probably are not
• Avoid broad, general topics as they run the risk of containing competing concepts
– Eg. How do improve health systems in LMIC? – How to improve pediatric polio vaccine delivery among
rural populations born in 2015 in Northern Nigeria?
Key Aspects of a Problem Definition
• Nature of the problem/focus
• Framing of the problem – Person – Place – Time
• Specificity of the problem
• Causality
• Language considerations
28
29
Framing of the Problem Person • Who is affected by this problem?
• Be as specific as possible…
– …using basic demographics: • Gender • Age • Race/ethnicity • Socioeconomic status (education, income, etc)
– …and using problem-specific demographics • Eg. HIV
– Sexual practices – Drug use – Housing situation – Occupation
Estimates of new HIV Infections in the US, by Transmission Category, 2006
Hall HI, Song R, Rhodes P, et al. Estimation of HIV Incidence in the US. JAMA 2008;300: 520–529.
Place
• Where is the problem located?
• Be as specific as possible: – Geographical – Cultural – Social – Political – Economic – Historical – Ethnic
30
Framing of the Problem
Cholera cases (dots) and outbreaks (orange), 2007-2009
http://gamapserver.who.int/mapLibrary/Files/Maps/Global_CholeraCases0709_20091008.png
Time
• What is the specific time frame for the problem?
• Be as specific as possible – Dates (1990-2000) – Time since a specific event (post 9/11)
31
Framing of the Problem
http://www.cdc.gov/Features/dsSpinaBifida/
Summary Problem Statement
• Summarizes the purpose of the study • Establishes the direction and captures the essence
of the study • Reflects the research design of the study • Leads logically to the implementation research
objectives/questions/specific aims
32
IR Questions
• Of interest to stakeholders – Government, community, academia, implementing partners
• Answerable • Shaped by the problem and in turn shape the design of the
research • Clear, specific, and feasible
• Three Broad Areas of IR Questions
– Describe Health Situation or Interventions – Provide Data to Evaluate Ongoing Interventions or Information
Needed to Adjust Interventions – Analyze missed targets and potential solutions
33
34
Language Considerations
• Be clear and concise • Define and reference technical terms
• Define local terms
• Generally a paragraph rather than a single statement • Incorporate general objectives and use action words to
succinctly outline the purpose of the study – Purpose, Intent, Objective
35
Causal Claims
• Are etiologic factors or causes part of your problem statement?
• AVOID causal claims
• Be short and clear – this is the problem definition, not the answer to the problem