Upload
amusten
View
200
Download
0
Tags:
Embed Size (px)
Citation preview
Overview
Definitions Characteristics Conceptual Frameworks
Evidence and Outcomes Methods
Qualitative Quantitative Descriptive Analytic Experimental
Conclusions
2
Definitions
Implementation The use of strategies to introduce or change evidence-based health
interventions (policies, programs, individual practices) within specific settings
Implementation Science in HIV Implementation science is a multi-disciplinary field that seeks
generalizable knowledge about the behaviour of stakeholders, organizations, communities, and individuals in order to understand the magnitude, reasons for and strategies to close the gap between evidence and routine practice for health in real world contexts
Key Themes Multidisciplinary Generalizable Multiple stakeholders Closing gap between evidence and practice Real world contexts
3Lobb and Coldtiz, Implementation Science and Its Application to Population Health Annual Review of Public Health, 2013; Odeny, Padian, Doherty, Baral, Beyrer, Ford, Geng, Definitions of implementation science used in the HIV/AIDS literature: a synthetic review. The Lancet Infectious Diseases, In Press, 2015
Implementation Research and Other PH Study Designs
4Source: Olakunle Alonge, Lobb and Coldtiz, Implementation Science and Its Application to Population Health Annual Review of Public Health, 2013; https://catalyst.harvard.edu/pathfinder/t2detail.html
Characteristics of Implementation Research
5
Findings are Warranted to Inform Policy/Program There is “sufficient evidence” to support the conclusions
of the work What is sufficient evidence?
Transparency of Methods Support Critical Assessment of the Study
Whether processes are adequate Conclusions justified Repeatability
Don’t be afraid of “failure” A well done study is still a success in terms of
generating generalizable knowledge
Traditional Scientific Method
6http://www.sciencebuddies.org/science-fair-projects/project_scientific_method.shtml
Differences with IR
Competencies on a IR team: Research Methodologist
Qual, Quant, Mixed Methods Ministry, Government, Agencies
Either as members of team or study oversight committee
Health Professionals Involvement of health professionals from study settings
Communications Public Health Professionals
Health Commissioner/Associate Health Commissioner Public Health Inspector/Public Health Nurse
Privacy Expert Stakeholder Assessment
Community
7http://www.who.int/tdr/publications/year/2014/ir-toolkit-manual/en/
Traditional IR Approaches
8http://www.sciencebuddies.org/science-fair-projects/project_scientific_method.shtml
Active Consultation
Active Consultation
Appropriate Team, Active Consultation
Active Consultation
Active Consultation
Active Consultation
IR Specific Objectives: Three Broad Areas
Three Broad Areas of IR Specific Objectives
1. Describe Health Situation or Interventions
2. Provide Data to Evaluate Ongoing Interventions or Information Needed to Adjust Interventions
3. Analyze missed targets and potential solutions
9
Describe Health Situation and Intervention
Magnitude of the problem Distribution of health needs of the population Risk factors for some problems People’s awareness of the problem Utilization patterns of services Cost-effectiveness of available and potential other
interventions
10
Evaluate Interventions
Coverage of priority health needs Coverage of target groups Acceptability of the services Quality of services Cost-effectiveness of the intervention Impact of the program on health
11
Analyze Missed Targets
Availability Acceptability Affordability Service delivery problems
Fidelity
12
Conceptual Frameworks Commonly used in IR
RE-AIM Reach, Efficacy/Effectiveness, Adoption,
Implementation, Maintenance
Stages of implementation National Implementation Research Network Exploration and Adoption, Program Installation (Prep),
Initial Implementation (pilot/adapt), Full Implementation (>50% coverage), Sustainability
Consolidated Framework for Implementation Research Intervention Characteristics, Inner Setting, Outer
setting, Individuals in the Intervention, Implementation process
Many others….14
Source: Glasgow et al 1999, National Implementation Research Network, 2005, Damschroder, 2009
Outcomes in Implementation Research
15
Clients Outcome
Satisfaction
Symptomatology
Function
Population-Based
Incidence of diseases
Morbidity
Mortality
DALYs
Health Outcomes
Efficiency
Coverage
Equity
Responsiveness
Services Outcomes
Acceptability
Adoption
Appropriateness
Costs
Feasibility
Fidelity
Penetration
Sustainability
Implementation Outcomes
Source: Olakunle Alonge, Proctor et al 2011
Implementation Outcomes
16
Implementation Outcome
Working Definition* Related terms**
Acceptability Perception among stakeholders that an intervention is agreeable
Related factors: (e.g. Comfort, Relative advantage, Credibility)
Adoption Intention, initial decision, or action to try to employ a new intervention
Uptake, Utilization, Intention to try,
Appropriateness Perceived fit or relevance of the intervention in a particular setting or for a particular target audience (e.g. provider or consumer) or issue
Relevance, Perceived fit, Compatibility, Perceived usefulness or suitability
Feasibility The extent to which an intervention can be carried out in a particular setting or organization
Practicality, Actual fit, Utility, Trialability
Fidelity The degree to which an intervention was implemented as it was designed in an original protocol, plan, or policy
Adherence, Delivery as intended, Integrity, Quality of programme delivery, Intensity or dosage of delivery
Implementation cost
Incremental cost of the implementation strategy Marginal cost, Total cost***
Coverage Degree to which the population that is eligible to benefit from an intervention actually receives it.
Reach, Access, Service Spread or Effective Coverage, Penetration
Sustainability The extent to which an intervention is maintained or institutionalized in a given setting.
Maintenance, Continuation, Routinization Institutionalization, Incorporation
Source: Proctor et al 2011; Peters, Adams, Alonge et al 2013
What is Sufficient Evidence?
Evidence-based medicine is a global standard Double-Blinded (DB) RCT is gold standard
Evidence-based PH interventions should also be a global standard Often limited evidence, PH decision still needs to be
made Precautionary Principle for PH?
When there are threats of serious or irreversible damage, lack of full scientific certainty shall not be used as a reason for postponing cost-effective measures to prevent environmental degradation
To develop guidelines Need to characterize
Efficacious Effective Sustainable and Scalable programs
17
Tension Between Internal and External Validity Challenges for Evidence Combination Prevention
Internal Validity Minimal study biases suggesting confidence in
ultimate conclusion of the study External Validity
Generalizability of ultimate findings to broader population
Traditional Question for Clinicians/Programmers Does it work? What is effect size?
Should I use it? Implementation Questions
How, when, why, and where does it work? What factors influence effectiveness?
Should I use it? How should I use it?
Traditional Research Pathway
Effectiveness Research (and guideline development) generally happens prior to implementation research Are there more time-effective approaches to integrate
implementation research with effectiveness/efficacy research
Assess barriers/facilitators to intervention uptake acceptance/adoption/routinization
Diagnose quality gaps Fidelity
Characterize Sustainability Maintenance, Cost-Effectiveness
Research Objectives Qualitative Methods
Exploration Using inductive methods to explore concepts, constructs,
phenomena, situations to develop hypotheses Quantitative Methods
Explanation Characterizing the
Relationship between concepts and phenomena Reasons for occurrences of events
Prediction Use knowledge to forecast events
Experiment Intervene/manipulate different settings or variables to produce
a desired outcome Mixed Methods
Description Identify and describing the precursors/antecedents, nature, or
etiology of an outcome/phenomena
Qualitative Methods in Implementation Research
To explore a health problem Evidence-based approach to characterizing individual, network, and
structural determinants of the health issue To identify variables that can later be measured
Measuring the “right” implementation outcomes To develop a complex and detailed understanding of an issue
Formative work to design the implementation plan for an effective intervention
Maximizing Acceptability, Appropriateness To understand the contexts or settings surrounding an
experience or phenomenon Case-Study of a program or a policy
To understand the meaning behind an issue or experience Explore the “why” or “how” an intervention works or why not
Fidelity
Quantitative Research Designs
Observational Categories
Descriptive Analytic
Cross-Sectional Prospective Quasi-Experimental Studies
no randomization of individuals/communities/institutions
Experimental
Rapid Innovation in Implementation Design
Cohorts In Implementation Research Method
Traditional “Active” Cohort includes Enrollment of participants and active follow-up of individuals Expensive to ensure retention
“Passive” Cohorts (registries, EHR Repositories, etc) Rapidly becoming the standard in IR, but limitations include
quality and extent of data collection
What is Different from Traditional Cohort Studies? Traditional Cohorts
Incidence, Prevalence, Relationship between Exposure and Outcome
Cohorts in IR focus on measuring traditional IR Outcomes Acceptability, adoption, appropriateness, feasibility, fidelity
of interventions, implementation costs (cost-effectiveness), Determinants of Coverage, Sustainability/Maintenance
Implementation Outcomes
Outcome of Interest Definition Data Source
Proportion of HIV-positive men who enter care
At least one clinic visit attended (where they see a clinician) after positive test result within 3, 6 and 12-months of study visit
Clinic records, NHLS documentation of lab tests completed
Proportion of men who receive CD4 results
Receive POC CD4 results at study visit or get CD4 tests at clinic and return to receive results
Study visit CRFs, Clinic records, SMS surveys
Proportion of ART-eligible men who initiate ART
ART initiated by 12 months Study visit CRFs, Clinic records
Time to ART initiation for ART-eligible men
Length of time between receiving positive HIV test result and CD4 <350 to initiating ART
Study visit CRFs, Clinic records
Proportion of ART-eligible men who initiate ART and are retained in care
Attend 6-month clinic visit (see a clinician)
Attend 12-month clinic visit (see a clinician)
Attend 2+ clinic visits at least 3 months after within a 12-month period
Clinic records, NHLS documentation lab tests completed (CD4, viral load, others)
Proportion of treatment ineligible HIV-infected men who receive a CD4 test within 6 months following their study visit CD4
Receipt of repeat CD4 test Clinic records, NHLS documentation lab tests completed (CD4, viral load, others)
Implementation Outcomes
Outcome of Interest Data SourceAcceptability of outreach/CBO-based testing intervention
Questions on study visit CRFs, surveys about why participants chose to test
Relative advantages of non-clinic-based ART initiation and retention package compared to standard of care
CRFs, questions in surveys, qualitative data from both participants and providers
Perceived credibility of CBOs to initiate ART as compared to standard ART clinics
Survey indicators, qualitative data
Adoption of experimental interventions including intention of use of decentralized NIMART-trained nurse initiated ART and peer navigator based support by participants
CRFs, survey indicators, qualitative data
Implementation costs associated with experimental condition
Review of clinic-based budgets and ultimate costs to assess marginal and total costs of interventions as compared to standard of care
Maintenance and routinization of using clinic-based approaches and peer-navigators for retention as indicators to describe potential sustainability of the interventions
Provider and participant qualitative data, survey indicators
Experimental Studies
Explanatory (Traditional Gold Standard) Understand and explain benefit of an intervention
under controlled conditions Maximize internal validity
Pragmatic Trials Focus on the intervention in routine practice Intentional maximization of variability in how study is
implemented Variability of research settings (communities,
practice settings, types of providers, patients) Maximize external validity
Adaptive Designs Emerging area of implementation research that
attempts to balance internal and external validity
Pragmatic Trials
Testing a new intervention while maximizing external validity Formative Period
Qualitative work, some descriptive or analytic observational work Consider different types of outcomes of effectiveness
Directly Measured Health Outcomes Service Outcomes Implementation Outcomes Resources
Institutional Human Financial
Cost-effectiveness, Cost-Utility, Cost-Minimization, Cost-Benefit, etc
Indirect Assessment/Modeled Benefits Increasing use of Mathematical Models to Scale Results for
potential longer term outcomes, etc.
Effectiveness-Implementation Hybrid Trials
Goal Measure markers of implementation and impact in the same
study Three Broad Designs
Differentiated by prioritization of data collection Type 1
1st priority - Impact of health intervention 2nd Priority - gathering measures of implementation
Feasibility/Acceptability using qualitative/mixed methods
Type 2 Equal priority to impact and implementation
Type 3 1st priority – Implementation
Fidelity of intervention, measures of adoption 2nd Priority Impact of Health Intervention
Stepped Wedge Cluster Randomized Designs
Method Assess baseline situation in all communities, but randomly phase
in intervention activities in steps, evaluating impact of intervention time on outcomes
http://www.biomedcentral.com/1471-2288/6/54
Stepped Wedge Designs
Pros Differences in exposure time allow each community site to
receive the intervention Ethics
Mixed views on the ethics of stepped wedge Some believe more ethical to give intervention to all and
more feasible to implement, others believe trial not warranted if success of intervention is certain and standard of care can be justified so why not assess more cleanly with parallel design
Cons Analysis concerns around when an intervention starts – e.g. if a
community starts receiving the intervention today but takes 2 months to scale up and reach a substantial number of people, exposure time will be diluted as everyone starts receiving the exposure at the same time within the cluster
If the outcome cannot be expected to happen over the time period of one step, stepped wedge designs will be underpowered
Adaptive Designs For Implementation Studies
Adaptive Intervention/Adaptive Implementation Strategy Specific decision rules for the implementation of an
interventions based on individual/community needs Trying to maximize both internal and external validity
Trial Design Sequential, multiple assignment, randomized trials
(SMART) Use outcome data to inform the implementation of
the intervention being evaluated Can be at multiple steps
http://methodology.psu.edu/ra/adap-inter
SMART Study ExampleResearch Question: Among clinics not responding to Replicating effective interventions/REP, how much does external or internal facilitation help improve mood disorders program
Source: Kilbourne, Almirall, Implementation Science, 2014
Adaptive Implementation Strategy A priori decisions about intervention based on response Randomize to control for confounders but being done in real world
setting Improved balance of Internal/External Validity
Measure Implementation Outcomes Throughout
Power and Sample Size Calculations
P&S Calculations for IS studies are complicated Powered to assess the “preponderance of evidence”
of the benefit of interventions Most realistic, but murky
Powered for at least primary outcome (Eg. Viral suppression)
Cleaner, but is this really implementation research?
Powered on Outcome and Implementation Outcomes Limited resources, etc.
De-Implementation Science Studies
The science of dissemination and implementation confronts two problems Getting wider uptake of evidence-based interventions
in clinical or public health practice Elimination from clinical or public health practice of
tests and interventions that use resources without enhancing patient outcomes
As a field, we focus more on increasing interventions than we do reducing unnecessary ones More incentive to discover new tools than to try and
more politically sensitive to try and remove services for folks
De-Implementation Science Methods Use many of the same experimental methods (CRCT,
SW, etc) but in reverse