View
214
Download
0
Tags:
Embed Size (px)
Citation preview
Using Evidence To Improve Public Health Infrastructure:
Let the evidence guide our actions
January 7, 2004
Jonathan E. Fielding, M.D., M.P.H., M.B.ADirector of Public Health and Health Officer
L.A. County Department of Health ServicesChair, CDC Task Force of Community Preventive Services
Professor of Public Health and PediatricsUniversity of California, Los Angeles
If we did not respect the evidence, we would have very little leverage in our quest for truth
Carl Sagan
Quality of the Evidence
We hear about it oftenTV NetworksUsually related to high profile trials
Public Health evidence is different than legal evidenceIt is the available information on a
particular questionWe want the best available evidence
in making decisions
Decisions and Evidence
Evidence takes many formsOpinion of leadersOpinion of “experts” Studies without controlsStudies with controls Studies of variable quality in design
and execution
Decisions and Evidence
Our commitment: Improve public health
Health problems well defined Our job:
Make a difference through policies and programs
Inaction is not an option Hard to identify best evidence to
inform decision making
Decisions and Evidence
Decisions on policies and programs are often made based on:Personal experienceWhat we learned in formal trainingWhat we heard at a conferenceWhat a funding agency required/
suggested What others are doing
Evidence and Public Health Decision Making
Good news Strong evidence on the effect of
many policies/ programs aimed to improve public health
Major efforts underway to assess the body of evidence for wide range of public health interventions
What works to improve the public’s health?
Bad news Many public health professionals are unaware
of this evidence Some who are aware don’t use it Many existing disease control programs have
interventions with insufficient evidence –while others use interventions with strong evidence of effectiveness
Lack of use of effective interventions can adversely affect fulfilling mission and getting public support
How do we know what works in improving the health of populations? Background Many community health improvement efforts
have not achieved desired results Interventions often chosen based on opinions
and personal preferences Evidence based medicine---Clinical
Preventive Services Task Force –mid 80s Evidence based population health ---
Community Preventive Services Task Force– mid 90s
Systematic Search for the Best Evidence
U.S. Community Preventive Services Task Force Appointed by CDC Director in 1996
Non-Federal independent task force of experts in multiple relevant disciplines Epidemiology Public Health Practice Behavioral Sciences Evidence based medicine/ public health Other relevant areas of expertise
Goals
Conduct careful analytic reviews of acceptable evidence for population health interventions and make related recommendations
Use peer reviewed literature Standard rules of evidence Standard rules for translating
evidence into recommendations for interventions
Systematic Reviews of Public Health Interventions are Useful
Methods first developed by social scientists (e.g., Glass, ‘76)
Distill and summarize large and diverse bodies of evidence
Reduce errors and biases in interpretation Make assumptions explicit
Systematic Reviews Are Not:
Limited to randomized controlled trials Limited to healthcare interventions Restricted to a “biomedical model” of
health
- Petticrew, 2001
Task Force on Community Preventive Services
Members
Jonathan E. Fielding, Chair Patricia Dolan Mullen, Vice-chair Noreen M. Clark John M. Clymer Mindy T. Fullilove Alan Hinman
George J. Isham Robert L. Johnson Garland Land Patricia A. Nolan Dennis E. Richling Barbara K. Rimer Steven Teutsch
ConsultantsRobert S. LawrenceJ. Michael McGinnis
Lloyd F. Novick
Who Is the Audience?
People who plan, fund, or implement public health services and policies for communities and healthcare systems Public health departmentsHealthcare systems and providersPurchasers Government agenciesCommunity organizations
Community Guide Topics
Environmental Influences Sociocultural Environment
Physical Environment
Risk Behaviors Specific Conditions Tobacco Use Alcohol Abuse/Misuse Other Substance Abuse Poor Nutrition Inadequate Physical Activity Unhealthy Sexual Behaviors
Vaccine Preventable Disease Pregnancy Outcomes Violence Motor Vehicle Injuries Depression Cancer Diabetes Oral Health
Methods for Systematic Reviews of Effectiveness Evaluations
Develop conceptual framework Search for and retrieve evidence Rate quality of evidence Summarize evidence Translate strength of evidence into finding
Strongly recommended Recommended Insufficient evidence
Logic Framework: Vaccine Preventable Disease
Treatmentof Vaccine-Preventable
Diseases
Morbidity
andMortality
Population
Attendance in
Public, Private, or
Joint Healthcare
Systems
Exposure toVaccine-
Preventable Disease
Intervention Types
Determinants
Intermediate Outcomes
Public Health Outcomes
Reviewed
Not ReviewedReducingExposure
Environment
VaccinationCoverage
Vaccine-
PreventableDisease
Provider-Based
Interventions
IncreasingCommunityDemand for
Vaccinations
Enhancing Access to
Vaccinations
Reduced Disease Instance
Standardized Analysis Process
Systematic review of literature Abstracting of relevant studies Grading of evidence
Study design Execution
Translating from quality of evidence to recommendations
Economic analysis Other benefits and harms
How Does the Task Force Define Suitability of Study Design?
GreatestProspective with concurrent comparison
ModerateMultiple before-and-after measurements
but no concurrent comparison ORRetrospective
LeastSingle group before-and-afterCross-sectional
What Factors Determine Quality of Execution? Description of intervention and study
population Sampling procedures Exposure and outcome measurements Approach to data analysis Interpretation of results
Follow-upConfoundingOther bias
Other issues
How Does the Task Force Draw an Overall Conclusion About the Strength of a Body of Evidence?
Number of studies Design suitability Quality of execution Consistency Effect size
Evidence of Effectiveness
Quality of Execution
Design Suitability
Number of Studies
Consistent Effect Size
1. Strong
Good Greatest > 2 Yes Sufficient
Good Greatest or Moderate
> 5 Yes Sufficient
Good or Fair Greatest > 5 Yes Sufficient
Meet criteria for sufficient evidence Large
2. Sufficient
Good Greatest 1 -- Sufficient
Good or Fair Greatest or Moderate
> 3 Yes Sufficient
Good or Fair Greatest, Moderate or Least
> 5 Yes Sufficient
3. Insufficient Insufficient design or execution Too few No Small
Physical Activity:Review of One Intervention
Goal: increase the amount of time students spend doing moderate or vigorous activity in PE class through curricular change
Interventions reviewed included changing the activities taught (e.g., substituting soccer for softball) or modifying the rules of the game so that students are more active (e.g., in softball, have the entire team run the bases together when the batter makes a base hit). Many interventions also included health education.
14 acceptable studies; in all students’ physical fitness improved.
5 studies measured activity levels during PE class; all found increases in amount or percentage of time moderately/
vigorously active and/or intensity level of physical activity during class.
Median estimates--modifying school PE curricula as recommended will result in an 8% increase in aerobic fitness
School Curricular Interventions
to Improve Physical Fitness
School Curricula to Improve Physical Activity
Modifying school P.E. curricula was effective across diverse racial, ethnic, and socioeconomic groups, among boys and girls, elementary- and high-school students, and in urban and rural settings.
In a separate literature review, having students attend school PE classes was not found to harm academic performance.
Economic analysis pending.
Different Ways to Improve Public Health
Focus on diseases/ injuries e.g. sexually transmitted diseases, intentional injury, diabetes
Focus on risk factors e.g. tobacco use, nutrition, physical activity
Focus on underlying factors that impact multiple dimensions of health e.g. poverty, social isolation
SocialEnvironment
GeneticEnvironment
IndividualResponse
• Behavior• Biology
Healthand
Function
Diseaseand
Injury
Healthand
Medical Care
Well-Being Prosperity
PhysicalEnvironment
Underlying Health Determinants
Impact of Early Childhood Home Visitation Programs
Program can prevent child maltreatment in high-risk families. In studies reviewed, home visiting resulted in a
40% reduction in child maltreatment episodes. Longer duration programs produce larger
effects; programs of less than 2 years duration did not appear to be effective.
Professional home visitors may be more effective than trained paraprofessionals but longer-duration programs with trained
paraprofessionals can also be effective.
Impact of Early Childhood Home Visitation Programs
All programs reviewed were directed at families considered to be at high risk of child maltreatment, (e.g., single or young mothers, low-income households, families with low birth weight infants).
Other benefits Health benefits for premature, low birth weight
infants and for disabled and chronically ill children
Improved maternal educational attainment, reduced public support, improved child educational performance, reduction in drug use and contact with juvenile justice etc.
Tenant-based Rental Voucher Programs
Background Tenant-based vouchers allow very low income
families to rent safe, decent, and affordable privately owned housing in neighborhoods of their choice.
Rental voucher programs, known as “housing mobility programs,” work with landlords and tenants to find rental property outside of neighborhoods of concentrated poverty and relocate families to neighborhoods of greater prosperity.
Tenant-based Rental Voucher Programs
Findings from the Systematic Review 6 studies: rental voucher programs resulted in
decreases in victimization of tenants or their property Families enrolled in rental voucher programs who
moved to better areas were• 6% less likely to have a household member victimized • 15% less likely to experience neighborhood social
disorder. Changes in victimization in both urban and suburban
settings. Other benefits: substantially reduced symptoms of
maternal depression, boys’ behavioral problems in school, and childhood illnesses and accidents requiring medical attention.
General Comments on Evidence Based Reviews
More evidence than sometimes expected, however
Insufficient evidence common outcome Very resource intensive process Quality of studies vary widely Economic data still uncommon Important to consider harms even though
uncommon
TASK FORCE REVIEWS AND RECOMMENDATIONS
www.thecommunityguide.orgResults of all reviews to dateFrequent updatingDownloadable Slide Sets
How Evidence Can Improve Public Health Infrastructure
Explore evidence underlying options to reach each public health goal
e.g. smoking control, reducing disparities in infant mortality, increasing physical activity, increasing immunization rates
Comprehensively review the best sources of evidence reviews
Community Guide (best source when topic of interest has been covered)
Recent review articles in peer reviewed journals Other meta-analyses funded by responsible federal
agencies Compare results and recommendations of different
sources
Use evidence to decide among possible interventionsHow does each possible intervention
suit the problem and the population? • E.g. was it tried on particular racial/
ethnic/ age/ gender groups?• Is there reason to belief it would not be as
effective for some of these on whom it was not tried?
• Is the problem now similar to what it was when the major studies took place?
How Evidence Can Improve Public Health Infrastructure
Deciding on Interventions
Single versus multiple component interventionsSingle component interventions easier
to develop, implement, control and assess, but
Multi-component interventions usually more effective
• E.g. Tobacco control in California
Deciding on Interventions
Consider both policies and programs Programs
Greater control over all aspects Organizational unit has primary
responsibility for design, implementation and outcomes
Policies Control varies: broad policies often made by
elected officials Policies have potential for greater public
health impact Credit needs to be shared e.g. LAUSD
Nutrition Policies, increase in tobacco tax
Deciding on Interventions
Consider effect sizeMedianConsistency
Consider breadth of target population Together effect size and target
population define the overall population effect
How Evidence Can Improve Public Health Infrastructure Use evidence to determine realistic goals by estimate effect
size (i.e.. how much you move the needle!)
Relative percentage changes in exposure to environmental tobacco smoke attributable to workplace smoking bans and restrictions from studies that qualified for inclusion in this review (“a” and “b” in Study names refer to first or second study by the same author in that year, included in this review)
Deciding on Interventions
What is the slope of the effect curve? • Larger initial effects with significant recidivism• Smaller initial effects with Increasing impact over
time
What is the time frame for observed health benefits?
• How long were the follow-up periods for the best studies?
• For equal benefit, shorter is better, but• Long term benefit is primary interest
Deciding on Interventions
What is the cost of the intervention? Personnel Dollars i.e. contracts Time to implement Likelihood of funding for sufficient period to
get effect Potential for dedicated or incremental
funding Opportunity cost (i.e. cost of not doing other
things)
Deciding on Interventions
Relative cost-effectiveness Cost effectiveness is dollar cost per health outcome
(including clear intermediate outcomes) e.g. smoker prevented
lead poisoning preventedSTD cureddrug treatment completed
Note: some interventions have multiple health benefits e.g. smoking affects CVD, some cancers, respiratory disease etc.
Deciding on Interventions
Who else needs to be involved to be successful? Within public health Within personal health services Voluntary agencies Health care organizations Health plans Employers
How difficult is it to get agreement on: Roles and responsibilities Interventions?
Time cost versus partnership benefit
How Evidence Can Improve Public Health Infrastructure
Use evidence to help decide on construction of intervention
Interventions with same name can be very different
Follow the design used in most successful interventions
Talking to those who did the studies is very helpful in refining intervention
Using the Evidence
Use evidence to frame objectives Use evidence to develop evaluation plan
and related evaluation Approach Measures Data collection plan
Develop internal evidence through performance measurement system
Frequent monitoring essential
What more is needed?
More research on public health practice; for many interventions---insufficient evidence
Increased funding for evidence based reviews using consistent methodologies
More training on appropriate sources and uses of evidence
in schools of public health and others training public health professionals
in public health practice settings e.g. state and local health departments
Political leaders and others who influence the decision making process to improve health
DHS Public Health Opportunities
Become sophisticated user of evidence based information and recommendations
Make use of best evidence key aspect of performance of program directors and key managers
Contribute to the literature on what works in public health practice
Presentation available on DHS Intranet
Also see Evidence-Based Public Health, Ed. Ross C. Brownson, Elizabeth Baker, Terry L. Leet etc.
Oxford University Press, 2003