Global Health Decisions:Global Health Decisions: a web tool for HIV decision-making a web tool for HIV decision-making

Mohsen Malekinejad MD DrPH Mohsen Malekinejad MD DrPH 11 Elliot Marseille DrPH, MPP Elliot Marseille DrPH, MPP 2 2

James G. Kahn MD, MPHJames G. Kahn MD, MPH11 (PI) (PI)

1 - Philip R Lee Institute for Health Policy Studies1 - Philip R Lee Institute for Health Policy StudiesUniversity of California San Francisco, San Francisco, CA University of California San Francisco, San Francisco, CA

2 - Health Strategies International, Oakland, CA2 - Health Strategies International, Oakland, CA

OverviewOverview

First section (~20 min)First section (~20 min)• Background Background • MethodsMethods• GHD website conceptual model GHD website conceptual model

Q&A (10 min)Q&A (10 min) Second section (~15 min)Second section (~15 min)

• GHD live website tourGHD live website tour• Case study: GhanaCase study: Ghana• Next stepsNext steps

22

Background: Policy-making in HIV Background: Policy-making in HIV global settings – Challengesglobal settings – Challenges

EfficacyEfficacy (risk reduction in ideal setting) essential (risk reduction in ideal setting) essential but insufficient alonebut insufficient alone

Evidence on intervention Evidence on intervention effectivenesseffectiveness often often scarcescarce - reliance on research from - reliance on research from less-relevant less-relevant settingssettings

Complexity and inconsistency Complexity and inconsistency of data reported in of data reported in scientific journals is dauntingscientific journals is daunting

Policy-makers often Policy-makers often lack advanced lack advanced epidemiological or cost-effectiveness tepidemiological or cost-effectiveness trainingraining

33

GHD – GoalGHD – Goal

To Provide health To Provide health policymakerspolicymakers around around the globe with an the globe with an evidence-based evidence-based and and easy-to-use easy-to-use web application that web application that translates translates scientific datascientific data into the likely into the likely costs and health benefits costs and health benefits of prevention of prevention and treatment portfoliosand treatment portfolios. .

44

GHD: Specific AimsGHD: Specific Aims1.1. Assess evidence of Assess evidence of effectivenesseffectiveness (vs. Efficacy) of (vs. Efficacy) of

HIV prevention and treatment: How well does the HIV prevention and treatment: How well does the intervention evidence base reflect real-world intervention evidence base reflect real-world practice?practice?

2.2. Translate the effect of specified interventions Translate the effect of specified interventions implemented at implemented at large scale large scale into reduced into reduced population-level population-level disease burden measured in disease burden measured in DALYsDALYs

3.3. Calculate the Calculate the cost-effectiveness cost-effectiveness of sets of of sets of interventions, incorporating effectiveness, burden interventions, incorporating effectiveness, burden reduction, and cost reduction, and cost

4.4. Make findings easily accessible to Make findings easily accessible to decision-decision-makersmakers and other technical and non-technical end and other technical and non-technical end users thorough a users thorough a web-based resourceweb-based resource

55

Methods – OverviewMethods – Overview Studies search & screening: Studies search & screening: systematic review of systematic review of

selected HIV prevention and treatment interventions selected HIV prevention and treatment interventions Data Extraction: Data Extraction: relevant HIV outcome and cost datarelevant HIV outcome and cost data Data analysis: Data analysis: translate outcome data into a translate outcome data into a

standard metric of RRR and conduct meta-analysis standard metric of RRR and conduct meta-analysis Assess internal validityAssess internal validity: rate scientific quality of : rate scientific quality of

evidence evidence Assess external validity:Assess external validity: rate relevance of evidence rate relevance of evidence

to a specific target settingto a specific target setting Epi modelEpi model: develop parsimonious model to evaluate : develop parsimonious model to evaluate

intervention effects intervention effects Cost-effectiveness model: Cost-effectiveness model: calculate incremental cost-calculate incremental cost-

effectiveness ratio effectiveness ratio

66

Methods – Search & ScreeningMethods – Search & Screening Search: Search: conduct searches of conduct searches of systematic reviews systematic reviews in major in major

biomedical and public health databasesbiomedical and public health databases

• Interventions: Interventions: HIV testing and counseling, needle and syringe HIV testing and counseling, needle and syringe exchange program, opioid substitution treatment, sex worker exchange program, opioid substitution treatment, sex worker programs, prevention of mother to child transition, antiretroviral programs, prevention of mother to child transition, antiretroviral treatment. treatment.

Screening: Screening: dual independent screening based on pre-defined dual independent screening based on pre-defined inclusion and exclusion criteria in three levels:inclusion and exclusion criteria in three levels:

1.1. Systematic reviews: stepwise Systematic reviews: stepwise (Title & Abstract, Full text) (Title & Abstract, Full text)

2.2. hand search references of SRs to extract individual studieshand search references of SRs to extract individual studies

3.3. Individual studies: stepwise Individual studies: stepwise (Title, Abstract, Full text) (Title, Abstract, Full text)

Expert consultation. Expert consultation. contact topic experts for studies or contact topic experts for studies or documents potentially documents potentially missed missed by search process. by search process.

77

Methods – IOPT ExtractionMethods – IOPT Extraction extract all relevant data points at the level of extract all relevant data points at the level of Intervention, Intervention, Outcome, Population Trio (IOPT)Outcome, Population Trio (IOPT) different than conventional systematic reviews with different than conventional systematic reviews with narrow narrow scopescope Why IOPT? Why IOPT? Studies report Studies report >1 useful data point >1 useful data point share share some overall aspects of the study (e.g., settings, some overall aspects of the study (e.g., settings, investigators, etc.), but vary in respect to other aspects:investigators, etc.), but vary in respect to other aspects:

1.1. InterventionIntervention: subjects may receive different : subjects may receive different intervention due to variations in dose, frequency, intervention due to variations in dose, frequency, and content of interventions. and content of interventions.

2.2. Outcomes: Outcomes: different time-interval, or type (e.g., different time-interval, or type (e.g., morbidity, mortality, behavioral)morbidity, mortality, behavioral)

3.3. Population: Population: sub-analysis of findings by severity of sub-analysis of findings by severity of underlying diseases (e.g., HIV serostatus)underlying diseases (e.g., HIV serostatus)

88

Methods – Data AnalysisMethods – Data Analysis Studies report data in Studies report data in various typevarious type (RR, OR, pre-post (RR, OR, pre-post

prevalence) prevalence)

transformed all data type to standardized outcome metric transformed all data type to standardized outcome metric of “of “relative risk reduction” - RRRrelative risk reduction” - RRR

RRR: the proportionate reduction in risk of RRR: the proportionate reduction in risk of negative health negative health outcomes outcomes associated with the intervention – e.g.: associated with the intervention – e.g.:

RRR = control event rate (CER) – experimental event rate (EER) control event rate (CER)

RRR = 1- RRRRR = 1- RR

RRR facilitates RRR facilitates comparison across studies and intervention comparison across studies and intervention types types with varying levels of baseline riskwith varying levels of baseline risk

Meta-analysis: Meta-analysis: random effect model using inverse of random effect model using inverse of variance to calculate summary effect measure and 95% CIvariance to calculate summary effect measure and 95% CI

99

GHD Internal Validity (GIV) GHD Internal Validity (GIV) Scoring SystemScoring System

Objective: To assess and translates Objective: To assess and translates methodological aspects underlying data points into methodological aspects underlying data points into a a simple, transparent, and intuitively simple, transparent, and intuitively meaningful meaningful ordinal score (1 – 6)ordinal score (1 – 6)

Modified version and hybrid product of two Modified version and hybrid product of two existing tools existing tools EPHPP and Cochrane GRADEEPHPP and Cochrane GRADE

GIV score generated at IOPT level in three steps:GIV score generated at IOPT level in three steps:

1.1. Assign an Assign an initiainitial score based on l score based on study design (1–5)study design (1–5)

2.2. Rate the potential risks of bias Rate the potential risks of bias to adjust initial scoreto adjust initial score

3.3. Assign an overall GIV score (0 – 6)Assign an overall GIV score (0 – 6)

1010

GIV - DomainsGIV - Domains Study design (used to assign the initial score)Study design (used to assign the initial score)

Comparability of study armsComparability of study arms

Performance of intervention providerPerformance of intervention provider

Performance of outcome assessorPerformance of outcome assessor

Performance of participantsPerformance of participants

Accuracy of measurement toolsAccuracy of measurement tools

Withdrawal and differential drop-outWithdrawal and differential drop-out

Intervention contaminationIntervention contamination

Conflict of interestConflict of interest

1111

GIV - AdvantagesGIV - Advantages

Four important features representing Four important features representing improvement over earlier efforts:improvement over earlier efforts:

1.1. More precise capture of differences in study More precise capture of differences in study design and risk of biasdesign and risk of bias

2.2. Assessment of the risk of bias at the level of Assessment of the risk of bias at the level of an intervention-outcome pairan intervention-outcome pair

3.3. Relatively easy to implement by research Relatively easy to implement by research staffstaff

4.4. Sophisticated algorithm to automate Sophisticated algorithm to automate calculation of internal validity scorecalculation of internal validity score

1212

Methods – GHD External Methods – GHD External Validity (GEV) Validity (GEV)

ObjectiveObjective: Allow user to consider external validity in assessing body of effectiveness evidence.

Definition: Definition: Set of indicators that describe how likely it is that the results found in a study will be replicated in a target setting.

Methods – GEVMethods – GEV Indicators. Reflect geography, population,

implementation details, ability to scale and legal and cultural context.

Indicator weights. Assigned weights to reflect relative importance of each indicator.• Delphi two rounds• Principle component analysis (PCA)• Meta-regression

Scoring. GEV instrument generates a score for each IOPT and for intervention as a whole.

Methods – Epidemiologic ModelMethods – Epidemiologic ModelParsimonious models that relate epidemiology, RRR and

coverage, to change in disease burden for each intervention; integrated with other intervention models.

Inputs Epidemiologic data – population by risk group, prevalence and incidence. Estimates of RRR for different outcomes.

Outputs Estimate key health outcomes, e.g., averted infections, deaths, morbidity, DALYs.

Calibrated parsimonious models for each intervention against full, validated models.

Methods – CostMethods – CostGHD developed a methodology for rating published costing studies, and used that data in a costing tool that generates

costs in US $ today, adjusted from other years, other currencies, and across geographic locations.

Costs of components (e.g., personnel, goods and services) were identified and adjusted using a hierarchy of sources.

GHD generated costs of interventions• By units (per person served)• By duration (per person year of service)• By method of intervention delivery

Methods – Cost-EffectivenessMethods – Cost-EffectivenessDetermining CE involves: Summing cost across time and interventions; Calculating averted disease burden;

Comparing across intervention sets (e.g. current vs proposed); and

Calculating Incremental Cost-Effectiveness Ratio (ICER).

GHD – Web Conceptual ModelGHD – Web Conceptual Model1818

Q&AQ&A

1919

Detour – GHD LIVEDetour – GHD LIVE

www.globalhealthdecisions.orgdecisions.org

2020

What’s Next for GHD? To complete data extraction and analysis (RRR, To complete data extraction and analysis (RRR,

GIV, GEV) for 3-4 more interventionsGIV, GEV) for 3-4 more interventions

NIDA proposal – focusing on integration of HIV,HCV, NIDA proposal – focusing on integration of HIV,HCV, TB, and drug abuse services for injection drug usersTB, and drug abuse services for injection drug users

NIMH proposal – expand data extraction for more NIMH proposal – expand data extraction for more HIV interventions and other disease areasHIV interventions and other disease areas

Field test GHD with policy-makers in Africa to Field test GHD with policy-makers in Africa to assess usability and uptake assess usability and uptake

2121

GHD TeamGHD Team Investigators:Investigators:James G. Khan MD, MPH (PI)James G. Khan MD, MPH (PI)

Mohsen Malekinejad MD, DrPH Mohsen Malekinejad MD, DrPH Elliot A. Marseille DrPH MPPElliot A. Marseille DrPH MPP

Jonathan Showstack PhD MPH Jonathan Showstack PhD MPH

Ali Mirzazadeh MD PhDAli Mirzazadeh MD PhD

Senior researchers:Senior researchers:

Sabina S. Alistar PhD MSSabina S. Alistar PhD MS

Stephane Verguet MS PhD MPPStephane Verguet MS PhD MPP

Research support:Research support:

Devon McCabe MADevon McCabe MA

Alex J. Goodell Alex J. Goodell

Pam DeCarlo BAPam DeCarlo BA

Erin Barker MLISErin Barker MLIS

Justina Wu MPH Justina Wu MPH

Jeff Loi MS Jeff Loi MS

Leon Traister BS Leon Traister BS

Grant Storey Grant Storey

Lena LibatiqueLena Libatique

2222

Thank You

GlobalHealthDecisions.org

2323