Health Technology Assessments and Systematic

ReviewsMarcia Reinhart, DPhil CMPP

Principal, Tantalus Medical Communications

• Many definitions

Health Technology Assessments

“Technology assessment in health care is a multidisciplinary field of policy analysis. It studies the medical, social, ethical, and economic implications of development, diffusion, and use of health technology.”

– International Association of HTA (INAHTA)

“A form of policy research that examines short- and long-term consequences of technology. . . safety, efficacy, patient-reported outcomes, real-world effectiveness, cost, and cost-effectiveness as well as social, legal, the application of a health-care ethical, and political impacts.”

– International Society for Pharmacoeconomics and Outcomes Research (ISPOR)

Regardless of the technology assessed, HTAs include similar elements

HTA

Clinical efficacy

Safety

Cost-effectiveness

Clinicians and patients

• Prescribing decisions

• Practice guidelines

Public and private payers

• Drug plan formularies

• Level of coverage

Hospitals

• Technology acquisition

• Hospital formularies

HTAs are used to support many health care decisions

• Governments/private insurers determine the price and availability of drugs via formulary designation

What is 'Market Access'?

Figure: Eichler GH et al. Nat Rev Drug Discov 2010;9:277-91.

• Just as health care systems vary worldwide, the payer assessment process differs from country to country

Market Access around the world

• National/regional government-based agencies

• Independent academic or consulting groups

Who conducts and uses HTAs?

Manufacturer

• Conducts literature review, economic analysis, etc.

• Submits HTA-like document based on agency template

submission HTA agency

• Assesses manufacturer submission

• May conduct independent review or rely on HTAs from 3rd parties

• Submits recommendation to payer

report Payer

• Makes final decision on reimbursement and availability of drug or device

• HTAs as part of the market access continuum

How do HTAs fit into the decision-making process?

Payer often agrees with HTA agency, although may be influenced by additional factors

• Cochrane Collaboration definition:“… attempts to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a given research question. Researchers conducting systematic reviews use explicit methods aimed at minimizing bias, in order to produce more reliable findings that can be used to inform decision making.”

Systematic review – an essential step of the HTA process

1. Define research question2. Develop study inclusion/exclusion criteria

Patients Intervention(s) Comparator(s) Outcomes Study type

3. Develop literature search strategy Typically conducted across multiple databases Search strategies should be broad enough to capture all relevant

publications, but narrow enough to avoid excessive irrelevant information

4. Screen studies for inclusion and collect relevant data

Systematic review: process

PICOS

1. Define research question

Example – acupuncture for the treatment of fibromyalgia

1. Define research question2. Develop study inclusion/exclusion criteria

Patients

Example – acupuncture for the treatment of fibromyalgia

Age?

Diagnostic criteria?

Comorbidities?

Disease severity?

Ethnicity?

1. Define research question2. Develop study inclusion/exclusion criteria

Patients Intervention(s)

Example – acupuncture for the treatment of fibromyalgia

Definition of acupuncture?

Frequency?

Adjunct therapies?

1. Define research question2. Develop study inclusion/exclusion criteria

Patients Intervention Comparator(s)

Example – acupuncture for the treatment of fibromyalgia

No treatment?

Placebo?

Other TCM techniques?

Drugs?

Physical therapy?

1. Define research question2. Develop study inclusion/exclusion criteria

Patients Intervention(s) Comparator(s) Outcomes

Example – acupuncture for the treatment of fibromyalgia

Pain reduction?

Days off work?

Physical function?Quality of life?

Adverse events?

Fatigue?

1. Define research question2. Develop study inclusion/exclusion criteria

Patients Intervention(s) Comparator(s) Outcomes Study type

Example – acupuncture for the treatment of fibromyalgia

RCTs?

Non-RCTs?

Quasi-randomized?

Economic evaluations?

Real-world evidence?

Small group exercise – develop PICOS criteria

3. Develop literature search strategy Use both indexing terms (e.g. MeSH) and plain text

terms Draft searches to identify studies in categories (e.g.

disease terms, intervention terms), combined at end

Example – acupuncture for the treatment of fibromyalgia

Disease terms

3. Develop literature search strategy

Example – acupuncture for the treatment of fibromyalgia

Therapy terms

Combined

3. Develop literature search strategy

Example – acupuncture for the treatment of fibromyalgia

Medline Embase

Cochrane congress abstracts

ALL COMBINED (duplicates removed)

Other sources?

4. Screen studies for inclusion

Example – acupuncture for the treatment of fibromyalgia

• Multiple studies are combined statistically to reveal the overall effect of an intervention A summary (pooled) effect estimate is calculated as a

weighted average of the effects estimated in the individual studies

Data extraction and synthesis: The meta-analysis

Individual studies• Box represents estimated effect• Lines represent 95% CI• Size of box represents weighting

Summary intervention effect

• Meta-analysis for mean difference in pain

Example – acupuncture for the treatment of fibromyalgia

• A bias is a systematic error, or deviation from the truth, in results or inferences Selection, performance, detection, attrition, reporting

• Several tools have been developed to assess bias

Risk of bias

• NICE single technology appraisal (STA)Section A – Decision problem1. Description of technology under assessment2. Context3. Equality4. Innovation5. Statement of decision problem

May include >1 research question

Role of medical writers in HTAs 

• NICE single technology appraisal (STA)Section B – Clinical and cost effectiveness6. Clinical evidence7. Cost effectivenessSection C – Implementation8. Assessment of factors relevant to

NHS/other parties9. References10. Appendices11. Related procedures for evidence

submission

Role of medical writers in HTAs • Systematic review• Meta-analysis• Non-RCT evidence• Safety data

• Systematic review for published analyses

• Report and interpret results of own economic model(s)

Section B – Clinical and cost effectiveness6. Clinical evidence

Example – new class of drug to treat rheumatoid arthritis

Section B – Clinical and cost effectiveness6. Clinical evidence

Results from the systematic review: Phase III clinical trial of new drug (B) vs placebo (A) Phase II clinical trial of new drug (B) vs placebo (A) 3 RCTs of competitor (C) vs placebo (A) 2 RCTs of competitor (C) vs other competitor (D) 3 single-arm non-RCTs

How do we use this information to compare the new drug (B) to competitors C and D??

Example – new class of drug to treat rheumatoid arthritis

• Network meta-analysis can be used to infer the comparative effectiveness of treatments of interest that have not been directly compared in trials

Network meta-analyses/indirect treatment comparison

Section B – Clinical and cost effectiveness6. Clinical evidence

Safety data AEs from RCTs and non-RCTs Post-marketing surveillance data (if available) Other sources of real-world evidence

Example – new class of drug to treat rheumatoid arthritis

Section B – Clinical and cost effectiveness6. Clinical evidence7. Cost effectiveness

Example – new class of drug to treat rheumatoid arthritis

Cost difference (+)

Cost difference (−)

Effectdifference

(+)

Effectdifference

(−)

Dominated

Dominant

Trade-off

Trade-off

SC

• A new treatment can be compared with an existing therapy based on its effect on cost and effect differences

The cost-effectiveness plane

SC = standard care

• In cost-utility analyses, effectiveness is measured in quality-adjusted life years (QALYs) QALYs incorporate both quality and quantity of life

gained from an intervention

Key concept: QALYs

Improvement of quality of life with

new treatment

• Estimated using utility values

Improvement of quantity of life with

new treatmentQALYs without new

treatment(standard care)

Time

QoL

Figure: adapted from McCabe C. Hayward Group Ltd. 2009

• Incremental cost-effectiveness ratio “costs per outcome” (cost-effectiveness analysis) or

“costs per QALY” (cost-utility analysis)

Key concept: ICER

CostsTreatment – CostsStandard care

QALYsTreatment – QALYsStandard care= ICER

Key concept: ICER

CostsTreatment – CostsStandard care

QALYsTreatment – QALYsStandard care= ICER

Total costs Life-years gained (LYG)

Utility for LYG QALYs

Treatment $20,000 7 0.5 3.5Standard Care $10,000 5 0.6 3.0

Incrementalcost, Treatment vs

Std. Care

Incrementallife-years

Incrementalcost/LYG

IncrementalQALYs

ICERcost/QALY

$10,000 2 $5000/year 0.5 $20,000/QALY

Optional group exercise – calculate the ICER of our new

arthritis drug

Economic analysis: “willingness to pay” threshold

Cost difference (+)

Cost difference (−)

Effectdifference

(+)

Effectdifference

(−)

Reject

Accept

WTP threshold

SC

• Some countries/health care systems use an official or unofficial threshold of acceptable ICERs for new technology assessments

Example – new class of drug to treat rheumatoid arthritis

Cost difference (+)

Cost difference (−)

Effectdifference

(+)

Effectdifference

(−)

Reject

Accept

WTP threshold

SC

• An economic model determines that the ICER for the new drug is $40,000/QALY

• If the WTP was $50,000/QALY, this drug would fall into the range of acceptable cost/QALY

X

• Calculating costs per QALY requires estimation of several parameters utility values cost of treatment duration of treatment/patient lifespan (“time horizon”)

• Base case = best estimate of all parameters • Sensitivity analyses test alternative parameter

estimates to assess the range of possible results

Cost-utility: sensitivity analysis

ICER ($thousand/QALY)

• Each parameter varied one-at-a-time over plausible range

One-way sensitivity analysis

Figure: Reynolds MR et al. Circ-Arrhythmia Elec 2009;2:362–69.

Tornado diagram

base case

Example – new class of drug to treat rheumatoid arthritis

Cost difference (+)

Cost difference (−)

Effectdifference

(+)

Effectdifference

(−)

Reject

Accept

WTP threshold

SC

• Sensitivity analysis shows that the new drug may rise above the WTP threshold in certain circumstances

XXX

XX

X X

• Scenario analysis Similar to a one-way sensitivity analysis, however,

assessments are made by varying multiple parameters at the same time Example = best-case and worst-case analyses

• Probabilistic sensitivity analysis “Drug X has a 86% probability of falling within the WTP

threshold of $50,000/QALY” More statistically complex; assesses distributions of data

for multiple inputs over the course of multiple simulations

Other sensitivity analyses used in HTA

Cost-utility is not the only economic analysis used in HTAs

Cost-consequences analysis Estimates cost and value of interventions, but leaves it to the reader to draw conclusions

Cost-minimization analysis Compares input costs, but assumes outcomes are equivalent (e.g., bioequivalent drug comparisons)

Cost-effectiveness analysis Measures costs in dollars and reports outcomes in natural health units (e.g., mmHg reduction) or ratios (differences in cost/difference in outcomes)

Cost-utility analysis Measures outcomes based on years of life and quality of life obtained with treatment

Cost-benefit analysis Enumerates and compares costs and benefits achieved in monetary terms

Budget impact analysis Estimates effect of intervention on overall cost to organization or health plan

• Well-versed in literature searching Experience with multiple databases and multiple

interfaces (e.g. PubMed vs Ovid)

• Attention to detail – screening and data extraction• Understanding of evidence grading systems• Knowledge of statistical methods (e.g. meta-

analysis, NMA, sensitivity analyses)• Understanding of health economic models• Excellent writing skills

Desirable qualities in HTA medical writers

Thank you