ADDIS - drugis.org | Homedrugis.org/files/drugis-pres-cvz-20140327.pdf · 1 Introduction 2 Trials database 3 Network meta-analysis 4 Break 5 MCDA 6 Disease progression ... Applicability

Introduction Trials database Network meta-analysis Break MCDA Disease progression ADDIS and CvZ References

ADDIS

Hans Hillege Douwe Postmus Gert van ValkenhoefBob Goeree Bert de Brock

CvZ, Diemen (NL), 27 March 2014


1 Introduction

2 Trials database

3 Network meta-analysis

4 Break

5 MCDA

6 Disease progression

7 ADDIS and CvZ


Making Better Use of Clinical Trials

Development of ADDIS (Aggregate Data Drug InformationSystem) for aiding Benefit Risk Assessment of new medicines

Hans Hillege (UMCG, MEB)


Outline

Background

ADDIS & the Escher project

Benefit-risk

Regulator / payer interface


Intraclass differences?


Dose-response curve-fitting (A-II antagonists)


Dose-response curve-fitting (A-II antagonists)


ISDB assessed EMA transparency

Lack of a clear and consistent policy on the reporting ofclinical trial data, e.g. in the case of irbesartan:

details about the optimal dose was missingonly 2 of the 3 trials were described in detailthe risk-benefit ratio of irbesartan was not clearly comparedwith that of enalaprilthe adverse effects section was far more detailed than theefficacy section, etc.

ISDB recommended that the EMA should develop a clearpolicy that ensures consistency from EPAR to EPAR.

ISDB Assessment of nine European public assessment reports, June 1998


The solution

Clinical assessments using a systematic review format

Systematically organised warehouse of drug information

Web-based drug knowledge network system on an XMLplatform

The system contains information at the level of detail requiredfor publication in scientific medical journalsRelational database systemTabulated and graphical output


The solution

Clinical assessments using a systematic review format

Systematically organised warehouse of drug information

Web-based drug knowledge network system on an XMLplatform

The system contains information at the level of detail requiredfor publication in scientific medical journalsRelational database systemTabulated and graphical output


http://escher-projects.org/


The Escher project


Project Escher, work package 3.2

Bridging the gap between aggregated clinical data andevidence-based drug regulation using state of the artmethods for benefit risk decision making

Implementing in usable software to be deployed not only in theregulatory domain but also in the decision-making domain ofe.g. HTA agencies, hospital and community pharmacists,medical specialists, general practitioners and patients


Prototype: global requirements

Interviews with major stakeholders of different domains:

Repository of clinical trials

Based on aggregated data

Should answer on-demand different efficacy/safety questionson a efficient, transparent and accountable way within andacross compounds

Should streamline benefit-risk decision making

Intended use at first for regulatory authorities and at a laterstage for others

Intuitive and user friendly interface


ADDIS

ADDIS: Aggregate Data Drug Information System

Key ingredients:

Structured database of clinical trials dataOn-the-fly statistics, evidence synthesisBenefit-risk decision modelling / decision supportBridging efficacy/safety to relative effectiveness

ADDIS 1.x is free/open source software

Download: http://drugis.org/addis

ADDIS 2.0 under development (IMI GetReal)

Collaborative web-based platform


http://drugis.org/


Overview of the initiatives since 2000


EMA Benefit Risk Assessment


EMA Benefit Risk Assessment

Benefits

Beneficial effectsUncertainty in the knowledge about the beneficial effects

Risks

Unfavourable effectsUncertainty in the knowledge about the unfavourable effects

Balance

Importance of favourable and unfavourable effectsBenefit-risk balanceDiscussion on the benefit-risk assessmentConclusions


EMA benefit risk project

Objectives

Improve consistency, transparency and communication ofbenefit-risk assessment

Implicit → Explicit

Five Work Packages

Description of current practice

Applicability of current tools and methods

Field tests of tools and methods

Development of tools and methods for B/R

Pilot and training (ongoing)


EMAs PrOACT-URL Framework


Effort versus Precision Trade-off








Making better use of clinical trials


The best approach to demonstrating a medicine’s value

[Bergmann et al., 2014]


Structured clinical trials databases

Enabling efficiency and transparency through automation

Gert van Valkenhoef (UMCG)


Why do we need a structured database of trials?

Evidence-based decision making time-consuming/error-prone

No comprehensive source of trial information existsTrial information is insufficiently structured

Missed opportunities to introduce more structure

Trial registration, regulatory submission and systematic review

With ADDIS, we aim to solve these problems

[van Valkenhoef et al., 2012b, 2013]


Current status: document-based workflow


The future: data-based workflow


Advantages of data-based workflows

Efficiency: automation, data re-use

Transparency: trace results back to underlying data

Flexibility: on-demand analyses without re-extraction

Learning: compare data and decision to past cases


Enabling data-based workflows

Information must be stored in structured format

Need both data and (at least some of) its semantics


Structured databases of trials

For example, the following are insufficient:

Meta-analysis data tables

Captures some dataTrial structure and data semantics unclearResult of interpretation of original trials!

ClininicalTrials.gov

Very important step in right direction!Captures most data (AEs often incomplete)Missing structural information (e.g. time dimension)Semantics and internal consistency insufficient

[van Valkenhoef et al., 2012b, 2013]


Data management in ADDIS: demo

Brief demo: Canagliflozin for type II diabetes


Network meta-analysis

Consistent estimates derived from all relevant trials

Gert van Valkenhoef (UMCG)


Why network meta-analysis?

Often, > 2 treatments must be compared

Traditional (pair-wise) meta-analysis limited

Pair-wise results: insufficient insightConcerns about inconsistencySome comparisons not studied

For n treatments, n(n − 1)/2 comparisons

Indirect evidence?

Network meta-analysis enables

Simultaneous synthesis for ≥ 2 alternativesUsing both direct and indirect evidenceAnd to assess consistency

[Dias et al., 2013b; van Valkenhoef et al., 2012c]







Indirect evidence?










Indirect evidence?





What is network meta-analysis?

Extension of pair-wise meta-analysis

Same assumptions, but applied to entire network of trials

Systematic differences between comparisons (inconsistency)difficult to detectMust be especially careful about (pure) indirect comparisonsExamine trial characteristics: are the trials similar enough?

When comparing multiple treatments, only NMA can provideconsistent basis for decision making

[Dias et al., 2013a; Jansen and Naci, 2013]


Example network 1

Dangerous: no way to check assumptions!


Example network 1

Dangerous: no way to check assumptions!


Example network 2

Assumptions can be cross-checked in loops


Example network 2

Assumptions can be cross-checked in loops


Network meta-analysis in ADDIS

ADDIS automates network meta-analysis

Supports OR and mean difference

ADDIS 2 uses GeMTC R package: many more outcomes

Methods to detect heterogeneity, inconsistency

Methods to assess convergence (MCMC)

Rank probabilities

[van Valkenhoef et al., 2012a, 2014; van Valkenhoef and van den Heuvel,

2014]


Network meta-analysis in ADDIS: demo

Brief demo: Canagliflozin for type II diabetes


Break!


Multiple Criteria Decision Analysis

MCDA for health policy decision making

Douwe Postmus (UMCG)


Determining the reimbursement status of canaglifozin

Indication: the use of canaglifozin as an add-on therapy withmetforin

How does the magnitude of the health benefits and harms ofcanaglifozin compare with existing pharmaceuticals?

The assessment should include a comparison with the mostappropriate healthcare intervention(s)The assessment should primarily focus on data derived fromusual circumstances of health care practiceThe assessment should present the uncertainties affectinginterpretation of reliability and clinical relevance of the results

[EUnetHTA Work Package 5, 2013]


Informal versus formal decision making

[Baltussen and Niessen, 2006]


The process of MCDA

Problem structuring phase

GoalA clear formulation of objectives, reached through consensus between stakeholders. Identification of all aspects relevant to the decision problem

MethodsStructured discussions, focus group meetings. First stage of divergent thinking, followed by convergent phase aimed at structuring the problem

Intermediate outcomes- Hierarchical structure of problem with decision criteria (decision tree)

- Set of decision alternatives

Scoring phase

New information obtained in the scoring phase may require a re-structuring of the decision problem

Intermediate outcomeTable with every alternative scored on each criterium, either cardinal (values) or ordinal (ranking)

Decision gateDoes the scoring table indicate a dominating alternative, or is further analysis necessary to support a decision?

OutcomeDecision based on information in scoring table

Preference modeling phase

GoalTo formalize the decision maker's preference structure in order to identify the best alternative or to rank them from best to worse

MethodsThe problem is decomposed into a set of smaller subproblems, for which preference information is obtained. Using a mathematical function this is compiled into a preference for the full problem

OutcomeDecision based on preference model

StartDecision maker confronted with decision problem

MethodsSearches for existing data in literature and databases. Interviews to elicit expert opinion


Using ADDIS to select criteria and alternatives


From relative to absolute effect measures

For reasons of statistical robustness, evidence synthesismethods estimate only relative effects

Without an estimate of the baseline risks, these relativeeffects are difficult to compare across criteria

Does a relative risk reduction of 0.5 on criterion A outweigh arelative risk increase of 1.1 on criterion B?

To overcome this problem, we have developed a simpletwo-stage procedure to convert relative effects to absoluteeffects

[van Valkenhoef et al., 2012c]
















How is this process supported in ADDIS?


ADDIS 2: implementation of EMA’s effects table

[European Medicines Agency, 2012]


Preference modeling

Objective: to construct a formal model of the decisionmaker’s preferences so that the alternative technologies canbe compared relative to each other in a systematic andtransparent way

Within ADDIS, we make use of the additive value function

v(a,w) =n∑

i=1

wivi (a)

vi (a) is the value score reflecting alternative a’s performanceon criterion iwi is the weight assigned to reflect the importance of criterion i


Preference modeling

Objective: to construct a formal model of the decisionmaker’s preferences so that the alternative technologies canbe compared relative to each other in a systematic andtransparent way

Within ADDIS, we make use of the additive value function

v(a,w) =n∑

i=1

wivi (a)

vi (a) is the value score reflecting alternative a’s performanceon criterion iwi is the weight assigned to reflect the importance of criterion i


Interpretation of the weights

The weights indicate how much more important the swingfrom worst to best on one criterion is compared to the swingfrom worst to best on the other criteria

wk > wl implies that if the decision maker had to choosebetween improving either criterion k or criterion l from theworst to the best value, he would improve criterion k

An ordinal ranking of the weights can be obtained by askingthe decision maker to rank order the swings from worst tobest on all criteria

Given an ordinal ranking of the weights, different techniqueshave been developed to assign exact values to them














Demonstration ADDIS 2 MCDA web service


Disease progression models

Extrapolating short-term and surrogate outcomes

Bob Goeree (University of Groningen)


Disease state modelling

Definition A mathematical representation of diseaseprogression

Goal To evaluate long term effects of a treatment. E.g. lifeyears gained, QALY gained, costs-effectiveness.

Predominant approach in cost-effectiveness analysis, which isthe focus of this presentation

Each model is different, however the underlying methodologyis always the same, with deviating modeling choices based onavailable data

For my master thesis I developed a prototype that supportsperforming a cost-effectiveness analysis, it remains work in progress


Disease state modelling: Disease states

No

Diabetes Diabetes

Dead

Figure : Simple disease state model

[Postmus et al., 2012]


Disease state modelling: Transition probabilities

Goal Transition probabilities are used to simulate diseaseprogression

Disease state models can be used to extrapolate for futureeffects, traditionally done in discrete time events (e.g. cyclesof one year)

Each cycle a patient can ’travel’ from one state to another,which represents disease progression

These rates are obtained from clinical trials


Disease state modelling: Utility weights and costs

Goal To approximate the achieved effects based on theamount of cycles a patient spends in a disease state

We award states with different utility rates. E.g. the ’Nodiabetes’ state has an utility weight of 0.84 (which couldrepresent effect on quality of life)

Similarly, costs are just an achieved effect (e.g. one year instate diabetes costs EUR 1805).

Often effects are also discounted for future effects, e.g. 1.5%per year.


Disease state modelling: Simulation

Goal Once we have a mathematical approximation to thedisease progression and the effects each alternative achieves,this approximation needs to be evaluated

E.g. We suppose 500 patients start in state no diabetes andwe simulate them for 20 years.


Disease state modelling: Prototype

Current integration into ADDIS, live demonstration


Disease state modelling: Limitations

Limitations of current prototype:

Does not address patient heterogeneity

Only a select set of modeling choices available

All inputs are from the decision maker. Ideally inputs arederived, in an automated way, from the available clinicalevidence

For the prototype I assumed that the data available is from asingle clinical trial. However using multiple clinical trials toinform the disease state model provides more meaningfulresults


Disease state modelling: Future work

I would like to conduct research into creating a formal approach tocreate, and evaluate, a disease state model (in an automatedfashion).

Main challenges:

Using multiple clinical trials, in an automated fashion, toinform a disease state model (Network meta-analysis)

How can we extrapolate the observed, short term, effects tolong term effects / ’hard’ clinical results like life years / QALYgained?


What can ADDIS do for you?


Thank you!

Thank you very much for your attention!The ADDIS team:

Researchers: Gert van Valkenhoef, Bert de Brock, HansHillege, Tommi Tervonen, Douwe Postmus, Hans vanLeeuwen, Joel Kuiper

ADDIS 2 developers: Daan Reid, Connor Stroomberg

ADDIS 1 developers: Maarten Jacobs, Ahmad Kamal, HannoKoeslag, Joel Kuiper, Wouter Reckman, Daniel Reid, FlorinSchimbinschi, Tijs Zwinkels


Baltussen, R. and Niessen, L. (2006). Cost Effectiveness andResource Allocation, 4(1):14.

Bergmann, L., Enzmann, H., Broich, K., Hebborn, A., Marsoni, S.,Goh, L., Smyth, J. F., and Zwierzina, H. (2014). Actualdevelopments in european regulatory and health technologyassessment of new cancer drugs: what does this mean foroncology in europe? Annals of Oncology, 25(2):303–306.

Dias, S., Sutton, A. J., Ades, A. E., and Welton, N. J. (2013a). Ageneralized linear modeling framework for pairwise and networkmeta-analysis of randomized controlled trials. Medical DecisionMaking, 33(5):607–617.

Dias, S., Welton, N. J., Sutton, A. J., and Ades, A. E. (2013b).Evidence synthesis for decision making 1: Introduction. MedicalDecision Making, 33(5):597–606.


EUnetHTA Work Package 5 (2013). Hta core model for rapidrelative effectiveness assessment of pharmaceuticals - version3.0. EUnetHTA report.

European Medicines Agency (2012). Benefit-risk methodologyproject work package 4 report: Benefit-risk tools and processes.European Medicines Agency Report No. EMA/297405/2012 -Revision 1.

Jansen, J. P. and Naci, H. (2013). Is network meta-analysis asvalid as standard pairwise meta-analysis? it all depends on thedistribution of effect modifiers. BMC Medicine, 11(1):159.

Postmus, D., de Graaf, G., Hillege, H. L., Steyerberg, E. W., andBuskens, E. (2012). A method for the early health technologyassessment of novel biomarker measurement in primaryprevention programs. Statistics in Medicine, 31(23):2733–2744.


van Valkenhoef, G., Dias, S., Ades, A. E., and Welton, N. J.(2014). Automated generation of node-splitting models for theassessment of inconsistency in network meta-analysis.(Submitted manuscript).

van Valkenhoef, G., Lu, G., de Brock, B., Hillege, H., Ades, A. E.,and Welton, N. J. (2012a). Automating network meta-analysis.Research Synthesis Methods, 3(4):285–299.

van Valkenhoef, G., Tervonen, T., de Brock, B., and Hillege, H.(2012b). Deficiencies in the transfer and availability of clinicalevidence in drug development and regulation. BMC MedicalInformatics and Decision Making, 12:95.

van Valkenhoef, G., Tervonen, T., Zhao, J., de Brock, B., Hillege,H. L., and Postmus, D. (2012c). Multi-criteria benefit-riskassessment using network meta-analysis. Journal of ClinicalEpidemiology, 65(4):394–403.


van Valkenhoef, G., Tervonen, T., Zwinkels, T., de Brock, B., andHillege, H. (2013). ADDIS: a decision support system forevidence-based medicine. Decision Support Systems,55(2):459–475.

van Valkenhoef, G. and van den Heuvel, E. R. (2014). Modelinginconsistency as heterogeneity in newtork meta-analysis. (Draftmanuscript).

Documents

ADDIS - drugis.org | Homedrugis.org/files/drugis-pres-cvz-20140327.pdf · 1 Introduction 2 Trials database 3 Network meta-analysis 4 Break 5 MCDA 6 Disease progression ... Applicability