View
2
Download
0
Category
Preview:
Citation preview
Page 1
PROSIT Publication Guidance
PROSIT Technical Document
Version 1.1
20.04.2012
Edited by: Jens Attumalil and Wendelin Schramm
Editors of previous versions:
Version 1.0: Adel Al-Ayyash, Katharina Chruscz, Franziska Herget, Lennart Koester, Klaus Kurtiak,
Monika Pobiruchin, Serife Soenmez, Sinthu Vijayanathan, and Wendelin Schramm
http://www.prosit.de
Page 2
Preamble
The PROSIT Publication Guidance (PPG) has been written to help PROSIT community members to
create scientific and evidence-based health-economic publications that fulfil a defined minimum
level of quality. Approaches as well as proper methods are explained in detail in this guide.
It’s main goal is to provide the readers with a set of tools and a clear path to follow so they can either
contribute valuable content or information to the PROSIT project or enable them to assess the
quality of information that is already part of the project.
The guidance initially relied on the Drummond and Jefferson paper “Guidelines for authors and peer
reviewers of economic submissions to the BMJ” (1). In later years this document was step-by-step
enhanced to match new requirements such as the German IQWIG method paper (2, 3).
First, the section Preparation guides the choice of methods adequate to investigate defined research
questions.
The following section Performing a health-economic study describes the methods of a good
modelling study.
Then the section How to structure and write a publication deals with the requirements of a HE
publication or reporting of HE results.
We do hope that readers find this guidance though short and in telegram style helpful to their work.
Heilbronn, July 2012
Wendelin Schramm, Jens Attumalil
How to cite this guidance:
PROSIT Disease Modelling Community. PROSIT Publication Guidance Version 1.1 [monograph on the
Internet]. Heilbronn: Heilbronn University; 2012 [cited 201X XYmonth XYday]. Available from:
http://www.prosit.de/index.php/Prosit_Publication_Guidance
License:
This document is published under the GNU Free Documentation License (FDL).
PROSIT© is a registered trademark.
Page 3
Contents
Preamble ................................................................................................................................................. 2
Preparation .............................................................................................................................................. 4
Research question ............................................................................................................................... 4
Study types and questions posed .................................................................................................... 4
Selection of alternatives .................................................................................................................. 5
Searching for evidence ........................................................................................................................ 5
Search for trials and reviews ........................................................................................................... 5
Costs .................................................................................................................................................... 6
Performing a health economic study ...................................................................................................... 9
How to structure and write your publication ........................................................................................ 11
References ............................................................................................................................................. 16
Page 4
Preparation
Research question
When formulating the research question it is necessary to meet certain criteria:
The question should be economically important.
The research question or hypothesis should be phrased in a way that takes both cost and
outcome into consideration.
The hypothesis of the economic evaluation should also be stated clearly.
The question should indicate the viewpoint of the economic evaluation (e.g. societal,
sicknessfund, patient etc.).
Study types and questions posed
Depending on the research question different types of HE studies are adequate means for the inves-
tigation.
Study type Measurement of benefits Question posed
Cost minimisation
analysis
Benefits found to be equivalent Which is the most efficient way of
achieving a given goal
(or objective)?
Cost effectiveness
analysis
Natural units (e.g. life years gained) What is the most efficient way of
spending a given budget?
Cost-utility analysis Healthy years (e.g. quality adjusted life
years (QALY), healthy years
equivalents)
See cost effectiveness analysis
Cost-benefit
analysis
Monetary terms Should a given goal (or objective)
be pursued to a greater or lesser
extent?
Table source: Drummond, Jefferson, BMJ 1996 (1)
The form of an evaluation should always be stated and a clear justification given for choosing thi
distinct type of analysis.
Page 5
Selection of alternatives
When comparing two treatments it should clearly be stated why the alternative was chosen,
furthermore the alternative should either be the most widely used treatment or the treatment that
provides best cost effectiveness. If current common practice is "doing nothing" then a comparison
with placebo should be conducted as it may open up the possibility of comparing the study with
other studies that used placebo as comparator.
The chosen alternatives should be described in sufficient detail to provide the reader with enough
information to relate the information on costs and outcomes to the alternative courses of action. In
addition, the chosen alternative must be designed to help to get as close a measure as possible of the
opportunity cost of the new treatment (1). Decision trees or other decision analytic techniques can
help the reader to follow the paths the treatments might take and provide them with information
about costs and outcomes associated with each path. Information about probabilities, cost, and
outcomes linked to paths of each treatment will help the reader to make their own decisions about
each treatment.
Searching for evidence
Economic evaluations are based on effectiveness data which can be derived from either single clinical
trials, registries or, and this is frequently necessary, from an overview of several clinical trials (meta-
analysis).
Depending on which source the evaluation relies, different questions arise and several aspects have
to be considered.
Search for trials and reviews
A good and reliable source for systematic reviews is the Cochrane Collaboration: The Cochrane
Library hosts all systematic reviews executed under the conditions and rules of the Cochrane
Collaboration.
Clinical trials, i.e. their results, are often published in renowned journals. We recommend searching
for such publications in databases like:
Pubmed/MEDLINE
EMBASE
Database of Abstracts of Reviews of Effects (DARE)
Current Controlled Trials
NHS Economic Evaluation Database
Health Technology Assessment Database
For the German health care system:
Page 6
IQWIG systematic reports
Federal Health Monitoring System (also English pages!)
Databases of the German Institute of Medical Documentation and Information DIMDI (for
the German health care system)
Be aware that for the purpose of disease modelling results have to be reported in a manner that
allows for instance deriving transition probabilities over time. Not necessarily do publications and
reviews support these requirements which means that occasionally highest level information has to
be neglected for a modelling study. Before sorting out high level evidence contacting the authors
directly and a question for additional information could be helpful.
Grading data
If possible only rely on clinical trials which are randomized, double-blinded and controlled. This type
of study design is called a gold standard and ensures high internal validity.
In order to judge the quality for publications it might be useful to allocate the Jadad Score (4) as first
orientation.
The PROSIT community grades according to the Scottish Intercollegiate Guidelines Network (SIGN)
methodology.
The GRADE network provides a similar high level alternative and is used by the Cochrane
Collaboration.
The study protocol of a clinical trial should report the design and result of the study, method of
allocation of subjects, effect size with confidence intervals, whether it is an intention-to-treat
analysis or an evaluable cohort analysis. If it is a trial with placebo control take into account to
mention suitable active comparators.
Reviews should report details about the method of synthesis or meta-analysis, search strategy,
criteria for inclusion or exclusion of studies.
Costs
What should be considered?
Costing involves estimating the amount of allocated resources (2) used and their prices (unit costs)
for example, days in hospital at a defined DRG base-case rate. Both aspects should be reported
separately.
All clinical procedures of a patient should be reported. But not all procedures take place in a regular
clinical care (efficacy) therefore it should be considered whether all procedures in a clinical trial are
distinctive of normal clinical practice (effectiveness).
Page 7
Estimates of resource quantities should be based on data of real patients collected from medical
records. Regarding the German health care system estimates of resources can be either based upon
Microcosting or Top-down-approach (5). The prices of the resources can be obtained from the
finance departments of particular institutions or from national statistics but it should be considered
that charges or fees can differ from real costs. The evaluation should comment on the extent of
which using charges may bias their estimates.
In most health economic studies only direct and indirect costs are considered, but other view points
are availbale depending on the economic perspective pursued in the HE study:
Direct costs
Direct costs refer to materials, diagnostics, treatments, pharmaceuticals, expenses related to the
production of a product/service or in this case to the diagnosis/treatment/follow-up of a health
condition. Direct cost can be summed up easily.
Indirect costs
Indirect costs are costs not linked “directly” to the medical treatment or disease like loss of income,
absenteeism and prospective treatment costs. This type of costs can’t be measures as easily as direct
cost.
Less frequently used costs
Intangible costs: This type of costs is the hardest type of costs to quantify because it consists of costs
which are caused by subjective phenomena such as physical pain, adverse effects and other types of
individual feelings. These phenomena have to traslated into costs for instance by determing a
willingness-to-pay for avoiding certain outcomes and adverse effects.
Opportunity costs: The cost of an alternative that must be forgone in order to pursue a certain
action. Put another way, the costs (and benefits) you could have received by taking an alternative
action.
Friction costs: In health care systems friction costs are sometimes used in the societal or employer
prespectives in the context of productivity losses caused by the necessary replacement of staff that
has become sick. This may comprise direct and indirect costs. Loss in work force due to illness adds
additional costs to the loss in work productivity.
In Economics: the direct and indirect costs associated with the execution of financial transactions.
Friction costs include the commissions and fees, interest rates, research time (opportunity costs),
loan origination fees, tax implications and the time value of money associated with the transaction1.
Where to search for prices of resources in Germany?
For medications ( www.rote-liste.de )
o The Rote Liste is a list of medicines in Germany and is published by the Red List
GmbH. It contains brief information on medicinal products marketed in Germany. It
is aimed at medical and pharmaceutical professionals (doctors, pharmacists,
1 http://www.investopedia.com/terms/f/frictioncost.asp#axzz1nxgBlJnb
Page 8
hospitals, etc.) with the purpose to inform them of the commercially available
preparations. It is published annually updated as a book since 1933.
G-DRG-System ( www.g-drg.de )
o German Diagnosis Related Groups, short G-DRG system. For the remuneration of
general hospital services was introduced to German hospitals in accordance with §
17b hospital financing law (KHG), an integrated, performance-based and flat-rate
compensation system. This is based on the G-DRG system (German-Diagnosis
Related Groups) system, whereby each inpatient case is remunerated by an
appropriate DRG fee.
Fees for physicians in the Statutory Health Care System ( http://www.kbv.de/8156.html, in
German)
Fees for physicians privately insured( http://www.e-bis.de/goae/defaultFrame.htm, in
German )
Currency conversion should be based on real purchasing power, rather than financial exchange rates,
which fluctuate according to money market changes.
Selection of statistical tests
The statistical tests which are used should match the data set they are used on as there are a lot of
tests which differ in target size (e.g. variance or average). The chosen test should be justified for
better traceability. This verifies the usability of the test. Normally the choice should be made before
collecting data, a minimum sample size is given by the choice of the test. As this is a complex process,
it is recommended to contact a biometrician if necessary.
The details of the statistical tests should be given as well as the confidence intervals of the observed
variables.
Page 9
Performing a health economic study
Evaluation Outcomes
An explanation about every used (comparative) outcome should be given.
Common outcomes fall into one of three categories:
o the medical events (e.g. myocardial infarction, numerb of amputations etc.), the
quality-of-life (“QALY'S”)
o the costs like direct and indirect costs as well as the "willingness to pay".
o Combined outcomes (medical and economic) like costs per life-year gained (CLYG)
If intermediate results are used as final outcomes discuss their trustworthiness.
Modelling
What is a model?
There are several definitions for the term "model". One provided by the former EU project HARMET 2
has been: “a disease model describes and simulates on the computer different courses of a disease
considering possible intervention strategies and their costs”. Such definitions have in common, that a
model may not be a perfect copy but a simplified representation of reality. The extent of this
simplification depends on the needs of those, who want to use the model. Basically there are no
good disease models, but some of them may be helpful for instance to draw conclusions or to make
decisions.
Why modelling?
In most cases economic data that were collected in clinical trials, cannot be used alone for a fully
informed decision. As a rule of thumbs all events/outcomes and all costs should be assessed in a HE
study. Thereby is the modelling of the outcome amount an essential component of evaluation. In
addition, modelling techniques loow it to expand an evaluation beyond what was observed.
What demands should be met on a modelling?
One of the main requirement is that the model is explicit and clear. To ensure this, the following
factors are essential:
full transparency with clearly described and well-founded model inputs and assumptions
sufficient depth for adequate representation of the modelled diseases with its associated
cost and the corresponding measures
flexibility to calculate multiple scenarios with varying constellations and adoption settings
possibility of determining the uncertainty in the predicted cost
use of relevant data
2 HARMET = The Harmonisation by Consensus of the Methodology for Economic Evaluation of Health Care Technologies in the European Union. Project no. BMH1-CT94-1252.
Page 10
Handling uncertainty
To figure out the influence of uncertainty on the results a sensitivity analysis should take place. By
varying input variables by +/- 10% PROSIT Markov models can show the influence of varying input on
the modelling outcomes. Extreme value calculations can help to test the robustness of the model
over the complete range of possible model inputs.
Discounting
Should the data be discounted?
All costs and effects hould be presented in discounted and nominal (i.e. undiscounted) values. The
Hannoveraner consensus statement recomends a discount rate of 5% (6) for Germany. Wheras the
IQWIG Institute demands sensitivity analyses with the discount rates of 0%, 3%, 7% and 10%.
The used rates should be justified.
What about the influences of inflation?
If inflation data is used, mention why. The inflation rate for the health care system is published by
the Ministry of Health and/or the national statistics office3.
3 For Germany: http://www.destatis.de
Page 11
How to structure and write your publication
Report contents
Health-economic publications and reports follow a different line of argumentation as for instance
when reporting a prospective clinical trial.
We suggest following structure of a health-economic publication:
Introduction and objectives:
o Introductions should be short and arresting, and they should tell the reader why you
have undertaken the study. The introduction is what motivates the audience to read
a paper and the first sentence is particularly important. The introduction should be
concise and include the key papers in that field of research. Introduction should
provide a context or background for the study (i.e., the nature of the problem and its
significance). State the specific purpose or research objective, or hypothesis tested
by the study. The research objective is often more sharply focused when stated as a
question.
Definitions
o Each thematic area has its own professional vocabulary. Therefore it is necessary to
explain concepts and definitions in advance.
Limitations
o Not every question can be answered fully, so that responses can be interdisciplinary.
Each question is under a certain limitation in terms of the appropriate response.
Methods: Patient characteristics, Cost data set, clinical data etc.
Health-economic study type and study design
Economic perspective
Model used for the analyses
o This section should be written in great detail of what was done giving a sufficiently
clear overview, so that others could reproduce the study. The guiding principle
should be clarity about how and why a study was done in a particular way. Strike a
balance between brevity and completeness, but the reader must be able to assess
the following:
What type of study was performed? (i.e. was it a randomized controlled
clinical trial? How was randomization done etc.?)
How many subjects (or samples) were included?
Who were the subjects?
Where did the subjects come from?
What were the inclusion and exclusion criteria?
What intervention (if any) was offered?
How long was the follow-up (if relevant)?
What was the response rate (for surveys/questionnaires)?
What outcomes were measured and how?
Page 12
What statistical tests were used?
Scenarios investigated
Data sources and data handling (incl. synthesis)
Underlying assumptions (if any)
Results: Medical outcomes, Economic outcomes, combined health-economic outcomes (e.g.
ICERs, CLYG), other outcomes
o Results are the core of the paper. The results should be presented in a clear,
accurate and concise format. There should be no redundancy. Be sure to include
basic descriptive data. The text should tell the story and the tables give the evidence
while the figures illustrate the highlights. Present your results in logical sequence in
the text, tables, and illustrations, giving the main or most important findings first. Do
not repeat in the text all the data given in the tables or illustrations.
Validation (Sensitivity analyses, extreme value analysis)
Influence of discounting
Discussion / Limitation
o The key questions here are a What and a How and a Where. What were the
strengths and weaknesses of your study? Then how do your findings fit in with work
published by others? Finally, where is this line of research going next? This is when
you face the challenge of deciding whether your original hypothesis is still standing.
The discussion aims to summarize your work and put it into perspective. In the era of
evidence-based medicine (EBM), this section should include the interpretation of
results within the available body of systematically collected and evaluated
knowledge. It is important to acknowledge potential limitations of the study — no
article is perfect, but equally these limitations should not be ignored. Always
remember to comment on the clinical significance of your findings. Do not repeat in
detail data or other material given in the Introduction or the Results section.
Common mistakes include citing as first study in the world, emphasizing the
strengths of the study more than its weaknesses, reiterating & inflating selected
results, going beyond the evidence and drawing unjustified conclusions.
Conclusion
o The concluding paragraphs of 2-3 lines mentioning the principal findings & their
relevance makes up the conclusion of the study. Link it with the aims of the study. It
is useful to present these as bullet points as this provides maximum impact.
Unjustified conclusions not supported by the data should never be made
Study report
In order to meet the standard outlined in this document the study should concern itself only with
relevant and comparable topics. At the beginning of the study it should be explained why the
research question was chosen and how it is important to the key potential decision makers. When
mentioning the key potential decision makers it should also be made clear which point of view the
Page 13
study takes, e.g. health care system or health insurance, and why this point of view was chosen for
the study in order to make it apparent to the reader if the study can be of importance to him or her.
Lastly, it should be explained how the costs used for calculation are made up.
Transparency
Data of one economic study is closely bound to a specific setting. Decision makers need to generalize
results or transform them so that it suits their point of view. Transparency of data helps alleviate this
process.
Economic evaluations should be executed under conservative assumptions. Report results of
sensitivity analysis which help the readers assess the robustness of study conclusions.
Furthermore every conclusion should be exclusively drawn from published data, which enables the
reader to understand and verify it.
Report on disaggregated or aggregated data?
At first report components of costs or benefit (direct costs, indirect costs, life years gained,
improvement in quality of life) in a disaggregated form. Afterwards combine them in appropriate
indices or ratios.
Readers should be able to calculate other ratios which might fit better regarding their respective
purpose or point of view.
Reporting about cost data
Estimates of resources used and their prices have to be reported separately to help the reader judge
their relevance to his/her setting. Beside of this, the costs of all procedures done to a patient during
a clinical trial should be included because these costs could possibly influence outcome.
The dates of both the estimates of resource quantities and prices should be recorded, along with
details of any adjustments to a more recent price level. Adjustments to the inflation of preceding
years, modifications to show the relevant perspective as well as discounting is to be reported (2).
Rating data
If possible only rely on clinical trials which are randomized, double-blinded and controlled. This type
of study design is called gold standard and ensures highest internal validity.
In order to judge the quality for publications it might be useful to allocate the Jadad score.
The study protocol of a clinical trial should report the design and result of the study, method of
allocation of subjects, effect size with confidence intervals, whether it is an intention-to-treat
analysis or an evaluable cohort analysis. If it is a trial with placebo control take into account to
mention suitable active comparators.
Page 14
Overviews should report details about the method of synthesis or meta-analysis, search strategy,
criteria for inclusion or exclusion of studies.
At the end of the publication / report provide the reader with information how you assessed the
sources used in this publication. For example give the full table of Jadad scores you obtained from
your sources.
Missing data
If there is no available source (study, overview) it is justifiable to use other sources like expert
opinion. To collect opinions of several experts a delphi panel could be the method of choice.
Naturally information gained under such conditions is not as reliable as outcomes from a clinical trial
for example.
Maybe not every information was reported in the publication. Contact the author to gain additional
knowledge and information you might need.
There is also the possibility that the clinical trial was too insignificant or maybe the result was not
positive or neutral (p-Value > alpha value). Those trials are likely to be never published in journals
although their data might be essential for an economic evaluation. This phenomenon is called
publication bias. Funnel plots help to assess whether there is publication bias or not.
Missing data can be acquired by modelling.
Regardless which kind of source is used in an economic evaluation it should be considered that any
limitations which weaken the assessment of effectiveness (e.g. clinical trials, overview) deteriorate
any economic evaluation.
Information about tests used
The result depends on the Statistical tests used. That means a sample can be interpreted in different
ways in which all of them look evident. So it is important to justify the Selection of Statistical Tests.
Especially if approximated tests are used. Strengths and Weak Points of the Test should be alluded,
also resulting consequences for the sample and the Research Question. Reporting results does not
mean reporting the final conclusion. For better understanding it is helpful the present all
intermediate results. During this the reader can follow the chain of evidence by himself. Confidence
intervals around the main variables should be specified (5).
Comparison
Authors often compare their cost effectiveness ratios with other studies using league tables.
Those rankings should be treated with caution.
Page 15
Firstly, differences might arise from different methodological approaches rather than real
differences in intervention.
Secondly, take into consideration how the cost effectiveness or cost-utility ratio of each
figure in the league was generated; maybe one intervention was compared to placebo
control. Another intervention was compared to current gold standard in healthcare.
Summarising it, only comparisons with health care interventions should be made which are quite
similar in setting and study design.
Page 16
References
1. Drummond MF, Jefferson TO. Guidelines for authors and peer reviewers of economic submissions to the BMJ. The BMJ Economic Evaluation Working Party. BMJ. 1996;313(7052):275-83. Epub 1996/08/03. 2. (IQWiG) IfQuWiG. Entwurf einer Methodik für die Bewertung von Verhältnissen zwischen Nutzen und Kosten im System der deutschen gesetzlichen Krankenversicherung. Cologne2009. p. 74. 3. Schwalm A, Danner M, Seidl A, Volz F, Dintsios CM, Gerber A. [IQWiG's methods for the cost-benefit assessment : Comparison with an international reference scenario]. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. 2010;53(6):615-22. Epub 2010/05/08. Wo steht die Kosten-Nutzen-Bewertung des IQWiG : Abgleich mit einem internationalen Referenzszenario? 4. Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Controlled clinical trials. 1996;17(1):1-12. Epub 1996/02/01. 5. (IQWiG) IfQuWiG. Technischer Anhang Kostenbestimmung. 2008. 6. J.-M. Graf von der Schulenburg WG, F. Jost, N. Klusen, M. Kubin, R. Leidl, T. Mittendorf, H. Rebscher, O. Schöffski, C. Vauth, T. Volmer, S. Wahler, J. Wasem, C. Weber. Deutsche Empfehlungen zur gesundheitsökonomischen Evaluation - dritte und aktualisierte Fassung des Hannoveraner Konsens. 2007.
Recommended