Building a logic model of reinforced models of palliative care

Anke Rohwer, Andrew Booth, Lisa Pfadenhauer, Louise Brereton, Ansgar Gerhardus, Kati Mozygemba, Wija

Oortwijn, Marcia Tummers, Gert Jan van der Wilt, Eva Rehfuess

The INTEGRATE-HTA project is co-funded by the European Union under the Seventh Framework Programme (Grant Agreement No. 306141)

The logic of logic models

The Logic of Logic Modelling

Complexity of health technologies and systems

Strong assumptions about the structure, content and objectives of

a health technology as well as its implementation

Negligence of system within which the intervention is intended to act and the potential interplay and co-

evolution

Describes components of complex interventions and relationships between them

Makes underlying theories of change and assumptions about causal pathways explicit

Describes interactions between the intervention and the system within which it is implemented

Logic Models“… a graphic description of a system … designed to identify important elements and relationships within that system”.

Added value of using logic models in systematic reviews

Scoping the review

• Refining review question• Deciding on lumping or splitting a review topic• Identifying intervention components

Defining and conducting the review

• Identifying relevant study inclusion/exclusion criteria • Guiding the literature search strategy • Explaining the rationale behind surrogate outcomes used in the review• Justifying need for subgroup analyses (e.g. age, sex/gender,

socioeconomic status)

Making the review relevant to policy and practice

• Structuring reporting of results• Interpreting results based on intervention theory and systems thinking• Illustrating how harms and feasibility are connected with interventions• Interpreting results based on intervention theory and systems thinking

(Anderson et al, 2011)

Method of development of logic models: four step process• (i) systematic searches for published examples of

logic models• (ii) searches for existing guidance on the use of logic

models in primary research, SRs and HTAs• (iii) development of two draft templates for system-

based and process-orientated logic models and • (iv) application of these templates in three SRs and

one HTA of different complex health technologies.

Logic models in HTA and Systematic Reviews

HTA/SR research question

Check literature for published logic models

Adoption of published logic model Adaptation of logic model

De novo logic model

Type of logic model

Subtype of logic model

A-priori logic model Iterative logic model Staged logic model

Process-based logic model System-based logic model

Three types of logic model

Label Meaning

Type

A priori logic model

A logic model that is specified as close to the inception of an HTA or SR as scoping the literature and/or stakeholder consultation permit and that remains unchanged during the HTA/SR process.

Iterative logic model

A logic model that is subject to continual modification and revision throughout the course of an HTA or SR.

Staged logic model

A type of iterative logic model that pre-specifies points at which major data inputs are anticipated to prompt a subsequent version of the logic model, thereby increasing transparency and minimising problems with version control.

Subtype: System-based logic model

Subtype: Process-oriented logic model

Populating the logic model template

SAP and expert input

Discussion within team

Literature review

Logic models in the INTEGRATE-HTA Process

Create specific logic model regarding the

theme e.g. palliative care based on the data from

step 1

Create logic model architecture and attributes for specific technologies according to a generic logic model template

Review and adaptaptation of specific model by SAPs and HTA researchers

Construction of the ELMMAR (Extended Logic Model to Map the

Assessment Results) Model: Integration of the assessment

results (effectiveness, ethics etc.)

Evidence summaries about different assessment aspects (e.g.

effectiveness, ethics)

Selection of theme for assessment e.g. palliative

care

Specific scoping

procedures for each

assessment aspect

considered

Decision-making body, HTA commissioning agency

Scoping literature overview

Literature review, SAP consultations

Identify and assess patient preferences, moderators, context and

implementation

Result

Definition of HTA research question, assessment criteria and preliminary

definition of technologies e.g. models of care and relevant issues

ResultSpecific logic model to start evidence

collection including A,B,C,D,E

Scientific assessment of evidence

Result Evidence reports and evidence summaries for each assessment

aspectResult

ELMMAR Model and synthesised evidence according

to the HTA research questionResult HTA decision/ recommendation

Step 1: HTA Objectives and Technology Step 2: Logic Model to define evidence needs Step 3: Evidence assessment Step 4: Mapping of the evidence Step 5: HTA decision-making

Summarizing and structuring the assessment results into specific assessment criteria of the HTA

research question

Review by stakeholders (HTA researchers, SAPs, decision-maker)

Selecting a tool to structure a deliberative discussion (in

cooperation with the decision committee)

Refinement of A,B,C,D, E:A) Definition of specific technologiesB) Relevant issuesC) Outcome parameterD) Relevant patient characteristics (preferences, moderators)E) Context and implementation issuesDefinition of relevant issues and

assessment criteria regarding the assessment theme (e.g. access,

continuity)

Deliberative reflections of stakeholders/decision-makers

about unanswered issues/ uncertainty/

limitations of the assessment process (steps 1- 4)

Deriving conclusions from the ELMMAR Model (HTA researchers,

SAPs, decision-maker)

Definition of stakeholder

advisory panel (SAP)

Definition of functional

requirements of the decision-making body HTA researchers

Specific requirements and evidence needs according to the specific logic model, context, implemantation and

patient groups (moderators/preferences), relevant issues

Evidence collection for all assessed aspects (effectiveness, economics,

ethical, legal, cultural, and social aspects, relevant issues)

Review of the assessment results by HTA researchers and SAPs

Completing evidence summary templates

about different assessment aspects (e.g. effectiveness,

ethics)

Presentation of HTA results obtained from steps 3 and 4 to a decision committee comprising stakeholders/decision-makers

INTEGRATE-HTA model

Ideal application of Logic Models in INTEGRATE-HTA: Staged system-based logic model

Based on the agreed HTA question and a combination of team discussions, literature and SAPs.

Initial Logic Model

1

Based on an analysis of context and implementation, patient preferences and moderators of treatment.

Iteration 1

2Based on the results of the effectiveness review and economic assessment, as well as insights generated through the socio-cultural, ethical and legal assessments.

Iteration 2

3

Based on any additional findings generated through an assessment of context and implementation.

Final Logic Model

4

System-based logic model of reinforced and non-reinforced home-based palliative care (Brereton et al. 2015)

Discussion

• Added value– Getting team on same page– Integrating various types of evidence/knowledge– Structuring subsequent SR/HTA process– Graphical means of communication

• Strenghts and limitations:– parallel development of methods and application in

demonstration HTA– Empirical testing in various SR– Each type and subtype of logic models has its specific

strengths and limitations

Conflict of Interest

• The authors have no conflicts of interest to declare

<DISCLAIMER: The sole responsibility for the content of this presentation lies with the authors. It does not necessarily reflect the opinion of the European Union. The European Commission is not responsible for any use that may be made of the

information contained therein.>

References

• Anderson LM, Petticrew M, Rehfuess E, Armstrong R, Ueffing E, Baker P, et al. Using logic models to capture complexity in SRs. Research Synthesis Methods. 2011;2(1):33-42.

• Rohwer, A., Booth, A., Pfadenhauer, L., Brereton, L., Gerhardus, A., Mozygemba, K., Oortwijn, W., Tummers, M., Van Der Wilt, G.J., Rehfuess, E. (2015) Guidance on the use of logic models in health technology assessments of complex interventions [Online]. Available from: http://www.integrate-hta.eu/downloads/