Analytic FrameworksPrepared for:

Agency for Healthcare Research and Quality (AHRQ)Training Modules for Systematic Reviews Methods Guide

www.ahrq.gov

Systematic Review Process Overview

To understand the use of analytic frameworks within systematic reviews

To develop an understanding of principles for developing analytic frameworks

To understand which elements of a review can be represented in an analytic framework

Learning Objectives

A type of evidence model that links and defines clinical concepts, evidence, and populations as they relate to outcomes

Sometimes referred to as causal pathways Alternative and related concepts are:

Conceptual frameworks Influence diagrams Theoretical frameworks Logic models

What Is an Analytic Framework?

To provide clinical, biological, or health services underpinnings for the mechanism of action

To clarify implicit assumptions To identify logical flaws as key questions are

developed To provide a starting point for discussions

with key informants To identify distinct bodies of evidence to be

included in the review

Why Use Analytic Frameworks?

Specifies populations, interventions, and outcomes, and sometimes comparators, timing, and settings

Identifies potential modifiers and mediators of effectiveness

Clarifies links between intermediate and health outcomes

The Importance of an Analytic Framework

Complex bodies of literature with multiple key questions: Analytic frameworks help define key questions, and thus

direct specific literature searches. Complex chain of logic between

intermediate and clinical outcomes: Analytic frameworks help users understand decisional

context.

When To Use Analytic Frameworks

Population of interest Intervention Linkages that demonstrate key questions Intermediate outcomes Ultimate health outcomes (including harms)

Components

The population, intervention, and outcomes should be clearly identified in the analytic framework.

All key questions should be represented clearly on the framework.

Contextual information about modifiers and mediators may be appropriate.

Analytic frameworks are drafted with the purpose of clarifying complex questions and will need iterative revisions through the topic refinement process (see the module, “Topic Refinement”).

Revising the Framework

Depicting Essential Components: An Example

Graphical chain of logic Arrows

Linkages Preventive service or

treatment Questions

Dotted lines Associations

Rectangles Intermediate outcomes

Rounded corners Health states

Square corners Curved arrows

Lead to ovals Harms

Harris RP, et al. Am J Prev Med 2001;20(Suppl):21-35; Woolf SH, et al. Annu Rev Public Health 1996;17:511-38.

Key Research Questions

The numbers that represent the key questions should be placed in the relevant position within the framework.

11

22

33 44

Whitlock EP, et al. Am J Prev Med 2002;22:267-84.

Sample Working Framework

Harris RP, et al. Am J Prev Med 2001;20(Suppl):21-35.

Sample Screening and Treatment Framework

Hartmann KE, et. Evidence Report/Technology Assessment No. 187. Available at: http://www.ahrq.gov/downloads/pub/evidence/pdf/bladder/bladder.pdf.

PopulationPopulation InterventionIntervention OutcomesOutcomes

Capturing the ContextAnalytic frameworks can provide a means of capturing contextual issues important to a review

Vanderbilt University Evidence-based Practice Center. Systematic review protocol: traumatic brain injury and depression. Available at: http://www.effectivehealthcare.ahrq.gov/ehc/products/77/367/TBI%20and%20Depression%20(2-9-2010).pdf.

Do not build the framework and forget it Use the framework throughout the review

process to: Revisit inclusion/exclusion criteria Keep a handle on scope Reconsider the appropriateness of key questions Guide interviews with key informants

Let the framework help in structuring the report and results

Using the Framework

Analytic frameworks help reviewers clarify specific key questions and direct literature searches of complex bodies of literature.

Analytic frameworks help end-users understand the decisional context of key questions when there is a complex chain of logic.

Analytic frameworks can help the iterative process of identifying specific researchable key questions in the context of complex clinical or policy issues.

Key Messages

Agency for Healthcare Research and Quality. Topic development. In: Methods reference guide for effectiveness and comparative effectiveness reviews. Version 1.0 [Draft posted Oct. 2007], Chapter 2, Finding evidence. Rockville, MD: Agency for Healthcare Research and Quality; Draft Posted October 2007. p. 10-15. Available at: http://effectivehealthcare.ahrq.gov/repFiles/2007_10DraftMethodsGuide.pdf.

Bravata DM, McDonald KM, Shojania KG, et al. Challenges in systematic reviews: Synthesis of topics related to the delivery, organization, and financing of health care. Ann Intern Med 2005;142(Pt 2):1056-65.

Harris RP, Hefland M, Woolf SH, et al. Current methods of the US Preventive Services Task Force: a review of the process. Am J Prev Med 2001;20(Suppl):21-35.

Hartmann KE, McPheeters ML, Biller DH, et al. Treatment of Overactive Bladder in Women. Evidence Report/Technology Assessment No. 187 (Prepared by Vanderbilt University Evidence-based Practice Center under Contract No. 290-2007-10065-I). Rockville, MD: Agency for Healthcare Research and Quality, August 2009. AHRQ Publication No. 09-E017. Available at: http://www.ahrq.gov/downloads/pub/evidence/pdf/bladder/bladder.pdf.

References (I)

Mulrow C, Langhorne P, Grimshaw J. Integrating heterogeneous pieces of evidence in systematic reviews. Ann Intern Med 1997;127:989-95.

Vanderbilt University Evidence-based Practice Center. Systematic review protocol: traumatic brain injury and depression. Rockville, MD: Agency for Healthcare Research and Quality, Posted December 9, 2009. Available at: http://www.effectivehealthcare.ahrq.gov/ehc/products/77/367/TBI%20and%20Depression%20(2-9-2010).pdf.

Whitlock EP, Orleans CT, Pender N, et al. Evaluating primary care behavioral counseling interventions: an evidence-based approach. Am J Prev Med 2002;22:267-84.

Woolf SH, DiGuiseppi CG, Atkins D, et al. Developing evidence-based clinical practice guidelines: lessons learned by the US Preventive Services Task Force. Annu Rev Public Health 1996;17:511-38.

References (II)

This presentation was prepared by Melissa L. McPheeters, Ph.D., M.P.H., a member of the Vanderbilt University Evidence-based Practice Center.

The module is based on an update of chapter 2 in version 1.0 of the Methods Guide for Comparative Effectiveness Reviews (available at: http://www.effectivehealthcare.ahrq.gov/ehc/products/60/294/2009_0805_principles1.pdf) .

Author