9.45 am Introducing Three QES Methods – Framework Synthesis, Meta-Ethnography and

Realist Synthesis

Drs Andrew Booth and Chris Carroll

Framework for systematic reviews of qualitative research (Garside, 2010)

Stage Typical activities

Developing research question Assemble team; Consult; Agree approach

Scoping exercise Identify relevant research; Refine methods

Identifying relevant literature Develop Inclusion/Exclusion criteria; Focused searches; Citation searches

Initial assessment of study reports Preliminary reading; Identify theories; Assess utility/relevance

Analysis and synthesis Reading & rereading study reports; Constant comparison; Assess validity

Preliminary synthesis Categorising; tabulating; mindmaps; Explore relationships

Full synthesis Thematic analysis; translation of findings; Theory development; rival explanations

Dissemination Target audiences; Limitations of review

Throughout Multiple viewpoints; Reflexivity; Audit trail; Ongoing consultation; revisit review purpose

Three QES Methods……..

Best Fit Framework Synthesis

A Framework

How do you create a framework? Develop own framework or themes or concepts

for coding Oliver S et al: A multidimensional conceptual framework for analysing

public involvement in health services research. Health Expectations 2008, 11:72-84.

Brunton G, Oliver S, Oliver K, Lorenc T. A Synthesis of Research Addressing Children’s, Young People’s and Parents’ Views of Walking and Cycling for Transport London. London, EPPI-Centre, Social Science Research Unit, Institute of Education, University of London; 2006.

Identify or create a thematic framework or conceptual model Carroll C, Booth A, Cooper K. A worked example of “best-fit” framework

synthesis: A systematic review of views concerning the taking of potential chemopreventive agents, BMC Medical Research Methodology 2011; 11: 29

An existing conceptual framework

Conrad K, et al (1996). The worksite environment as a cue to smoking reduction. Research in Nursing & Health, 19 21-31.

A Framework

Framework synthesis

Thematic synthesis

Meta-Ethnography

Meta-Ethnography

• Why look at meta-ethnography?– Most common and longest established QES method– Hannes K, Macaitis K. A move to more systematic and transparent

approaches in qualitative evidence synthesis: update on a review of published papers, Qualitative Research, 2012; 12: 402-442.

• What do we mean by meta-ethnography?• How do you complete the stages of meta-

ethnography?

Meta-ethnography“... involves taking relevant empirical studies to be synthesised,

reading them repeatedly and noting down key concepts (interpretive metaphors). These key concepts are the raw data for the synthesis. Noblit and Hare suggested that the process by which a synthesis is achieved is one of translation. This entails examining the key concepts in relation to others in the original study and across studies ... The purpose of the translation is to try to derive concepts that can encompass more than one of the studies being synthesised. The synthesised concepts may not have been explicitly identified in any of the original empirical studies”

1. Campbell R et al. Evaluating meta-ethnography: systematic analysis and synthesis of qualitative research, HTA 2011; 15: 43

2. Noblit GW, Hare RD. Meta-Ethnography: Synthesising Qualitative Studies. Newbury Park (CA): Sage Publications, 1988

Conducting a meta-ethnography (Atkins et al 2008)

Step 1: Getting startedStep 2: Deciding what is relevant to the initial interestStep 3: Reading the studiesStep 4: Determining how studies are relatedStep 5: Translating studies into one anotherStep 6: Synthesising translationsStep 7: Expressing the synthesis

Meta-ethnography: 7 phases

Phase 1: Getting startedPhase 2: Describing what is relevant to initial interestPhase 3: Reading the studies – repeated reading and

noting of concepts or metaphorsPhase 4: Determining how studies are related – task of

creating a list of key metaphors, phrases, ideas or concepts (and their relations) used in each account, and juxtaposing them. Leads to initial assumptions about relations between studies, e.g. Are they related? Are they answering the same question, considering the same problem, from the same perspective?

Meta-ethnography: 7 phasesPhase 5: Translating the studies into one another – the

metaphors and/or concepts in each account and their interactions are compared with the metaphors and/or concepts and their interactions in other accounts. First level of meta-ethnographic synthesis.

Phase 6: Synthesizing translations – unique metaphors/concepts or one able to encompass those of other accounts, which are able to produce a new interpretation/conceptual development.

Phase 7: Expressing the synthesis – communication of findings that takes account of the intended audience’s own culture and its concepts and language.

Realist Synthesis

In a nutshell, realist synthesis…

• Configures beginning theory to explain “what works for whom in what circumstances and in what respects” (Pawson, 2006, p.74)

• Configures Contexts, Mechanisms and Outcomes (CMO configurations)

• E.g. Hospital league tables cp. School league tables, Surgeon performance tables [Mechanisms?]

CMO Configurations

Example CMO Configurations

Example CMO Configurations

QUANTITATIVE e.g. Different effect sizes change on motivation scale

QUALITATIVEe.g. Explanations for employee motivation “We had low morale because…”

Realist synthesis can

• Discover weak links in an “implementation chain” (critical success factors)

• Adjudicate between rival theories• Identify contextual factors leading to

improved or impaired effectiveness• Compare anticipated performance with actual

performance

Steps in a realist synthesis – Iterative!

Identify review questions

Search for primary studies

Extract data &

appraise evidence

Synthesise findings

Sounds easy doesn’t it?

Now select you preferred workshop choices (1st and 2nd)

A.Best Fit Framework SynthesisB. Meta-EthnographyC. Realist Synthesis

And to Coffee Break……..