View
50
Download
1
Category
Preview:
DESCRIPTION
Realist Synthesis 101. Dr Andrew Booth. Illustrating a Realist Review approach. For Which Schools Do School Performance League Tables Work? How Do School Performance League Tables Work? Do they work in a similar way to Hospital Performance League Tables? - PowerPoint PPT Presentation
Citation preview
Illustrating a Realist Review approach
• For Which Schools Do School Performance League Tables Work?
• How Do School Performance League Tables Work?
• Do they work in a similar way to Hospital Performance League Tables?
• What about Individual Surgeon Performance League Tables?
Developing Programme Theory • Programme Theory #1: Proponents of performance league tables believe their
publication stimulates competition, • Programme Theory #2: and that, as each provider adopts “best practice,” the
quality of services will improve.6 • Supporting Data: Some published evidence seems to support this. New York
probably has the best established system for providing the public with information… Soon after publication of performance league tables based on providers, the risk adjusted mortality for bypass surgery declined, leading some to conclude that this was a direct result.7
• Counter Argument: One alternative explanation being that, once providers know their data will be used for comparative purposes, they may resort to “creative reporting.” 8 9
• Programme Theory #3: Performance league tables may also improve patients' choice, and proponents argue that this is necessary for an efficient market economy by encouraging consumers to seek out high ranking providers.9
Developing Propositional Statements (IF-THENs)
• IF Performance Tables are published THEN:– Staff at Well-performing schools (Context1) will change their
reasoning/actions because (Mechanisms1) (e.g. Maintain competitive advantage)…..with the result (Outcome1) that……
– Staff at Average-performing schools (Context2) will change their reasoning/actions because (Mechanisms2) (e.g. Identification of good practice)…..with the result (Outcome1) that…….
– Staff at Poor-performing schools (Context3) will change their reasoning/actions because (Mechanisms3) (e.g. Shame)……with the result (Outcome1) that……..
• CMO configurations where Mechanisms = changes that occur in people's reasoning and actions
Realist synthesis
Programme intervention
Context
Altered contextContext triggers
and Interacts with
Mechanisms Mechanisms
Outcomes (intended and unintended)
Steps for conducting a realist synthesis
1. Formulate review question2. Scope the primary literature3. Decide on the scope of the review4. Identify the programme components5. Map the programme theory(s)6. Assess effectiveness of components, considering context and mechanisms7. Develop propositional statements8. Identify patterns in propositional statements across cases (demi-
regularities)9. Search sources for theories related to the propositions10. Produce a set of summary statements on what works for whom, in what
circumstances and at what point in time (mid-range theory)
Formulating the review question
• Realist questions aim to identify diverse literature on the intervention of interest, in order to evaluate what works for whom in what circumstances
• They are relatively open ended, aiming to capture a broad spectrum of relevant primary research in the first instance
• They aim to identify the relationships between Intervention, context, mechanisms and outcomes
Our realist questions
(1) What approaches to community engagement are most effective in promoting peer support, to which people and in what circumstances?
• (2) How does community-based peer support impact on understanding of existing health information and use of health information and health services to improve health and reduce health inequalities?
Steps for conducting a realist synthesis
1. Formulate review question2. Scope the primary literature3. Decide on the scope of the review4. Identify the programme components5. Map the programme theory(s)6. Assess effectiveness of components, considering context and mechanisms7. Develop propositional statements8. Identify patterns in propositional statements across cases (demi-
regularities)9. Search sources for theories related to the propositions10. Produce a set of summary statements on what works for whom, in what
circumstances and at what point in time (mid-range theory)
Preliminary theory for COPES
• Community engagement aiming to promote peer support• increases health literacy• improves health • reduces health inequalities
Assumptions• Communities initiate and foster peer support• The process of providing community-based peer support
influences the ability to develop health literacy
• Health literacy creates a sense of empowerment, which is associated with a reduction in health inequalities
• Community-based peer support raises awareness of issues with literacy and empowerment
Community Peer support Health literacy
Steps for conducting a realist synthesis
1. Formulate review question2. Scope the primary literature3. Decide on the scope of the review4. Identify the programme components5. Map the programme theory(s)6. Assess effectiveness of components, considering context and mechanisms7. Develop propositional statements8. Identify patterns in propositional statements across cases (demi-
regularities)9. Search sources for theories related to the propositions10. Produce a set of summary statements on what works for whom, in what
circumstances and at what point in time (mid-range theory)
Scoping the literature
• For the initial scoping, do a comprehensive search to ‘map the territory’
• Abstract sift: Identify and categorise abstracts by programme theory, type of intervention
Scoping the literature for key concepts• Community engagement: Very small number of articles documenting ‘working
collaboratively with and through’ community engagement for peer support
• There were variations in community-based peer support in terms of • Amount of ‘emotional, appraisal and informational
assistance’ • Degree of ‘created social network member’• Degree of ‘experiential knowledge of a specific behaviour or
stressor’• Definitions of ‘Similar characteristics’
• The term health literacy was not used in peer support articles• so included all articles on health education/health
promotion relating to ‘Understanding and use of health information and health services’
• Reduction of health inequalities – or any explicit discussion of health inequalities – was absent
Identifying sources of information in real life settings
Define concepts
Compare definitions
Identify what works
Network with participants
Scope the literature
Explore gaps
Test the emerging theory
Propose a model
Research knowledge Community knowledge
PractitionersCommunity organizations Peer supportersExpert patients
Health librarianQualitative researchersSystematic reviewersCBPR researchers
Steps for conducting a realist synthesis
1. Formulate review question2. Scope the primary sources of information3. Decide on the scope of the review4. Map the programme theory(s)5. Identify programme components6. Assess effectiveness of components, considering context and
mechanisms7. Develop propositional statements8. Identify patterns in propositional statements across cases (demi-
regularities)9. Search sources for theories related to the propositions10. Produce a set of summary statements on what works for whom, in
what circumstances and at what point in time (mid-range theory)
Defining key concepts: community engagement and peer support
• Community engagement is ‘the process of working collaboratively with and through groups of people affiliated by geographic proximity, special interest, or similar situations to address issues affecting the well-being of those people…It often involves partnerships and coalitions that help mobilize resources and influence systems, change relationships among partners, and serve as catalysts for changing policies, programs, and practices (CDC, 1997)
• Peer support within a healthcare context is “...the provision of emotional, appraisal and informational assistance by a created social network member who possesses experiential knowledge of a specific behaviour or stressor and similar characteristics as the target population.” (Dennis, 2003). Emotional support includes expressions of caring, empathy, and encouragement, and is seen to enhance self-esteem. Appraisal support involves encouraging persistence and optimism for resolving problems, affirmation of a peer’s feelings, and reassurance that frustrations can be handled. Informational support is providing advice, suggestions, alternative actions, feedback and information relevant to the issue with which the peer is dealing.
Defining key concepts: health literacy
• Health literacy – ‘the personal, cognitive and social skills which determine the ability of individuals to gain access to, understand, and use information to promote and maintain good health. These include such outcomes as improved knowledge and understanding of health determinants, and changed attitudes and motivations in relation to health behaviour, as well as improved self-efficacy in relation to defined tasks. Typically these are outcomes related to health education activities…’ (Nutbeam, 2001)
• From a number of definitions for health literacy – we chose one that related to the notion of empowerment because it was relevant to health inequalities
Improved Health Literacy
Tailored information, communication, education
Improved health outcomes,healthy choices
and opportunities
Skills in negotiation and
self management
Prior understanding of individual capacity - reading fluency, numeracy,
existing knowledge
Developedknowledge
and capability
Skills in social organization
and advocacy
Participation in changing social
norms and servicepractices
Engagement insocial
action/advocacy for health
Figure 3: Developing interactive and critical health literacy skills
Changed healthbehaviours and
practices
(Nutbeam, 2009)
Health literacy model proposed by Sorensen et al (2012)
• A year into our review, a conceptual analysis of the definitions was conducted, and a unifying model was produced (Sorensen et al, 2011). This model focuses on the capabilities of accessing, understanding, appraising and applying health information.
World
Decide on the scope of the review
• Things to consider:– Amount of literature: Comprehensive versus sampling?– Diversity of literature: Consensus on programme theory?
Variations on theory by what, who, when, setting– Access to other sources of information– Interests of those who commissioned the review
• What was the original problem that needs research evidence?• International, national, regional, local?• What will the findings be used for?• Who is the audience? (this is sometimes wider than just
commissioners)– Time and money– Experience
MeTeam
World
Setting the parametres via sampling• Programme theory was not used for sampling because
– Theory was only explicitly cited in 28 instances– Studies combined elements from the Heath Belief Model,
Theory of Planned Behaviour, social cognitive and social learning theories, the social support literature, social comparison theory, social network approaches, and empowerment models
– Studies used both individual behaviour change and community-based theories, including community organization, diffusion of innovation, and social marketing
– Domains for all the individual behaviour change theories are included in the Capabilities Opportunity Motivation Behaviour theory (COM-B, Michie et al 2011)
• Pragmatic sampling on location: the most common United Kingdom health conditions in the peer support literature were reviewed first
Mapping literature topics to UK settings
TOPIC Advisory Network
Breastfeeding Sheffield Well Being Consortium; Sheffield Breastfeeding studyDarnall WellbeingAGE UK; Stroke AssociationIrish Gypsy & Travellers Movement in Britain; Leicester Health AmbassadorsCentre for HIV & Sexual Health; Parent to Parent, Shout; Sheena Amos Trust; Expert Patients ProgrammeSheffield Community Chronic Pain; Manor & Castle Development TrustMECOPP Minority Ethnic Carers of People Project; Autism PlusSharrow ShipShape; Sheffield PCTMIND; MencapSheffield City Council SOAR (Southey Owlerton Area Regeneration)ZEST Community Development Trust
Diabetes
Healthy Living Older People
HIV/AIDS
Health Trainers
Nutrition
Smoking
Steps for conducting a realist synthesis
1. Formulate review question2. Scope the primary literature3. Decide on the scope of the review4. Identify the programme components5. Map the programme theory(s)6. Assess effectiveness of components, considering context and mechanisms7. Develop propositional statements8. Identify patterns in propositional statements across cases (demi-
regularities)9. Search sources for theories related to the propositions10. Produce a set of summary statements on what works for whom, in what
circumstances and at what point in time (mid-range theory)
Identify the programme components• Programmes categorised by Stage
– Recruitment of peers– Training and supporting peers– Implementing peer support - what was
delivered– Outcomes
• Within each Stage, components were identified in the literature and through workshops with community members
• Components put together in CMO configurations by the topic leads
Steps for conducting a realist synthesis
1. Formulate review question2. Scope the primary literature3. Decide on the scope of the review4. Identify the programme components5. Map the programme theory(s)6. Assess effectiveness of components, considering context and mechanisms7. Develop propositional statements8. Identify patterns in propositional statements across cases (demi-
regularities)9. Search sources for theories related to the propositions10. Produce a set of summary statements on what works for whom, in what
circumstances and at what point in time (mid-range theory)
Mapping programme theories• The theories and assumptions were documented
using Context Mechanism Outcome (CMO) configurations for each health topic
Type of finding Ref ID
What works well/not well CONTEXT MECHANISM OUTCOME
authors' thoughts
2668 Retaining peer supportersAmongst those consented to be peer supporters attrition rates were low 87% of those nominated agreed to become peer supporters and 82% completed the trial
PS felt pride in being nominated by their peers. It was novel to be taken out of school for training days and to miss lessons to attend follow up sessions and they may have been committed to reducing smoking amongst their peers, also incentive offered to those who completed trial
The target of 15% of the cohort were trained and delivered the intervention to the end of the trial period
study protocol
6895 & LC001
Retaining PS £10 voucher and certificate of achievement offered to PS who completed the trial
PS responded to this incentive and remained committed to trial
Low attrition and majority of PS completed trial
study protocol
6895 High ratio trainer : student
maximum of 1:15 trainers to students
Group work easily facilitated and trainers developed good relationships with peer supporters. Role play activities were successful
Students motivated throughout training and follow up visits. Role play helped students anticipate the types of different reaction they might receive when attempting to disseminate information
Dealing with diverse theories• Some topics used a single theory, for example Diffusion of
Innovation• Other topics did not explicitly state how theory informed
design, but contained implicit assumptions about what would work
• Concepts from both stated/implicit theories identified via CMOs
• Dimensions of concepts for one topic used to question concepts in another topic area
• For example, ‘recognising someone as a peer’ was an important mechanism. We asked ‘what does recognising someone as a peer mean across different health topics?
• The dimensions we discovered included age, ethnicity, experience with the problem or condition
• We also used Advisory Network as a source of propositional statements
Steps for conducting a realist synthesis
1. Formulate review question2. Scope the primary literature3. Decide on the scope of the review4. Identify the programme components5. Map the programme theory(s)6. Assess effectiveness of components, considering context and
mechanisms7. Develop propositional statements8. Identify patterns in propositional statements across cases
(demi-regularities)9. Search sources for theories related to the propositions10. Produce a set of summary statements on what works for
whom, in what circumstances and at what point in time (mid-range theory)
Iterative
Turning CMOs into propositional statements: analysis of the literature
Overarching propositional statement
Supporting data by topicSmoking cessation young people
Healthy living in older people
Smoking in ethnic minorities
IF target community recognise PS as their peer THEN PS is effective
I 5/6:Target community must recognise the peer supporters as their peers.(Propositional statement)
I12:When age is only common characteristic, and differences like ethnicity exist (meaning potentially different needs and value systems), the peers may have a negative experience. (Propositional statement)
D88Smoking cessation service is offered by health professionals, who created a formal and impersonal atmosphere, but smokers feel a sense of distrust towards providers(Mechanism)
Steps for conducting a realist synthesis
1. Formulate review question2. Scope the primary literature3. Decide on the scope of the review4. Identify the programme components5. Map the programme theory(s)6. Assess effectiveness of components, considering context and mechanisms7. Develop propositional statements8. Identify patterns in propositional statements across cases (demi-
regularities)9. Search sources for theories related to the propositions10. Produce a set of summary statements on what works for whom, in what
circumstances and at what point in time (mid-range theory)
Searching for related theories
• CMOs and propositional statements (IF-THENs) considered in terms of higher order concepts, for example
Overarching propositional statements Concepts
IF the target community recognise the PS as their peer THEN peer support is effective
Social learning
Relational learning
Participatory parity
Trust
IF there is a positive relationship between the PS and the peers (trust and a sense that the person is believable, credible) THEN peer support is effective
Steps for conducting a realist synthesis
1. Formulate review question2. Scope the primary literature3. Decide on the scope of the review4. Identify the programme components5. Map the programme theory(s)6. Assess effectiveness of components, considering context and mechanisms7. Develop propositional statements8. Identify patterns in propositional statements across cases (demi-
regularities)9. Search sources for theories related to the propositions10. Produce a set of summary statements on what works for whom, in what
circumstances and at what point in time (mid-range theory)
Mid-range theory• ‘Theories that lie between the minor but necessary working hypotheses
that evolve in abundance during day-to-day research and the all-inclusive systematic efforts to develop a unified theory that will explain all the observed uniformities of social behaviour, social organization and social change.’ (Merton, 1967: 39)
Mid-range theory
• Mid-range theories relate empirical data to higher order constructs, and are ‘sufficiently abstract to deal with different spheres of social behaviour and social structure, so that they transcend sheer description or empirical generalisation’ (Merton, 1967: 68).
• Middle range theory involves abstraction, of course, but…close enough to observed data to be incorporated in propositions that permit empirical testing.”
References
Dennis, C.-L. (2003). Peer support within a health care context: a concept analysis. International journal of nursing studies, 40(3), 321–32. Kincheloe, J. and McLaren. Rethinking theory and research. (2005) In NK Denzin & YS Lincoln (eds) Sage Handbook of Qualitative Research. Merton R. On Theoretical Sociology. Five Essays, Old and New. New York: The Free Press, 1967Nutbeam, D. (2008). The evolving concept of health literacy. Social science & medicine (1982), 67(12), 2072–8. Pawson, R. (2013) the science of evaluation: A realist manifesto. London: Sage.Sørensen, K., Van den Broucke, S., Fullam, J., Doyle, G., Pelikan, J., Slonska, Z., & Brand, H. (2012). Health literacy and public health: a systematic review and integration of definitions and models. BMC public health, 12(1), 80. Wong, G., Greenhalgh, T., Westhorp, G., Buckingham, J., & Pawson, R. (2013). RAMESES publication standards: realist syntheses. BMC medicine, 11(1), 21. doi:10.1186/1741-7015-11-21
Recommended