Upload
cadth-acmts
View
194
Download
0
Embed Size (px)
Citation preview
On evidence in HTA: Bringing patient, caregiver and community experience in
Taking account of what can’t be counted: The place of qualitative evidence in HTA
Fiona A. Miller, PhD @FionaAliceMillAssociate Professor, IHPME, THETACADTH SymposiumApril 11, 2016, Ottawa
2
3
What is evidence?
• Origins (Upshur, 2001)
– Middle English: via Old French from Latin evidentia, from evident- 'obvious to the eye or mind' (Oxford)
• Definitions (Upshur, 2001)
– something that makes plain or clear; an indication or sign (Dictionary.com)
– something that furnishes proof (Merriam-Webster)• Facts alone are not evidence (Madjar & Walton, 2001)
– “facts can only become evidence in response to some particular question” (Chandler et al, 1994)
4
What questions might HTA need to answer?
Describing/ Characterizing• Patients
– The meaning and experience of illness
• Technologies– The meaning and
experience of current and proposed technologies
• Services/ Systems– The arrangement and
operation of current and proposed services
Interrogating/ Explaining• Patients
– How social, cultural and economic contexts condition illness
• Technologies– How social, cultural and
economic contexts condition the effects of technology
• Services/ Systems– How services and systems
condition experiences and outcomes
– What makes some interventions work better than others
5
Answering “other” questions: The need for “other” evidence
• From and about persons– ‘knowledge in the possession of people’ (JBI, 2014)– Physically, socially, culturally embodied and embedded
• Only partially captured in the clinical and economic evidence traditionally used in HTA
6
Sources of “other” evidence
Research-based• Often “qualitative”– Interviews, focus groups– Ethnographic
observation– Questionnaires,
documents• Descriptive• Interpretive• Theory
Opinion-based• Patient experts - Expert
opinion– Input– Participation– Deliberative engagement
• Descriptive• Interpretive• Judgment
Often seeks to “give voice”
May use research-based methods
7
ON RELEVANCE AND RIGOURThe quality of “other” evidence
8
Relevance
• On relevance– Being ‘pertinent to…’, ‘bearing upon…’, ‘connected with…’,
or ‘appropriate to…’, ‘…the matter at hand’, as well as ‘germane’, ‘apropos’, ‘material’, and ‘applicable’ (Dobrow et al, 2015)
• Opinion has a priori relevance– Local, specific• (And other functions beyond the probative)
• Research has complex relevance – Aggregative function – persistence of themes– Configurative function – unanticipated connections
between themes (Sandelowski et al, 2012)– Comparative function – the contingency of current practice
9
Rigour
• On rigour– The quality of being extremely thorough, exhaustive, or
accurate … meticulous, thorough, careful, diligent, scrupulous, exact, precise, correct
• A function of act and actor– Appropriate– Expert– Trustworthy
10
Critical appraisal tools• QARI, JBI– Critical appraisal of
qualitative evidence
• NOTARI, JBI– Critical appraisal of text
and opinion based evidence
11
QARI Critical Appraisal Criteria (JBI, 2014)
• 1. Congruity between the stated philosophical perspective and the research methodology
• 2. Congruity between the research methodology and the research question or objectives
• 3. Congruity between the research methodology and the methods used to collect data
• 4. Congruity between the research methodology and the representation and analysis of data
• 5. There is congruence between the research methodology and the interpretation of results
• 6. Locating the researcher culturally or theoretically • 7. Influence of the researcher on the research, and vice-versa, is addressed • 8. Representation of participants and their voices • 9. Ethical approval by an appropriate body • 10. Relationship of conclusions to analysis, or interpretation of the data
12
NOTARI Critical Appraisal Criteria (JBI, 2014)
• 1. Is the source of opinion clearly identified? • 2. Does the source of opinion have standing in the field of
expertise? • 3. Are the interests of patients/clients the central focus of the
opinion? • 4. Is the opinion’s basis in logic/experience clearly argued? • 5. Is the argument that has been developed analytical? Is the
opinion the result of an analytical process drawing on experience or the literature?
• 6. Is there reference to the extant literature/evidence and any incongruence with it logically defended?
• 7. Is the opinion supported by peers?
13
Rigour
• On rigour– The quality of being extremely thorough, exhaustive, or
accurate … meticulous, thorough, careful, diligent, scrupulous, exact, precise, correct
• A function of evidence systems (Pang et al, 2003)
– Stewardship– Financing: securing and allocating funds– Creating and sustaining resources– Producing, utilizing and synthesizing evidence
14
Health “evidence” systems (Pang et al, 2003)
15
In conclusion …
• There are questions that cannot be answered without “other” evidence– Concerning the embodied and embedded ways in which persons
understand and experience illness and its management, and how services and systems do and could condition these realities
• Such evidence can be generated through research (e.g., qualitative research) and/or expert opinion– Which can be more or less relevant– And more or less robust
• The quality of such evidence is a function– Of acts and actors– And of evidence systems
• If we need this evidence, we need systems to support its quality and use
16
References
• Upshur, RE. (2001). The status of qualitative research as evidence. Morse, Janice M., Janice Swanson, and Anton J. Kuzel. (Eds.) The nature of qualitative evidence. Sage, 2001: 5-26
• Madjar, I & Walton, JA. (2001). What is problematic about evidence? Morse, Janice M., Janice Swanson, and Anton J. Kuzel. (Eds.) The nature of qualitative evidence. Sage, 2001: 5-26
• Chandler, JK., Davidson, AI., Harootunian, HD. (1994) Editor’s Introduction, Questions of evidence: Proof, practice, and persuasion across the disciplines. Chicago: University of Chicago Press, 1-8
• The Joanna Briggs Institute. (2014) Joanna Briggs Institute Reviewers’ Manual: 2014 edition. The Joanna Briggs Institute
17
References
• Sandelowski, M., Voils, C. I., Leeman, J., & Crandell, J. L. (2012). Mapping the mixed methods–mixed research synthesis terrain. Journal of mixed methods research, 6(4), 317-331.
• Pang, T., Sadana, R., Hanney, S., Bhutta, Z. A., Hyder, A. A., & Simon, J. (2003). Knowledge for better health: a conceptual framework and foundation for health research systems. Bulletin of the World Health Organization, 81(11), 815-820.
• Dobrow, MJ., Miller, FA., Frank, C., Brown, AD., (2015) Understanding Relevance of Health Research: Considerations in the Context of Research Impact Assessment, Commissioned by the Ontario SPOR Support Unit, IHPME White Paper Series
18
Questions?