Upload
consortium-for-health-policy-systems-analysis-in-africa-chepsaa
View
50
Download
1
Tags:
Embed Size (px)
Citation preview
Rigour and ethics
IHPSR Presentation 8
www.hpsa-africa.org
@hpsa_africa
www.slideshare.net/hpsa_africa
Introduction to Health Policy and
Systems Research
Four key steps in HPSR
1. Identify research focus
(problem/concern/ opportunity) and
question
2. Design study
3. Ensure quality and rigour
4. Apply ethical principles
HPSR rigour
• Health policies and systems are politically and
socially constructed – issues of focus in HPSR are
complex phenomena
• HPSR demands:
– An active process of questioning and checking during the
inquiry
– A constant process of conceptualising and reconceptualising
– Crafting interpretive judgements
– Researcher reflexivity
‘Researcher as instrument’
• Enquiring mind
• Good listening
• Adaptiveness and flexibility
• Grasp of the issues
• Lack of bias (reflexivity)
Researcher reflexivityRobson, 2002
• Identify your personal issues in relation to topic
• Clarify your value system
• Identify areas of possible role conflict
• Identify gatekeepers and how they will influence
you
• Identify where you are not neutral
Study validity/trustworthiness
Fixed designs
• Are the findings valid?
• Are the findings
statistically
generalisable?
Flexible designs
• Are the findings
plausible?
• Do the findings provide
theoretical insights that
have a sufficient degree
of generality to be
projected to other
contexts? (analytic
generalisability)
Rigour in fixed design
• Validity
– does the study measure what is intended?
– internal validity: cause-effect relationships
• Threats to validity
– bias: selection, measurement, confounding
• Reliability
– participant
– observer
• Generalisability (external validity)
– representative sample
– valid and reliable methods
Rigour in RCTs(randomised control trials)
• Pre-specified study outcomes
• Control groups
• Random allocation to intervention and control
(randomisation)
• Intervention fidelity
• Contamination and co-intervention
• Blinding
• CONSORT guidelines
Challenges to RCTs in HPSR
• Sampling units in health systems research (costs!)
• Control groups not always possible
• Complex adaptive systems
– intervention process not under researcher control
– context specificity
• Improved performance requires combined (complex)
interventions
• Cause-effect relationships not easy to establish
• Often have to construct models of complex systems and human
behaviour based on theoretical or practical assumptions
Strategies to enhance rigour in
fixed HPSR design
• Focus on plausibility rather than probability
• Mixed methods and triangulation
• Sensitivity analyses in modelling
• Drawing on theory
• Statistical techniques
– multi-level analysis
– structural equation modelling
– quantitative modelling of system dynamics (causal
loops)
Rigour in flexible design Robson, 2002
Threats lie in:
– description
– interpretation
– theory (rival explanations)
Bias from:
– reactivity (e.g. Hawthorne effect)
– respondent (e.g. social desirability bias)
– researcher (e.g. preconceived ideas)
Strategies to enhance rigour in
flexible HPSR design
• Use theory (at different stages)
• Use literature and a careful protocol
• Clear exposition of methods
• Multiple sources of data collection
• Triangulation (data, observer, methodological, theory)
• Prolonged involvement
• Negative/’deviant’ case analysis
• Counterfactuals (rival explanations)
• Member checking (respondent validation)
• Peer debriefing/support
• Reflexivity
Case study
Validity in case study work requires
paying attention to the trustworthiness of
the interpretive analysis and the resulting
generalisable claim
Critical case study issues
• Case selection and sampling
• Data collection approach and procedures
– prolonged engagement with cases
• Analytic procedures:
– respondent validation (member checking)
– triangulation across data sets and with theory
– negative case analysis
– peer debriefing and support
• Clear report of methods of data collection and analysis
(audit trail)
Analytic/theoretical
generalisation
• Develop ‘theoretical’ insights’ (generalisable
claims) by:
– building or testing theory and/or
– comparative analysis across multiple cases
• These insights are universal enough to have
relevance in other settings
Four ‘tests’ Yin, 2009 (adapted with Robson, 2002)
Case study tactic Phase of research
Confirmability
(the usual
flexible
strategies)
Literature review, identify key concepts
Multiple sources evidence
Establish chain of evidence
Have key informants review draft (member
checking)
Research design
Data collection
Data collection
Write up
Credibility
(the usual
flexible
strategies)
Do pattern matching
Do explanation building
Do rival explanations
Use logic models
Triangulation
Negative cases
Data analysis
Transferability Use theory in single case studies
Use replication logic in multiple case studies
Research design
Dependability Use case study protocol
Develop case study data base (audit trail)
Data collection
Interpretive analysis
in HPSR
• Remember the question/s you are asking
• Analysis begins during data collection
– looking for patterns and possible explanations
• Always need to reduce and display data
– narratives, examples of broader phenomena,
diagrams, figures, pictures, tables
• Ask questions of the data:
– Is explanation plausible? Can you find
evidence confirming it? Can a finding be
replicated in another data set?
Be reflective!
• Contextualise in analysis
– think about and present relevant features of
the specific social and physical setting,
including historical factors, that support
explanation: identify from theory and in
analysis
Role of theory in ensuring
HPSR rigour?
To support generalisable claims:
• Testing hypotheses generated from theory
with empirical evidence
• Triangulation with theory
• Generating theory as a generalisable claim
Four key steps in HPSR
1. Identify research focus
(problem/concern/ opportunity) and
question
2. Design study
3. Ensure quality and rigour
4. Apply ethical principles
Principles of bio-medical
research ethics
• Are these relevant to HPSR?
How?/Why not?
– Scientific validity and collaborative
partnerships
– Social value and risk benefit ratios
– Informed consent and respect for
participants and communities
– Independent reviewEmanuel et al. 2004
What practices of real world research are
(most) important in HPSR and why? Robson, 2002
• Involving people without consent
• Coercing them to participate
• Withholding information about true nature of research
• Otherwise deceiving participants
• Inducing participants to commit acts diminishing of their self-
esteem
• Violating rights of self-determination
• Exposing participants to physical or mental stress
• Invading privacy
• Withholding benefits from some participants
• Not treating participants fairly or with respect
Examples of HPSR ethical
challenges
• Being informed about or observing unethical
behaviour of providers, managers, community
members
• How to present findings in ways that balance
confidentiality and give voice to those not usually
heard
• How to act in ways that have potential to stimulate
change in (rather than exacerbate) unequal
relationships between people
Other HPSR issues
• Who determines the questions?
• Gaining consent and relative power:
– from policy makers to marginalised groups
• Anonymity and small sample sizes
• Acting ethically towards field workers
• Recognising researchers’ privilege and power
• Generating social value: From substantive relevance
to where? Where does the researcher’s role end?
Copyright
Funding
You are free:
To Share – to copy, distribute and transmit the work
To Remix – to adapt the work
Under the following conditions:
Attribution You must attribute the work in the manner
specified by the author or licensor (but not in any way that
suggests that they endorse you or your use of the work).
Non-commercial You may not use this work for commercial
purposes.
Share Alike If you alter, transform, or build upon this work,
you may distribute the resulting work but only under the same
or similar license to this one.
Other conditionsFor any reuse or distribution, you must make clear to
others the license terms of this work.
Nothing in this license impairs or restricts the authors’
moral rights.
Nothing in this license impairs or restricts the rights of
authors whose work is referenced in this document.
Cited works used in this document must be cited following
usual academic conventions.
Citation of this work must follow normal academic
conventions. Suggested citation:
Introduction to Health Policy and Systems Research,
course presentation, Presentation 8. Copyright
CHEPSAA (Consortium for Health Policy & Systems
Analysis in Africa) 2014, www.hpsa-africa.org
www.slideshare.net/hpsa_africa
This document is an output from a project funded by the European Commission (EC) FP7-Africa (Grant no.
265482). The views expressed are not necessarily those of the EC.
The CHEPSAA partners
University of Dar Es SalaamInstitute of Development Studies
University of the WitwatersrandCentre for Health Policy
University of GhanaSchool of Public Health, Department of Health Policy, Planning and Management
University of LeedsNuffield Centre for International Health and Development
University of Nigeria Enugu Health Policy Research Group & the Department of Health Administration and Management
London School of Hygiene and Tropical MedicineHealth Economics and Systems Analysis Group, Depart of Global Health & Dev.
Great Lakes University of KisumuTropical Institute of Community Health and Development
Karolinska InstitutetHealth Systems and Policy Group, Department of Public Health Sciences
University of Cape TownHealth Policy and Systems Programme, Health Economics Unit
Swiss Tropical and Public Health InstituteHealth Systems Research Group
University of the Western CapeSchool of Public Health