Upload
alvin-henderson
View
218
Download
1
Tags:
Embed Size (px)
Citation preview
Objectives
To identify the advantages and disadvantages of employing multiple research methods;
To consider those factors that influence the choice of research method and sequence of the multiple methods;
To understand triangulation and its variations; To examine the management and analysis of
data in the context of multi method research; To consider the need for clear planning and
managing a mixed methods approach
Methods, Data Analyses, Data Displays & Stories
Text of the Story
Narrative & discourse analysis
Systemic networks, category systems, & tree structures
Multi-way Qualitative Matrix Displays (with or without pattern coding)
Diagrams, maps, models and other pictorial summaries
Statistical summaries & tables
Graphs & figures
Hybrid devices, including dynamic systems modeling & Monte Carlo simulations
Interviews Observations Rich pictures, mind maps, cause
maps, concept maps, cognitive maps Surveys/questionnaires/scales Stories & metaphors Drawings, pictures, photographs, videos,
films Documents and other traces Physical measures & traces Experiments/quasi-experiments Focus groups Diaries, journals, and other self-monitoring
techniques Structured analytical tools & templates Think aloud protocols & other process
tracing methods Simulation models & ‘flight’ simulators
(varying in degree of structure, depth, & intrusiveness)}
...
.. .
.
Text of the Story
Narrative & discourse analysis
Systemic networks, category systems, & tree structures
Multi-way Qualitative Matrix Displays (with or without pattern coding)
Diagrams, maps, models and other pictorial summaries
Statistical summaries & tables
Graphs & figures
Hybrid devices, including dynamic systems modeling & Monte Carlo simulations
Interviews Observations Rich pictures, mind maps, cause
maps, concept maps, cognitive maps Surveys/questionnaires/scales Stories & metaphors Drawings, pictures, photographs, videos,
films Documents and other traces Physical measures & traces Experiments/quasi-experiments Focus groups Diaries, journals, and other self-monitoring
techniques Structured analytical tools & templates Think aloud protocols & other process
tracing methods Simulation models & ‘flight’ simulators
(varying in degree of structure, depth, & intrusiveness)}
...
.. .
.
......
.. .
..
. ..
GOAL: Achieve Desired Balance to tell Different Types of Stories• Theme/category-based• Time-based• Event-based• Perception-based• Mental model-based• Role-based• Cognition-based• Cohort-based• Group-based• Intervention-based• Hypothesis-based• Pattern-based• Impact-based• Change-based• Relationship-based• Cross-case• Generalising• Data integration• Hybrids of above• Emergent stories . . .
•Interaction-based strategiesStructured, Semi-structured, Unstructured interviews; Focus groups
•Participant-centred strategiesMapping & drawing; Storytelling; Self-recording
•Observation-based strategiesParticipant observation; Systematic observation; Unobtrusive observation
•Document-based strategiesTextual evidence gathering; Multimedia evidence gathering; Meta-analysis; Secondary database methods
•Experience-structuring strategiesExperiments/Quasi-experiments; Organising frameworks; Projective techniques; Process-tracing methods; Simulations & games
•Measurement strategiesObjective tests & assessments; Self-report instruments; Physical measurements
STRATEGICCHOICES
Advantages Each method can enhance the unique qualities of the other and
result in vast opportunities for reciprocal advantages Multi-method research overcomes the weaknesses of individual
approaches, whereby one method’s strength may compensate for the other method’s weakness
It appreciates the complexity of a research problem Multi-method research reduces bias It might increase response rate It elicits more detailed information It enhances theory building It enhances hypothesis testing It enhances generalisations It enhances validity It enhances accuracy of perceptions It might bridge macro and micro levels of social reality Multi-method research exposes the researcher to different ways
of thinking (and behaving)
Disadvantages
Multi-method research requires more time in planning and analysing the data
It requires more effort in planning and analysing the data
It requires more sensitivity because researchers have to synthesise the techniques to achieve synergy among them
Merging approaches
Concurrent – convergent parallel design – triangulation models A two-phase design (qual and quant) A dominant / less-dominant design (embedded)
Sequential – exploratory design Emphasis on reflective learning and contextualised
understanding Emphasis on early learning providing feedforward
Sequential – transformative Quant – qual – action for change
Sequential – multiphase –discrete yet linked studies Study 1 informs Study 2 informs Study 3 (mixed methods)
Mixed Methods Approaches - Prototypes
[Adapted from: Creswell, J.W. & Plano Clark, V.L, (2011). Designing and conducting mixed methods research (2nd Ed.). Thousand Oaks, CA: Sage Publications, pp. 69-70.]
Quantitative Data
Collection and Analysis
Qualitative Data
Collection and Analysis
InterpretationCompare or relate
(a) Concurrent - Convergent parallel design ~ triangulation models
Quantitative Data
Collection and Analysis
Qualitative Data
Collection and Analysis
InterpretationFollow up with
(c) Sequential – Explanatory design – emphasis on reflective learning &contextualised understanding (d) Sequential – Exploratory design ~ emphasis on early learning providing feedfoward
Qualitative Data
Collection and Analysis
Quantitative Data
Collection and Analysis
InterpretationBuilds to
Quantitative (or Qualitative Design
Quantitative (or Qualitative)Data Collection and Analysis
Qualitative (or Quantitative) Data Collection and Analysis
(before, during, or after)
Interpretation
(b) Concurrent - Embedded design w/ differential emphasis on data types
(f) Sequential - Multiphase design ~ discrete, self-contained yet linked studies
Study 1:Qualitative
InformsOverallProgramObjective
Study 2:Quantitative
Study 3:Mixed Methods
Informs
(e) Sequential - Transformative design ~ guided by an over-arching framework for achieving change/improvement; may have implied or explicit feedback loops or learning cycles, as in action research
Quantitative Data Collection and
Analysis
Qualitative Data Collection and
AnalysisInterpretationFollow up with
Transformative framework
Actions for change
Sequence selection? The research topic
What does your discipline suggest about the order of data collection methods?
The research question Is there a validated research tool that will answer your question?
Timing Is there a holiday period that will impact your ability to recruit
participants? Should you interview or utilise your survey first? You should probably
administer the survey first and follow up with interviews to avoid biasing the responses of research participants to the survey.
The context within which the research will take place Where will you be conducting the research?
What else will be going on in your life at that time? The resources available Do you need money to conduct the research?
Do you need access to particular computer software packages?
Triangulation
Triangulation is the use of several different research techniques in the same study to confirm and verify data gathered in different ways.
Triangulation adds rigour, richness and depth to the design and to the data.
Types of triangulation
Source triangulation Investigator triangulation Theory triangulation Methodological triangulation Recruitment triangulation Data collection and analysis triangulation Research team triangulation Interdisciplinary triangulation
Developing a Mixed Methods Mindmap
[Adapted from: Cooksey, R.W. & McDonald, G. (2011). Surviving and thriving in postgraduate research. Prahran, Vic: Tilde University Press, p. 346.].
Research quality
Is the research story, as a whole, convincing with respect to the arguments being made? Context
Juxtapositioning with other research Contextual knowledge Researcher positioning
Realization Internal coherence Analytical integrity Extensional reasoning – implications to other contexts
Explication Handling of unexpected outcomes Value for learning Fertilization of new ideas Acknowledgement of limitations Presentation to audience
What have other researchers done and where does this
research fit in?
How well does the researcher use knowledge about context to add richness
and/or qualification to their research process and the resulting findings?
Where & how does everyone or everything else fit within the
research context?
Where & how does the researcher fit within the
research context?
Does the research, as a whole, hang together as a coherent process to
permit the conclusions the researcher seeks or claims?
Do the analyses lead to or support appropriate, defensible & clear conclusions
given the quality of data to hand?
Do the research findings have meanings or implications for
other contexts?
How well has the researcher dealt with surprises and unanticipated findings?
What can consumers/ readers/users take away from
the research as important messages or lessons?
Can others run with or build upon what has been shown by or learned from the research?
What constrains the learning value, implications and
applicability of the research?
Is the presentation of the research suitable for the
intended audience(s) (i.e., is the researcher hitting the mark)?
Is the research story, as an integrated whole,
convincing with respect to the arguments being made?
[adapted from Cooksey, R.W. (2008). Paradigm-independent meta-criteria for social & behavioural research. Proceedings of the 2nd Annual Postgraduate Research Conference, University of New England, Armidale, NSW, pp. 4-17.]
Some Useful Mixed Methods References
Bergmann, M.M. (ed.) (2008). Advances in mixed methods research: Theories and applications. London: Sage Publications.
Brocklesby, J. (1997). Becoming multimethodology literate: An assessment of the cognitive difficulties of working across paradigm. In J. Mingers & A. Gill (Eds.), Multimethodology: The theory and practice of combining management science methodologies. Chichester, UK: John Wiley & Sons, 189-216.
Cooksey, RW & McDonald, G. (2011). Surviving and thriving in postgraduate research. Prahran, VIC: Tilde University Press.
Creswell, J.W. (2003). Research design: Qualitative, quantitative and mixed methods approaches,(2nd Ed.). Thousand Oaks, CA: Sage Publications.
Creswell, J.W. & Plano Clark, V.L. (2011) Designing and conducting mixed methods research (2nd Ed.). Thousand Oaks, CA: Sage Publications.
International Journal of Mixed Methods in Applied Business & Policy Research, online journal, Sage International Journal of Multiple Research Approaches, e-Content Management. Journal of Mixed Methods Research, Sage. Morse, J.M. & Niehaus, L. (2009). Mixed method design: Principles and procedures. Walnut Creek, CA: Left
Coast Press. Plano Clark, V.L. & Creswell, J.W. (2008). The mixed methods reader. Thousand Oaks, CA: Sage Publications. Tashakkori, A. & Teddlie, C. (Eds). (2003). Handbook of mixed methods in social & behavioral research.
Thousand Oaks, CA: Sage Publications. Teddlie, C. & Tashakkori, A. (2009). Foundations of mixed methods research: integrating quantitative and
qualitative approaches in the social and behavioral sciences. Thousand Oaks, CA: Sage Publications.