Rigour and Robustnessin Research
Dr Jennifer LokePHD; MBA; MSC; PGDIP; BN; ADVDIP,
SFHEA; LPE; RN
© J. Loke 2014
Aims
• To gain insight to the meaning of rigour and its importance for robustness in all research.
• To gain understanding on how rigour can be maintained to achieve robustness in the different stages of the research process of the different research approaches (paradigms).
• To appreciate the strategies to achieve rigour in the different stages of research.
© J. Loke 2014
Learning Outcomes
At the end of the session the student will be able to: o understand the meaning and the importance of rigour in
research;o understand how rigour is judged in the different research
paradigms;o describe ways to achieving rigour in research studies;o identify some of the common pitfalls in the process of
achieving rigour in research;o highlight research knowledge and skills as appropriate to
address these pitfalls.
© J. Loke 2014
By John Stevens PUBLISHED:MailOnline 14:03, 20 June 2012 | UPDATED: 12:27, 21 June 2012
One McBurger with lies, please: Fast-food firm's own video reveals why items on the menu never look like what you are served
© J. Loke 2014
Still?!!
Research paradigms
Quantitative designs Qualitative approachesMixed methodology
approaches
LOKE, J. Jan2012
Located in a continuum
The goal of Quantitative research
© J. Loke 2014
Generally, to quantify the problem and understand how prevalent it is by looking
for projectable results to a larger population
The goal of Qualitative research
© J. Loke 2014
Generally, to define the problem or develop an approach to the problem.
Aim to contribute to an ongoing dialogue, in the form of debate, discussion or
argument about a phenomenon
Rigour in Qualitative Research – drawing parallels with quantitative research
ReliabilityValidity
CredibilityTrustworthiness
© J. Loke 2014
Validity
Dependability
Reliability
Strategies to achieve rigour and robustness
Research approach/paradigm• Methodological approach• Sampling techniques• Data collection • Data analysis
Strategies to achieve rigour and robustness
Research approach/paradigm• Professional/Personal bias– Implicitly expressed (quantitative research) – Explicitly expressed (qualitative research)
Rigour through proper selection of research paradigm/methodological approaches
• Personal/Professional Bias – prevent highlighting some issues while obscuring others?
• Address ethical aspects of research – which sometimes can be compromised due to political reasons
Reflexivity – qualitative research
© J. Loke 2014
Research methodologies
Randomised control Trial
(RCT)
Quasi RCT (cluster/group
randomisation
Survey-based approach
Discourse analytic work:
Corpus analytic work
Grounded theory
Discourse analytic work
Life history
Semiotic analytic work
Ethnography
Ethno-methodology
Phenomenology
Quantitative
in nature
Qualitative
in nature
LOKE, J. Jan2012
Selecting an appropriate research design
• Quantitative• Qualitative
– Triangulation of theoretical framework
• Mixed-Methods - approach
Credibility
Theoretical framework 1
Theoretical framework 2 Data Sources
© J. Loke 2014
Strategies to achieve rigour and robustness
Sampling Techniques
Participants selection
Document selection
Event selection
Representativeness
Strategies to achieve rigour and robustness
Data collection
Participants selection
Document selection
Event selection
what, where, when and how
Data collection in research
© J. Loke 2014
Common techniques for producing data
Indirect Observations
Direct Observations Elicitation Techniques
Mixed Methods
Continuous or Spot
monitoring
Scientific tools, Unstructured, Semi-
Structured, Structured, Mixed
Case studies, Participants
Observations
What Technique?
Strategies to achieve rigour
© J. Loke 2014
• Knowledge and experience of data collector
• Appropriateness of the collection methods for the research objectives and settings
• Relationship between data collector and data (participants/ texts/historical documents)
• Addressing ethical issues
Consider the different quantitative and qualitative techniques in Data Collection
• Data Collection
Reliability
Data Source 1
Data Source 3
Data Source 2
Credibility
Observer 2
Observer 3
Observer 1
Trustworthiness
Observer
Participant/member checking
Data source
© J. Loke 2014
Where/When/How
Key qualitative and quantitative distinctions
DATA
Analysis Quantitative Qualitative
Quantitative Statistical and mathematical analysis of the numerical
data as presented
Assigning meaning of words based on scoring system
E.g. Content analysis, word counts, free lists, pile sorts
Qualitative Search for and presentation of meaning in results of quantitative processing
Interpretative test studiesE.g. Hermeneutics, Grounded Theory, Phenomenology
© J. Loke 2014
Consider the different quantitative and qualitative techniques in Data Analysis
• Appropriate types of statistical tests for different types of quantitative data
• Appropriate level of statistical tests based on the size of data
• Appropriate analytic framework for the different theoretical framework
• Triangulation of analytic framework
Reliability
Data Source 1
Data Source 3
Data Source 2
© J. Loke 2014
Rigour for Robustness
Analyses involve attending to inconsistencies and diversity
Present the richness of detail in data, researcher’s assumptions must be clarified upfront and context of the study must be
explained
Validation through reference to coherence and fruitfulness of findings, participants’ orientation and
to new research problems
Seek out negative/deviant cases
© J. Loke 2014
Rigour through reporting on findings and making conclusions
• Personal/Professional Bias – prevent highlighting some issues while obscuring others?
• Address ethical aspects of research – which sometimes can be compromised due to political reasons
Reflexivity – qualitative research
© J. Loke 2014
Ways to address rigour
• Consider the quantitative and qualitative techniques in:
a. Data sources/collectionb. Data analysis
© J. Loke 2014
Selecting an appropriate research design
*Research Question
Further readings• http://www.economist.com/news/briefing/21588057-scientists-think-science-self-correcting-
alarming-degree-it-not-trouble• http://cogprints.org/7691/7/ICMLws09.pdf• http://www.sonoma.edu/users/k/koshar/n300/rigor_table.htm• Tobin, G. A., Begley, C. M. 2004. Methodological rigour within a qualitative framework.
Journal of Advanced Nursing. 48(4), 388-96.
© J. Loke 2014