Designing a Qualitative StudyProfessor Karen O’Reilly

Loughborough University

The broad nature of qualitative

research

The broad nature of qualitative

research (1)

Qualitative research is a research strategy that usually emphasizes words rather than quantification in the collection and analysis of data. As a research strategy it is broadly inductivist, constructionist, and interpretivist, but qualitative researchers do not always subscribe to all three of these features. (Bryman 2012:380)

The broad nature of qualitative

research (2)

qualitative methods entail and manifest the assumptions of the interpretive paradigm, the grounds of which lie in the need to grasp the meaning of social action in the context of the life-world and from the actors' perspective. (Vasilachs de Gialdino2009)

The broad nature of qualitative

research (3)

attempts to capture or study data which are detailed, rich and complex

concerned with meanings

uses flexible, non standardised methods

data mainly in form of words / ideas / themes / patterns / processes rather than numbers

sets data in context

The broad nature of qualitative

research (4) mainly ‘inductive’ rather than ‘deductive’ analytical

process

emergent categories and theoretical ideas in addition to / rather than a priori concepts

committed to retaining diversity and complexity in analysis

respect for uniqueness of individual cases as well as comparative themes and patterns

explanations at the level of meaning or in terms of intentions, norms, opportunities, barriers or micro-social processes rather than context free causal laws

Qualitative Research

ParadigmsA closer look at approaches and strategies

Positivism

“The application of empiricist natural science to the study of society and the development of policy”. (O’Reilly 2012a)

Empiricist

Emulates (a view of) natural science

Seeks positive outcomes

Helps us understand what some researchers are trying to achieve in their work (Comte, Malinowski, and some quantitative research)

Interpretivism

“For interpretivists it is essential to see humans as actors in the social world rather than as simply reacting as objects in the natural world. Some interpretivists argue, furthermore, that human behaviour needs to be understood in the context of their particular society or culture” (O’Reilly 2012a)

The interpretive tradition

Max Weber and interpretive understanding (verstehen)

Alfred Shutz and Phenomenology (the actor’s point of

view, understanding how people make sense of their world using their senses)

Hermeneutics (understanding the condition under which text

was produced, also understanding groups within and across cultures, merging horizons )

Symbolic Interaction, social constructionism, and relativism

Realism

“A realist philosophy of social science argues that what exists in the world is not only human beings’ conceptualisations and interpretations of events, but also wider processes that the individual might not even be aware of” (O’Reilly 2012a)

Realism and reflexivity

Ontological and

epistemological

assumptions

Naïve realism

External independent

reality, directly

accessible

Subtle realism

Reality exists but

mediated and only

accessible through

human constructions

Radical idealism

No shared reality, only

alternative human

constructions – only ideas

are real

Relationship

between

researcher and

subject

Reality is unaffected

by the research/

researcher

Research/researcher

has an impact –

reflexivity is required

Reality inseparable from

the researcher

Relationship

between facts

and values

Research can be free

of values

Neutrality a guiding

ideal; assumptions

documented

Value-free research

impossible; subjectivity

celebrated

Nature of

knowledge

Foundational/absolute:

direct correspondence

with reality

Fallibilistic/ tentative:

provisional accounts

as faithful as possible

Sceptical/relative: no

‘privileged’ or accurate

accounts

Methods Rigour linked to type of

method, standardized

methods viewed as

more scientific

Rigour a question of

transparency and

reflexivity rather than

standardisation

Tacit process of intuition,

impossible to explicate: no

definitive findings, just

qualitative narratives

Critical approaches

Critical theory a natural science approach does not permit a critical or

positioned approach The goal of social science is, through dialogue or

argument, to free ourselves from oppression or dominations

This involves making judgments

Feminist standpoint See Rubin and Rubin (1995)

Participatory and action research Works together with research participants to change

the world

Practice theory

People make societies but not in circumstances of their own choosing.

This is affected by how they think and feel

And social structures are the outcomes of social interactions

(see O’Reilly 2012b)

Workshop

The nature of qualitative research – what it can and cannot deliver

List the main qualities of quantitative research

List for each why they are not appropriate for qualitative study

For discussion – what are the main things a qualitative study can and cannot deliver

Developing your design Five difficult questions to get you started

Five Difficult Questions

1. What is the nature of the phenomena or entities, or social ‘reality’, which I wish to investigate? (Ontology)

2. What might represent knowledge or evidence of the entities or social ‘reality’ which I wish to investigate? (Epistemology)

3. What topic, or broad substantive area, is the research concerned with?

4. What is the intellectual puzzle? What do I wish to explain? What are my research questions?

5. What is the purpose of my research? What am I doing it for?

(Mason 1996, Chapter 2)

What is the nature of the phenomena or entities, or

social ‘reality’, which I wish to investigate?

People, social actors Minds, psyches Rationality, emotions Understandings, beliefs Self, individuals Texts, stories, narratives Experiences Languages Interactions Processes Institutions, the ‘material’,

Underlying mechanisms Perspectives One reality Nature, genes, Haphazardness Rules, norms, morality Interactions Social constructions

(adapted from Mason 1996)

What might represent knowledge or evidence of

the entities or social ‘reality’ which I wish to

investigate? (question 2)

Epistemology is the study or a theory of how things can be known, a theory of knowledge.

It is intrinsically linked to ontology

Ask how the things you are interested in can be learned about or known

Ask what would count as evidence or knowledge of the things you are interested in

What topic, or broad substantive area, is the research

concerned with?

(question 3)

This is about how your question is framed in relation to your ontology and epistemology

Eg. A study of racist attitudes among school children

Suggests we think racism affects society, that attitudes are a reality, and they can be known

But are we interested in discourses, institutional rules, actions, or feelings?

(see Mason 1996)

Potential topic areas

You would like a good understanding of the topic

The topic is personal / sensitive / complex

You need detailed information about people’s views

and experiences

You need to explore meanings and processes

You want to know what is going on behind the official

view / what happens in practice

You want understanding and explanations rather than

numbers or statistical generalizations

You are trying to included ‘hard to reach’ groups

What is the intellectual puzzle? What do I wish to

explain? What are my research questions?

(Question 4)

This is about the broad question you wish to address

It will lead to numerous other, research, questions

Research questions form the backbone of research

Qualitative researchers tend to use guiding theoretical questions rather than hypotheses

Research questions are of different levels and will develop as you go along

Potential puzzles

illustrative/definitional/contextual: to explore and portray in detail the form and nature of a

phenomenon; the range of meanings, beliefs, experiences; the structure of groups or settings; what it is like to………

explanatory: to examine the reasons for patterns and associations –

why people believe or behave as they do …….

evaluative: to appraise a phenomenon – what it is like to deliver or

be at the receiving end of something

generative: to aid the development of initiatives, strategies, policies,

theories

What is the purpose of my research? What am I

doing it for?

(question 5)

This is about

academic knowledge

substantive knowledge

policy debates

and what contribution you can make

in what socio-political context

Workshop

Designing Qualitative Research Questions

Select an imaginary topic

How do you address the five difficult questions?

What is your intellectual puzzle?

The purpose of this exercise is to do the groundwork that leads to the development of Qualitative Research Questions

Aims and Objectives Aims: a broad statements

desired outcomes, general intentions of the research, What not how long-term project outcomes

Objectives: a list steps to take or specific tasks how aims are to be accomplished focused and feasible address the more immediate project outcomes make accurate use of concepts and be sensible and precisely

described are usually numbered so that each objective reads as an

'individual' statement to convey your intention

The study aimed to explore how children, as social actors, perceive and experience the internal, spatial characteristics of different hospital environments. Four objectives were identified: - To document the physical and social characteristics of the spaces provided for children within a range of different hospital settings. - To explore children’s own experience and use of these different internal spaces and the meanings they attribute to them. - To develop strategies that enable children’s needs to be considered alongside those of adults in the planning and utilisation of internal spaces in hospital. - To develop a set of child-centred research tools that will contribute to theoretical and methodological developments within childhood studies.

Curtis, Penny (2007). Space to care : children's perceptions of spatial aspects of hospitals: Full Research Report. ESRC End of Award Report, RES-000-23-0765. Swindon: ESRC

Example study: SPACE TO CARE

Reviewing the literatureSubstance, policy, and theory

Lit review

Substantive review What is already known about this topic?

Scientific work should be cumulative

Policy review How is this topic viewed or shaped by policy?

Theoretical review How has this area been understood?

Leads to sensitising concepts rather than hypotheses

A word on inductivism

A deductive approach to research is one where a hypothesis is derived from existing theory and the empirical world is explored, and data collected, in order to test the truth or falsity of the hypothesis.

A simplistically inductive approach to research is one where the researcher begins with as open a mind and as few preconceptions as possible, allowing theory to emerge from the data.

A more sophisticated inductivism views theory as precursor, medium and outcome of ethnographic study and writing.

Key Idea

Qualitative research is usually iterative-inductive

This is a practice of doing research, informed by a sophisticated inductivism, in which data collection, analysis, and writing are not discrete phases but inextricably linked.

Inductivism and The role of theory

Theory before research?

Theory after research?

Sensitising concepts (Charmaz 2006)

Foreshadowed problems (Malinowski 1922)

Guiding theoretical problems (O’Reilly 2012a) These are inspired by (perhaps lay) theoretical

ideas about the world, and are not rigid or fixed, but can be adapted or discarded as research progresses.

A note on Grounded theory

Linking Research Questions,

Methodologies and MethodsSelecting your methods

Interview styles

Semi-structured interviews (enable some guidance, some standard questions, use a topic guide,

can be better for some types of people or topic)

Unstructured (you are unlikely to be completely unstructured, but you may just

have an event or time period or career you want people to describe in detail, you are still likely to want prompts)

Narrative (these understand accounts as story-telling, narratives are seen as

a way of making sense of past events and explaining them to others)

Life history (these relate the topic to the life or biography of the participant)

What sorts of data? Interviews

How people feel, express themselves

Their understandings and meanings

Their interpretations

Taps into aspects you might not have thought of

Enables some discussion of context (vignettes)

Gives people a voice

Participant Observation

A word on ethnography

Ethnography is a practice that evolves in design as the study progresses; involves direct and sustained contact with human beings, in the context of their daily lives, over a prolonged period of time; draws on a family of methods, usually including participant observation and conversation; respects the complexity of the social world; and therefore tells rich, sensitive, and credible stories. (O’Reilly 2012a)

Participant Observation is the main method of ethnography and involves taking part as a member of a community while making mental and then written, theoretically-informed observations.

What sort of data? Participant

Observation

The difference between what people do and what they say they do

The opportunity to ask questions in context

Taps into complexity (ambiguity, and ambivalence)

Enables you to build rapport and trust

Enables you to go back and check

Taps into what people cannot easily express

Can explore how understandings are shaped by communities

Can examine how norms are practiced

Focus groups and group interviews

Market Research Focus Group

typically involve 4 to 12 participants

People selected for relation to topic

Usually strangers

Moderators retain control

Conducted in a series

Institutional setting

Participants may be paid

Qualitative Focus group

any number, depending on the situation, small is more manageable

Naturally-occurring group

Usually not strangers

learn from interaction

Often one-off or may repeat with same people

Familiar setting

collaborative, ethical, and based on trust and rapport.

What sorts of data? Groups

Useful at exploratory stage Lots of information quickly Explores interaction People can feel safer or more comfortable Can make space for people who otherwise feel

overpowered But – some people may feel threatened Can observe how meaning is made in groups Can observe power play Acknowledges the creative nature of social life

Documents and Texts

Types of data

Diaries, letters, photos, other personal documents

Official documents

Mass media texts and documents

Virtual

Interpretive analysis

Creative methods

Arts-based methods Drawing Painting Making something Web diagrams Taking photographs Going for walks or even dance

Aim is usually to use them as prompts, to encourage response Morrow, V. (2001). Using qualitative methods to elicit young

people’s perspectives on their environments: some ideas for community health initiatives. Health Education Research, 16 (3): 255-268.http://her.oxfordjournals.org/content/16/3/255.abstract

What sorts of data? Creative and

Texts

Data as writing (usually refers to visual data, that you

produce, perhaps collaboratively, to support your case)

Found data (that you analyse, probably interpretively,

but maybe using discourse analysis or semiotics)

Creative uses (where data are produced

collaboratively to create meaning or used as prompts to encourage talk about meanings and feelings)

Collecting data on social structures

How to access norms, rules, procedures, wider context

Key informant interviews

Web sites

Documents

Policies

General gathering (desk-based work)

Sampling and selecting Designing a sample

Designing the sample (1)

qualitative research aims to map phenomena,

provide detailed illustration and examples,

identify processes, offer explanations, NOT

generate numbers or statistics

qualitative samples are usually

based on purposive logic

intended to be ‘information rich’

small to facilitate in-depth exploration

Designing the sample (2) :

Probability samples

units chosen randomly

all units have known chance of selection

sample is assumed to display characteristics in similar proportions to population as a whole

generalisation based on statistical probability

not appropriate for qualitative research

Designing the sample (3):

Convenience samples (ad hoc)

units chosen for ease of access

relationship to wider population is unknown / unspecified

should have justifiable rationale - not the default option for qualitative research

reasons for using ad hoc sampling

in observational studies when need to take opportunities as they present themselves

for hard to reach, ‘invisible’ populations (eg tax evaders, sex workers – use known groups or snowball sampling)

if purpose of exercise is participatory / democratic consultation (anyone who wants to can have a say)

if there are statutory requirements about consultees

Selections should always be made on the basis of their relevance for the study

A note on the intrinsic case study

Some situations, groups, institutions, people are studied because they are intrinsically interesting.

This may even be because the researcher has exclusive or privileged access.

EG. A study of decision-making practices in the British Government, or a study of citizenship education in an outstanding school

(see Stake 2003)

Designing the sample (4)

Purposive samples

units chosen to be ‘information rich’

linked to the wider population in terms of conceptual significance heterogeneous / maximum variation sampling

(broad cross-section)

comparative understanding

homogeneous sampling (specific sub-group/s)

in-depth understanding

atypical sampling - extreme case / deviant / vanguard

understanding from outliers

Designing the sample (5)

Theoretical samples

units chosen to help build and ‘test’ grounded theory

iterative process of selecting sample

collecting and analysing data

devising concepts

drawing further samples ……… to refine concepts and identify links between them

from ‘open’ (broad) to ‘discriminate’ (strategic) selection

Sampling units

people (individuals, groups)

organisations

settings (locations, places)

events (routine, special)

time frames, periods

documents

Eg. children of certain ages, people with specific disabilities

Political institutions, charities

Religious festivals, staff meetings

Day, evening, after work

Adverts, displays, news

Designing a qualitative sample

review aims / research question

devise appropriate sampling rationale

ad hoc, purposive, theoretical

heterogeneous / homogeneous / atypical (purposive)

identify target population(s)

set selection criteria

relevant socio-demographic characteristics

relationship to the research issue

Primary and secondary criteria

long list of selection criteria can lead to unmanageable sample design and size

simplify list by sorting into primary (most important) criteria secondary (less important) criteria

explore ways in which secondary criteria might be nested under primary criteria / primary might proxy for secondary (eg a range of occupations should also include a range of income levels)

Setting quotas

how many cases for each criterion

symbolic not statistical ‘representation’

quotas based on requisite range and

diversity not distribution in wider population

Sample size – how many is enough?

no standard rules on sample size

size determined by and result of diversity of target population

number of selection criteria

not probability theory or confidence levels

some guidelines in-depth interview samples often below 50, would

question rationale if more than 70

focus group samples often below 12 groups, would question rationale if more than 20 groups

Reasons for small samples in qualitative

research

feasibility

cost

time

handling data collection and analysis

‘saturation’

mapping range not incidence

already got key concepts / themes etc

diminishing returns

Making generalizations

The language of sampling suggests we wish to represent a wider group or phenomenon

But this may not be the case, except we usually aim to learn even from an intrinsic case

Generalizations are often made by sleight of hand. It is better to be clear about your aims and to write them into your design

Types of generalization

Jonesville-in-the-USA

Banal generalization

Transferability

Theoretical or conceptual generalizing

Intentional modest generalizing

Workshop

Design a sample for one aspect of your project.

This will entail considering appropriate methods

Consider a hierarchy of criteria

Give quotas and justifications Let’s examine our case study by Penny Curtis

(handout)

Also see KoR project handout

Tools for participants

Topic guides

These guide the interview for the interviewer, but might also be helpful for the interviewee

The first step is to note all the topics you wish to cover, and their elements

Consider a hierarchy of questions

Vignettes and Photos can be useful for elicitation

Types of question (see Bryman 2012)

Introducing questions

Follow-up questions

General statements

Probes

Specifying

Indirect

Flow statements

Silences

Ending

Participant information sheets

These are crucial for informed consent

You may need to produce slightly different versions

They should be professional

They will shape your interviews, so be sure they are enticing

Include: aims, general question Extent and nature of

participation Inclusion/exclusion

criteria (who the participants are)

Any risks or dangers Benefits Clarify privacy and

confidentiality Contact details and

affiliations

Visual aids

Consider using other things in an interview Photos the participants have taken/displayed

previously Photos you take along Photos you ask them to take in advance, with the

topic in mind Objects in the home Ask them to draw maps, or even play with lego Include in the design some photos that inspire or

explain the topic, but be careful not to lead or foreclose

The Quality of qualitative

research

Quality questions (1)

CONTRIBUTION - Is the study contributing to wider knowledge and understanding or has it had some utility within the original context?

Understanding How far has knowledge and understanding been

extended by the research? In what ways?

Worthwhileness / status of participants How far has the status of the participants been

enhanced?

Transferability How clear is the basis for drawing wider inference

(discussed more later)

Spencer, L., Ritchie, J., Lewis, J., and Dillon, L. 2003, Quality in Qualitative Evaluation: A Framework for Assessing Research Evidence, GCSRO.

Available at: www.policyhub.gov.uk/publications

Quality questions (2)

CREDIBILITY

Are findings are believable (face validity)?

How well does the study capture and portray the world it is trying to describe?

How well backed up are the claims made by the research / what is the evidential base?

How plausible are the findings? How well do claims fit within existing knowledge?

If not initially plausible, how strongly persuasive is the argument?

Quality questions (3)

RIGOUR

Transparency How clearly have the assumptions / theoretical

perspectives / values of the researcher(s) been conveyed

How clearly have the ethical implications of the research been addressed?

How well have decisions about / approaches to design, conduct and analysis been documented? How detailed is the account?

How clear is the rationale for key decisions?

Quality questions (4)

RIGOUR

Is the Research design appropriate and thorough?

How appropriate is the overall strategy for the study?

Has a convincing rationale for qualitative research been given?

How well can qualitative methods address the research aims / research question(s)?

Quality questions (5)

RIGOUR

How convincing / defensible is the sample design?

Why has a particular sampling rationale been adopted?

Why have particular sampling units been chosen (people, groups, organisations, locations, documents, accounts etc)?

Why has a particular sample size been chosen?

Are conclusions justifiable given sample?

Quality questions (6)

RIGOUR

How appropriate are the methods chosen? How well can the methods chosen address the

research aims?

How appropriate are the methods for the kind of people / documents included in the study?

How thorough was the data generation / collection? How far have depth, richness and nuance been

captured?

Quality questions (7)

RIGOUR

How thoroughly and systematically have data been analysed?

How well do analytical themes / concepts/ building blocks fit the data?

How well has the basis of categories and classifications been described and illustrated?

How well has the variation around these themes been identified and retained?

Drawing wider inference from qualitative

data – extrapolation?

Is it possible? NO, if we mean statistical generalisation

NO, if we believe that meanings and behaviours are context-bound

YES, if extrapolation is offered as a working hypothesis that findings may help us understand wider / other contexts

Is it desirable? depends on the researcher’s epistemological position

How can it be done? assertional rather than probabilistic logic

careful comparison and unassailable analysis rather than statistical representativeness or sampling theory

Drawing wider inference from qualitative

data

representational extrapolation – these views/experiences exist, we are not saying how frequently they will be found and we acknowledge that parts of the ‘map’ may be missing

empirical extrapolation – views, accounts of behaviour, processes, explanations applied by the reader to other settings based on typicality of case (transferability)

based on detailed case by case transfer

analytical or theoretical extrapolation

To conclude

We have addressed: the nature of qualitative research – what it can

and cannot deliver, developing qualitative strategies and designing

research questions, the main methods and techniques, drawing a qualitative sample, designing topic guides for use in interview or

focus group based studies, and judging the ‘quality’ of qualitative research.

Thank you for listening.

I hope you enjoy designing good quality qualitative research studies