Embed Size (px)
DESCRIPTION
Introduction to Qualitative Research. Aaron Louie, User Experience Architect. PURPOSE WHY QUALITATIVE RESEARCH? RESEARCH DESIGN VALIDITY & RELIABILITY WRITING INTERVIEW QUESTIONS ANALYZING DATA RESOURCES. PURPOSE WHY QUALITATIVE RESEARCH? RESEARCH DESIGN VALIDITY & RELIABILITY - PowerPoint PPT Presentation
Citation preview
ASCENTIUM225 108th Ave NE, Ste 225Bellevue, WA 98004t 425.519.7700f 425.519.7758ascentium.com
Introduction to Qualitative ResearchAaron Louie, User Experience Architect
asc
en
tiu
m.c
om
PURPOSE
WHY QUALITATIVE RESEARCH?
RESEARCH DESIGN
VALIDITY & RELIABILITY
WRITING INTERVIEW QUESTIONS
ANALYZING DATA
RESOURCES
asc
en
tiu
m.c
om
PURPOSE
WHY QUALITATIVE RESEARCH?
RESEARCH DESIGN
VALIDITY & RELIABILITY
WRITING INTERVIEW QUESTIONS
ANALYZING DATA
RESOURCES
asc
en
tiu
m.c
omWhat this presentation will do
Lay the basic groundwork for gaining skills in user research
Provide useful guidelines for writing interview questions
Point you to resources for deeper study in qualitative research methods
asc
en
tiu
m.c
omWhat it won’t do
Substitute for a college course on the subject
Make you an expert
asc
en
tiu
m.c
om
PURPOSE
WHY QUALITATIVE RESEARCH?
RESEARCH DESIGN
VALIDITY & RELIABILITY
WRITING INTERVIEW QUESTIONS
ANALYZING DATA
RESOURCES
asc
en
tiu
m.c
om
The purpose of qualitative research is…
asc
en
tiu
m.c
om…to make sense of reality…
asc
en
tiu
m.c
om…in a valid and reliable fashion.
Pre-Plan Set-up ExecuteManage
ResponsesReport & Analyze
Activities
Edit Activity Properties
Edit Activity Overview
Edit Activity Financial
Information
Edit Activity Budget Details
Edit Activity Owner
Enter Activity Name
Select CAPT Activity Type
Select Profit Center
Select Activity Start Date
Enter BudgetSelect Company
Code
Select Tax Jurisdiction
Enter Content Link
Enter Activity Description
U.S. Company Code
Yes
No
Assign P&L According to
Selected Profit Center
Assign P&L According to
Selected Profit Center
Select Activity End Date
Display Activity Budget
Availalble
Display IO Number
Edit Activity Marketing Attributes
Delegate Activity
View Activity Summary/Status
asc
en
tiu
m.c
omDescribing reality
Reality is complex.
All real problems are “wicked” problems.
Two ways to approach it: Quantify it. Then construct mathematical models describing it. Experience it qualitatively. Then make sense of it.
asc
en
tiu
m.c
omQuantitative vs. Qualitative
Controlled environment
Experimentation
Deductive
Numbers
Researcher is objective
Natural/complex environments
Interaction
Inductive
Words
Researcher is a participant
asc
en
tiu
m.c
om
Hypothesize ExperimentAnalyze
DataRefine
Hypothesis
The Scientific Method
asc
en
tiu
m.c
omQuantitative Research
Predict outcome &
design experiment to test outcome
Collect measurements in a controlled environment
Apply statistical
calculations
Identify patterns, revise
assumptions
Hypothesize ExperimentAnalyze
DataRefine
Hypothesis
asc
en
tiu
m.c
omQualitative Research
Identify assumptions &
formulate research questions
Collect data via observation, interviews,
surveys, etc.
Count, classify, &
structure data
Identify themes, revise assumptions
Hypothesize ExperimentAnalyze
DataRefine
Hypothesis
asc
en
tiu
m.c
om
PURPOSE
WHY QUALITATIVE RESEARCH?
RESEARCH DESIGN
VALIDITY & RELIABILITY
WRITING INTERVIEW QUESTIONS
ANALYZING DATA
RESOURCES
asc
en
tiu
m.c
omResearch Design
Determine scope What you’re researching What you’re not
Identify assumptions Models from literature Researcher’s biases
Draft research questions
Choose your methods Data collection methods Analysis methods
asc
en
tiu
m.c
omFraming Research Questions
List what you expect to find out, based on the scope and your assumptions.
For each finding, write a question (or series of questions) that must be answered to support or invalidate the finding.
Determine dependencies (which questions must be answered first).
asc
en
tiu
m.c
om
Qualitative Data Collection Methods
Surveys Pros: quantifiable, lower chance of affecting subject Cons: no interaction with subject, nuances of emotion or environment lost
Naturalistic observation Pros: gain understanding of environment without interfering with subject Cons: very little interaction with subject, no insight into subject’s mental
processes
Contextual inquiry (shadowing, on-site interviews, think-aloud, etc.) Pros: balanced level of interaction with subject, insight into subject’s
thoughts in context of tasks and environment Cons: researcher’s physical presence can affect subject’s behavior
Ethnography (embedded research) Pros: maximum level of interaction with subject, researcher gains deep
understanding by becoming a subject, subjects treat researcher as peer Cons: time- and cost-prohibitive
asc
en
tiu
m.c
omAnalysis Methods
Statistical Analysis
Content Analysis
Narrative Analysis
Strategic Analysis
Workflow Analysis
Task Analysis
Business Analysis
Cognitive Work Analysis
Systems Analysis
Grounded Theory
etc.
asc
en
tiu
m.c
om
PURPOSE
WHY QUALITATIVE RESEARCH?
RESEARCH DESIGN
VALIDITY & RELIABILITY
WRITING INTERVIEW QUESTIONS
ANALYZING DATA
RESOURCES
asc
en
tiu
m.c
omGood research design
Maximize validity Conclusions are true
Maximize reliability Findings are repeatable
Relatively easy to do in quantitative research
Impossible to do in qualitative research
asc
en
tiu
m.c
omThreats to validity
Changes in environment
Changes in society
Effects of time
Changes in person
Bias in subject
Interaction between subjects
(In other words, stuff you can’t control.)
asc
en
tiu
m.c
omThreats to validity(stuff you CAN control) Bias in researcher
Interaction between researcher and subject
asc
en
tiu
m.c
om
Researcher Subject
Ergonomics
Mental constraints & abilities
Tasks & activities
Work domain
Organizational structure
Business strategy
Personal resources & values
EthnographicClinical
Validity vs. Understanding
asc
en
tiu
m.c
om
Executive
Upper management
Supervisor
How does the subject view you?
asc
en
tiu
m.c
om
PURPOSE
WHY QUALITATIVE RESEARCH?
RESEARCH DESIGN
VALIDITY & RELIABILITY
WRITING INTERVIEW QUESTIONS
ANALYZING DATA
RESOURCES
asc
en
tiu
m.c
omComposing an Interview Script
List what must be known in order to answer each research question
For each item in that list, write a question
Organize the questions into categories
Sort general specific within each category
asc
en
tiu
m.c
omPhrasing Questions
Ask open questions for general questions “what”, “why”, “when”, “how”, “who”
Ask closed questions for specific questions “how often”, “how many”
Reserve yes/no questions for branching points in interview script(i.e., a line of questioning dependent on a previous answer) “Have you ever done X? If so, how did you feel about…”
asc
en
tiu
m.c
omRefining Interview Questions
Remove biased or charged language “What do you think about the problems with the intranet”
Avoid asking leading questions “Wouldn’t it be better if you did it this way?”
Be aware of your own biases! “How have consultants like Ascentium helped you get your job done
better?”
Structure questions to facilitate data analysis
asc
en
tiu
m.c
omConducting the Interview
Gain informed consent from the interviewee
Record audio/video whenever possible
Remain open-minded – don’t think too much
Listen, listen, listen
Avoid reacting too much (negatively or positively) to the interviewee’s responses
Take prodigious notes
Write up notes immediately after interview
asc
en
tiu
m.c
om
PURPOSE
WHY QUALITATIVE RESEARCH?
RESEARCH DESIGN
VALIDITY & RELIABILITY
WRITING INTERVIEW QUESTIONS
ANALYZING DATA
RESOURCES
asc
en
tiu
m.c
omAnalyzing The Data
Go through notes & transcript Identify frequent keywords Categorize responses Look for patterns
Match keywords, categories, and patterns to models and assumptions
asc
en
tiu
m.c
om
PURPOSE
WHY QUALITATIVE RESEARCH?
RESEARCH DESIGN
VALIDITY & RELIABILITY
WRITING INTERVIEW QUESTIONS
ANALYZING DATA
RESOURCES
asc
en
tiu
m.c
omResources
Books The Craft of Research by Wayne C. Booth, Joseph M. Williams, Gregory G.
Colomb [link] Qualitative Research Methods for the Social Sciences by Bruce L. Berg [link
] Cognitive Work Analysis: Toward Safe, Productive, and Healthy Computer-
Based Work by Kim J. Vicente [link]
Web sites Cognitive Work Analysis: Toward Safe, Productive, and Healthy Computer-
Based Work by Kim J. Vicente [link] Qualitative Research Consultants Association: FAQs about Qualitative
Research [link] Association for Information Systems: Qualitative Resarch in Information
Systems [link] UsabilityNet: Tools and methods [link] UW Libraries: Lit Reviews, Research Methods… [link] Virginia Tech: Annotated Bibliography on Qualitative Research
Methodology [link]
