6/6/08

Research Methods andInterview ExerciseDepartment of Urban and Regional PlanningAgrawal, Kos, McLaughlin, and Nixon

A Quick Primer on Epistemology

•What is epistemology?- It is the study of knowledge- How do we know what we know?

•Branches of epistemology: the RICE model- Rationality- Idealism- Constructivism- Empiricism (most often used in social

sciences)

2

Rationality

3

•Rationality- Knowledge is deductive- To know something requires logic:

- (1) All planners are intelligent- (2) 298A students are planners- (3) Therefore, all 298A students are intelligent

•Drawbacks- Validity dependent upon logic of argument- Are all planners intelligent?- Can lead to valid, but unsound arguments

Idealism

4

•Idealism- Knowledge is intuitive, and a priori.- We are born with what we will (or do) know

- E.g, my consciousness exists. - We can know things without observation:

- i.e., 1 + 1 = 2 (Mathematical reasoning)

•Drawbacks- Discounts influence of senses in “knowing”- “Reality” does not exist without our mind- Disregards environmental determinism

Constructivism

5

•Constructivism- Knowledge is a human construct- Objective knowledge does not exist - everything is

subjective- E.g, scientific knowledge is constructed by scientists- Social norms dictate our perception of the world

•Drawbacks- There are no universal truths- No person’s knowledge is more true than another’s- No such thing as common sense.

Empiricism

6

•Empiricism- Knowledge comes mostly from observation- At the heart of the scientific paradigm

- Theories developed and revised by testing hypotheses in the real world

- Evidence is key to knowing - though knowledge is never “proved.”

•Drawbacks- Observations can be prone to biases of observer- Data prone to measurement error- Impossible to explore “unobservable” ideas/concepts

Empiricism is dominant in social sciences

7

•Empiricism is most commonly used by planners - Others are used, but scientific approach dominates- Likely due to centrality of the natural, built, and social

environment in the discipline- Planners thus need approaches to knowledge creation

that are suited to observing these environments•Empirical observation relies on data gathering

- Two types of data:• Qualitative data: words and experiences• Quantitative data: numbers

- Each requires unique methodologies to analyze

A quick overview of Data Sources

8

•Two general sources of data- Primary data: information gathered at actual events• Interviews, surveys, personal reports, photos, etc.• Documents are also considered primary when it is the

object of study itself‣ Census data/official statistics, meeting minutes,

government documents- Secondary data: information based on others’ accounts• Newspapers, reference books, encyclopedias,

magazines• Generally considered less reliable than primary data

Data Caveats

9

•All data types and sources subject to ABCs issues:- Accuracy- Bias- Completeness

• As a researcher, you need to focus on two criticisms- External Criticism: use ABCs to focus on the quality of

the data gathering and compilation.- Internal Criticism: uses ABCs to focus on quality of

interpretation.

Data Caveat Examples

10

• An internet survey of aging and isolation, conducted by the American Association of Nursing Homes, shows older Americans are less susceptible to loneliness”- Is it accurate?- Is it biased?- Is it complete?

• A randomized study of 100,000 persons, conducted by the American Medical Association, shows owning a dog can lead to better heart health ”- Is it accurate?- Is it biased?- Is it complete?

Quantitative Methodologies

11

•Quantitative Analysis - Used to summarize, describe, and interpret numbers- In social science, consists of statistical methods- Two types of statistical tools• Descriptive: used to describe patterns data‣ Mean, median, standard deviation, outlier analysis

• Inferential: used to interpret data‣ Hypothesis testing, correlation, ANOVA, regression

• Will not go into further detail about quantitative analysis- Refer to your notes from URBP 204

Qualitative Methodologies

12

•Qualitative Analysis - Used to summarize, describe, and interpret words- Case studies often used.- Popular methods: interviews, surveys, and observation.- Interviews/surveys: will focus rest of session on these

methods- Three approaches to observation:‣ Participant Observation - researcher observes by

participating/interacting with subjects‣ Non-Participant Observation: researcher is present, but

does not participate/interact with subjects‣ Hidden Observation: researcher is out of sight. A “fly on the

wall” approach.

But what about case study research?

13

•Case study- More of a typology, than a method‣ “An intensive analysis of an individual unit (as a person or community)

stressing developmental factors in relation to environment.” - Merriam-Webster’s dictionary (2009).

- Key features of case studies‣ (1) Intensive Analysis (“thick” description)‣ (2) Looks at processes and causes‣ (3) Focuses on unit and its context‣ (4) Chooses small number of units for analysis

- Case studies are NOT just case descriptions.

Is it a case study?

14

•Example 1:‣ Research question: What policies can Happy Valley City adopt to

promote urban agriculture without negatively impacting residents with noise, odors, or other nuisances?

‣ Method: Review the policies in 5 or 10 cities known to have progressive urban ag policies, and then see which policies might fit well with Happy Valley.

•Example 2:‣Research question: Why does Happy Valley City have thriving

urban agriculture, while Sad Valley City does not?‣Method: Review the policies of each city to see what they are,

interview current and past planners about what policies have been in place and which seem to have worked, and survey residents to find out if they grow their own food and why or why not.

Is it a case study?

15

•Strengths of case study research: ‣Can explain how or why a complex process or event happens.‣Can explain a particular event or small set of events.‣Can show what happened in one or a small number of instances,

so that we can HYPOTHESIZE or theorize about general rules. (KEY: The results can’t be proven to be generalizable)

‣Can falsify (prove wrong) existing theory or conventional wisdom•Disadvantages of case study research:‣Can be extremely time-consuming, with multiple methods‣Easy to get sidetracked and lost . . . hard to define boundaries‣Can be hard to write up (How to be analytical? How to get in

relevant detail but not present tangents?)

The Case Study Approach

16

•How to choose a case study?‣You should have a theoretical basis for selecting cases

-(1) Cases might be a good/average representative of others•Allows you to better generalize findings

-(2) Cases could also be very unique or unusual•Allows you to possibly uncover breakthrough findings

•Case study methods/evidence:‣Methods are not just qualitative!

- In fact, mixed-methods are ideal.- Goal is to triangulate

• Triangulation: using multiple methods or sources of information to confirm that your findings are true.

Interviews

17

•What is the purpose of an interview?‣Interviews are excellent for uncovering perceptions/opinions:

• Learning how people see the world• Learning what people like/don’t like• Learning what their values are (directly or indirectly

expressed)‣Interviews are not as good for finding factual information

• Why?•How is an interview different from survey?

•You don’t have to know the answers in advance• You can ask why and how• It’s not anonymous (good/bad)

How do you conduct a good interview?

18

•Do you even need to do an interview?‣Ask yourself:

• What is my research question?• How will interview data help me answer it?• Can get the information another way?

- Don’t waste your time, or other people’s•Select the interviewees

•Who would you like to talk to?• How many do you need?• How will you contact them?•Do you need to contact any “gatekeepers” to gain access? •Will it be in person, or by phone? Can it be recorded?

Preparing your interview questionnaire

19

•There are many types of interviews- From unstructured and open ended, to a closed-answer script- We’ll focus on preparing for a standardized, open ended

interview•Basic Approach

- An opening to explain the interview- Have a set of questions you hope to ask of everyone- Draft a set of follow-up probes to use as needed- Be prepared to be flexible and skip questions or add follow-ups

as needed

Language and Sequence Matters!

20

•The language of your questions should:- be clearly worded for someone listening- consist of neutral wording (ie, not loaded/biased)- encourage discussion (not answered with yes/no)

•Sequence of questions- Start with easy questions to relax respondent‣ Facts‣ Non-controversial topics‣“Easy-to-know” answers

- Place important questions 2/3 of way through- End by asking respondents if they have any other information that

might be helpful to your study.

Always strive for improvement

21

•Ways to reduce error and/or bias- Sources of respondent bias‣ Memory problems‣Exaggeration‣Hidden agenda‣Misunderstanding‣Lack of expertise‣Desire to please interviewer

-Sources of interviewer bias‣Reaction to responses‣Wording of questions‣Verbal and non-verbal cues

Interview Tasks

22

•(1)Make contact and request interview- explain the project and don’t hide purpose- choose whether it will be by email, phone, or in-person- be persistent!

•(2)Pre-interview preparation- review questionnaire- test equipment

• (3) The interview- think how to make interviewee comfortable- think how to reduce your bias- remember to listen!

Interview Tasks

23

•(4)Post-interview- Rewrite and organize note immediately (1-2 hours)‣ Revise notes‣ Add thoughts/comments (what was interesting, unusual, etc.)

- Within a day or two‣Follow up with a thank-you note‣Ask any follow-up questions

- Within a week‣Reanalyze notes and/or transcript

24 Running Footer - Title of Subtitle