PA 552: Designing Applied Research

PA 552: Designing Applied Research

Bruce Perlman

Understanding Methods

Data Framework

RESEARCH DATA

SECONDARY DATA

PRIMARY DATA

QUALITATIVE DATA

EXPLORATION

QUANTITATIVE DATA

DESCRIPTION CAUSE

AND EFFECT

Deciding on Data Strategy

• Choice Depends on the Situation

• Techniques Depend on Situations

– More Appropriate in Some Situations than Others

• Caution: All Techniques are Subject to Bias

Methodological Strategies

1. Qualitative

2. Quantitative

3. Mixed

Distinguishing Methodological Strategies • Data Can be Collected and Analyzed:

1. Qualitatively 2. Quantitatively

• Quantitative Collection and Analysis – Separate Processes

1. Collection 2. Analysis

• Qualitative Collection and Analysis – Iterative, Conceptually Separate Processes

1. Collection Occurs During Analysis 2. Analysis Occurs During Collection

• Mixed Methods

Research Design

Research Strategy

Quantitative Qualitative

Experimental

Cross-sectional

Longitudinal

Case study

Comparative

Quantitative vs. Qualitative

Quantitative vs. Qualitative

• Not Methods: Adjectives for Types of Data • Quantitative

– Data Represented Through: • Words • Pictures • Icons

– Analyzed Using Thematic Exploration

• Qualitative – Data Represented Through Numbers – Analyzed Using Statistics

• Do Represent a Fundamental Debate on – Knowledge Production

Quantitative Tradition

• Scientific Study and Method – Hypothetico-Deductive Logic:

• Test Hypotheses deduced from Theory

• Value of Quantification

• Nomothetic Characteristics 1. Use or Generate a Theory

2. Generate Propositions or Hypotheses from Theory

3. Collect Quantitative Data (Experiment or Description)

4. Analyze Data Using Statistical Processes

5. Draw Conclusions Referring to Hypotheses

Hypothetico-deductive Methods

• Experiments – Controlled Environment

• Not “Naturalistic”

– Identified Variables – Groups with Random Assignment

• Population Studies and Descriptions – Data

• Primary • Secondary

– Samples – Instruments – Statistics

Inductive Strategies

• Ethnography – POV of the Subjects about Culture – Immersion to Understand Why not just What

• Phenomenology – Phenomena = lived Experience – How Subjects of Phenomena Studied Experience it

• How they Experience Object In their Own Words

• Ethnomethodology – How Interactions Take Place – Description of Patterns, Rules, Methods, to Make Sense

• Raw, Natural Data between Roles or Positions

Qualitative Tradition

• Alternative to Scientific Study and Method – Emphasize Inductive Logic:

• Explore Phenomena to Explain Them • Explanation may Build or Expand Theory • Value of Quality over Quantity

• In Depth Characteristics 1. Holistic 2. Emergent Methods 3. Non-Random Sampling 4. Rich Data 5. Interpretive Analysis

Principal Difference

Quantitative Strategies Qualitative Strategies

• Research in which the researcher decides what to study

• Research in which the researcher relies on the views of the participants

Characteristics

Quantitative Research Qualitative Research

• Ask broad, general Qs

• Collecting data consisting largely of words (text) or image (picture)

• Descriptions and analysis of words for themes

• Conducts more subjective, inquiry

• Ask specific narrow Qs

• Collects data from participants

• Analyzes numbers using statistics

• Conducts more the objective

Identifying a Research Problem


• Description of trends or an explanation of variables’ relationships

• An exploration in which little is known about the problem

• A detailed understanding of a central phenomenon

Reviewing Literature


• Major role through suggesting the RQ to be asked

• Justifying the R problem and the need for the direction of the study

• Minor role in suggesting RQ to be asked

• Justify the importance of studying the research problem

Research Purpose


• Specific and Narrow

• Measurable, Observable Data on Variables

• General and Broad

• Understand participants’ experiences

Collecting Data


• Instruments with preset Qs and Res

• Large number of individuals

• Forms with general, emerging Qs

• Permit participant to generate responses

• Word(text) or image(picture) data

• Small number of individuals or sites

Quantitative Sampling Assumptions

Want to generalize to the

population

Random events are

predictable

Therefore…

Can compare random

events to our results

Probability sampling is

the best approach

Qualitative Sampling Assumptions

Social actors are not

predictable like objects.

Randomized events are

irrelevant to social life.

Probability sampling is

expensive and inefficient.

Therefore… Non-probability

sampling is the best

approach.

Types of Samples

Analyzing and Interpreting data


• Statistical analysis

• Describe trends, comparing group differences, relating variables

• Comparing results with prior predictions and past research

• Text analysis

• Description of themes

• Stating the larger meaning of findings

Triangulating and Mixing

Using Mixed Strategies

• Illustrating and Extending Quantitative Work – Adding Depth to Surveys

• E.g. Focus groups for marketing surveys

– Getting In-Depth Information for Instrument Design • E.g Interviews to Develop Questions on Questionnaire

• Refining and Extending Qualitative Work – Adding Breadth to Ethnography

• E.g. Survey as Part of Study

– Specifying Qualitative Data • E.g. Counting, Coding, Observed Behaviors or Mentions

In Social Sciences

• More than Two methods are Used in a Study – Double (or triple) checking results

– Also called "cross examination"

• More Confident if Different Methods get Same Result – One Method

• Temptation to believe in findings

– Two Methods • Results may clash

– Three Methods • Two of three may produce similar answers

• Three may produce different results

Triangulation Strategy • Combination of Several Methods in Both:

– Quantitative (validation) – Qualitative (inquiry)

• Facilitates Data Validation – Cross verification from more than two sources

• Method Appropriate Strategy – Increase credibility of qualitative analyses – Alternative to traditional criteria like reliability and validity

• Preferred line in social sciences • Combines Multiple :

– Observers – Theories – Methods – Empirical Materials

4 Basic Types of Triangulation 1. Data

– Time – Space – Persons

2. Investigator – Multiple Researchers

• Theory – Multiple Theory Interpretation

• Methodological – Multiple Method to Collect Data

• Interviews • Observations • Questionnaires • Documents

– Denzin (1978)