Choosing Research Designs II

Nonexperimental Methods

The Purpose of Control VariablesThe Purpose of Control Variables• We use control variables to account

for possible alternative explanations we can think of.

• For example, when I examined whether democracies are generally more peaceful than autocracies I included several control variables.

Explaining Pacifistic DemocracyExplaining Pacifistic Democracy

• Peace (Y) = Democracy (X1) + State Power (X2) + Development (X3) + # of Bordering States (X4)

• In the model above, I have more confidence that Democracy is related to peace considering I control for the other variables that may skew my test.

• We need to take care that our theory is not missing other factors that may undermine the validity of our theory and tests.

• Our inferences will be flawed if we are actually capturing other processes through our variables.

• This means that the validity of our measures would be undermined.

• Several possible problems arise that are related to model misspecification and spurious relationships.

Thus, we need to control for confounding factors and alternative explanations!!!

Model Misspecification and Model Misspecification and SpuriousnessSpuriousness• Antecedent variable: A variable that

indirectly affects the relationship between two other variables.

• For example, Ivy league education increases income.

• However, parental wealth and legacy admissions affect Ivy league education. Thus, income of graduates from Ivy League schools may not be random.

Here Ivy League Parents is an antecedent variable

Ivy League Parents Ivy League Kids high income kids

Hence, admission to Ivy schools clearly not random or pure merit-based, and thus the income earned by these people.

Model Misspecification and Spuriousness

• Intervening Variable: These may be spuriously related to another relationship.• How can states fight each other if they are

not contiguous with each other? Only the strongest, with large navies, bases, etc., could do so.

• Hence, geographic contiguity or distance is an intervening variable. States may or may not be more peaceful, but it is hard to avoid conflict when it is on your borders.

Model Misspecification and Spuriousness

• Alternative Variables: We also want to control for variables that would bias our results if omitted.

• In this case, the X variables in a model would produce biased estimates, undermining their validity and producing error that leads to inaccurate inferences.

Here is a spurious relationship from my research

IGOs + conflicts + +Powerful states

Powerful states both in more IGOs and conflicts, but these two variables not directly related but a function of state power.

Classic Spurious Case

Ice Cream Consumption Crime

Summer Temperatures

???

+ +

+

Hence we see that despite the fact that ice cream consumption is correlated with crime, the real cause is that summer temperatures increase both ice cream consumption and crime.

Veronica Says, Beat Marshall!!!

Go Miners!!!

UTEP Fight!

UTEP Win!

I’m going to Homecoming,Are you?

Non-Experimental DesignsThese studies use data collected or

aggregated from surveys, history, or government indicators:

• Cross sectional studies• Panel (cross sectional over a few

time points)• Longitudinal (time series and pooled

cross-sectional time series)• Case studies and focus groups

CROSS SECTIONAL Designs• Statistical or case studies that

compare individuals or subjects across several variables:

• Surveys comparing peoples’ political views

• Comparison of countries, groups, organizations along different dimensions, such as countries with different levels of development (low, medium, high) relative to other factors.

Non-Experimental DesignsThese studies use data collected or

aggregated from surveys, history, or government indicators:

• Cross sectional studies• Panel (somewhat rare)• Longitudinal (time series and pooled

cross-sectional time series)• Case studies and focus groups

CROSS SECTIONAL Designs• Statistical or case studies that

compare individuals or subjects across several variables:

• Surveys comparing peoples’ political views

• Comparison of countries, groups, organizations along different dimensions, such as countries with different levels of development (low, medium, high) relative to other factors.

Cross-Sectional DataCross-Sectional DataID State

Abortions/1,000women

%Bush04

Conservative score for House delegation

1Alabama

15 62.5 733 Arizona 19.1 54.8 67

4Arkansas

11.1 54.3 485 California 33.4 44.4 416 Colorado 18 51.7 67.8

7Connecticut

23 44 37.6

8Delaware

34.4 45.8 4010 Georgia 21.2 58 63.712 Idaho 5.8 68.4 9013 Illinois 25.6 44.5 48.914 Indiana 10.6 59.9 6915 Iowa 9.8 49.9 64.616 Kansas 18.3 62 75

Example of a Panel StudyExample of a Panel StudyState Democracy Illiteracy HDI IslamicArgentina91 7 4.3 0.81 0Argentina95 7 3.7 0.832 0Argentina00 8 3.3 0.854 0Armenia91 7 2.57 0.751 0Armenia95 3 2 0.708 0Armenia00 5 1.69 0.754 0Australia91 10 0 0.892 0Australia95 10 0 0.932 0Australia00 10 0 0.942 0Azerbaijan91 -3 3 . 1Azerbaijan95 -6 3 . 1Azerbaijan00 -7 3 0.746 1Bangladesh91 6 65 0.417 1Bangladesh95 6 61.9 0.445 1Bangladesh00 6 59.2 0.497 1Belarus91 7 0.7 0.785 0Belarus95 0 0.5 0.752 0Belarus00 -7 0.5 0.775 0Belgium91 10 2 0.897 0

Time SeriesTime Series• Observations are made over time, which

can provide descriptive information or used to test hypotheses.

• If testing hypotheses, we track data for a dependent variable and at least one independent variable over time (based on some measure e.g. days, weeks, months, or years)

Example of a Time Series: Presidential Example of a Time Series: Presidential ApprovalApproval