40
The design of your study will help you identify causality Keeping all of this straight will help you make stronger and more interesting arguments about your findings. Research Design

The design of your study will help you identify causality

Embed Size (px)

DESCRIPTION

Research Design. The design of your study will help you identify causality. Keeping all of this straight will help you make stronger and more interesting arguments about your findings. Time Dimension. Cross-Sectional Studies Longitudinal Studies Trend-Repeated Cross-sectional - PowerPoint PPT Presentation

Citation preview

Page 1: The design of your study will help you identify causality

The design of your study will help you identify causality

Keeping all of this straight will help you make stronger and more

interesting arguments about your findings.

Research Design

Page 2: The design of your study will help you identify causality

Time Dimension

• Cross-Sectional Studies

• Longitudinal Studies– Trend-Repeated Cross-sectional– Cohort-Event-based design– Panel-Fixed sample

Page 3: The design of your study will help you identify causality

Nonspuriousness

• Random assignment

• Control Group

• Statistical controls– Multivariate Regression

• Including control variables (which ones? – theory)

• Multiple Cases– Most similar, or different case design

• Exploring alternative explanations

• Selecting a “hard” case

Page 4: The design of your study will help you identify causality

Crime and Support for Democracy

Kenneth E. FernandezDepartment of Political Science

University of Nevada, Las Vegas

Michele KuenziDepartment of Political Science

University of Nevada, Las Vegas

Page 5: The design of your study will help you identify causality

Research Question

Does crime influence how citizens evaluate democracies or influence their

level of support for democracy?

Page 6: The design of your study will help you identify causality

Why is this an Interesting Question?

• Scholars have noted that few studies, especially in the field of political science, have examined crime and its impact on society in a comparative manner (Beirne 1997; Howard, et al. 2000).

• Current public opinion research suggests that the issue of crime is of great concern to citizens across the globe (Quann and Hung 2002)

• Many countries that have experienced recent democratic reforms have also experienced increasing crime rates (Bergman 2006).

Page 7: The design of your study will help you identify causality

What theories inform the study of crime and democracy?

–Social Capital

–Rational Choice

–Modernization Theory

Page 8: The design of your study will help you identify causality

Crime is Often Ignored as a Factor in Democratization

– Much of the prior research on the development of democracy focuses on economic conditions (Arat 1988; Epstein et al. 2006; Lipset 1959; Przeworski et al. 2000).

– In fact, Kugler and Feng describe the recent application of modernization theory as positing that “economic development is a sufficient, rather than a necessary, condition for democratic transitions” (1999, 140).

– We argue that crime and public disorder may pose even greater challenges to the legitimacy and effectiveness of democratic institutions

Page 9: The design of your study will help you identify causality

Social Capital Theories: – Theory suggests that attitudes and feelings of

the masses regarding obligations within a group, civic attachments, trust, and efficacy are important determinants of democratic transition and sustainability (Muller and Seligson 1994).

– Yet crime and violence is likely to reduce trust and social capital (Lederman et al 2002).

Page 10: The design of your study will help you identify causality

Rational Choice: – Citizens are treated as utility maximizing

consumers– A political regime is expected to produce some

value or benefit to the consumer (Lane 1995).– Regime support depends on how well citizens’

demands are met– Thomas Hobbes argues that protection against

violence is the highest and most fundamental human need and a citizen of the state has a right to revoke its support and loyalty when the state proves incapable of protecting its subjects.

Page 11: The design of your study will help you identify causality

The Paradox of Modernization theory: – Modernization leads to increasing wealth,

leading to more literacy, education, urbanization, leading to democratization because citizens are less willing to put up with oppressive, authoritarian behavior and more willing to support democracy (Lipset 1959)

– But modernization may cause a breakdown in traditional structures and values, an increase in interaction, tension, and conflict. These social changes then contribute to rising criminal activity (Heiland and Shelley 1992).

Page 12: The design of your study will help you identify causality

Modernization Theory, Crime & Democracy– Consolidating both the political science and the

criminology/sociology literature on modernization theory, we argue that socio-economic conditions, as well as, crime and public safety influence citizens’ support for democracy.

Page 13: The design of your study will help you identify causality

Methodology: Macro or Micro Level Analysis?– Much of the research testing the modernization

theory uses a macrolevel approach utilizing the nation-state at time t as the unit of analysis. Aggregate national characteristics (i.e., per capita GDP) are then used as predictors of some measure of democracy.

– Yet Lipset (1959) posits that increases in wealth and income leads to democratic transition because it affects citizens’ values.

– We would like to examine this microlevel statement.

Page 14: The design of your study will help you identify causality

Microlevel Analysis:

– An examination of citizen attitudes is appropriate because the success of democratic consolidation in these emerging democracies is often linked to citizens’ support for democracy and perceptions of government performance (Hiskey and Bowler 2005; Lagos 2001; Diamond 1999; Lipset 1959).

Page 15: The design of your study will help you identify causality

DATA: • Afrobarometer (AB) and Latinbarometro (LB)

Survey Data• The AB and LB together contain 33 countries (16

in Africa and 17 in Latin America).

Two Statistical Approaches– Analysis of all countries pooled together (over

18,000 respondents in African and 13,000 in Latin America

– Analysis of 4 cases

Page 16: The design of your study will help you identify causality

Case Selection:– Initial Case - NIGERIA: Like many other

African countries, Nigeria experienced a transition to democracy in the 1990s. We chose Nigeria because crime is a very salient issue in this country, and Nigeria has been well known for mob justice or what has been called “jungle justice”

– Most Different Systems Design: We performed a cluster analysis to find a case in the Afrobarometer data that was “most different” to Nigeria in relation to three factors: crime, democracy and per capita GDP.

Page 17: The design of your study will help you identify causality

Cluster Analysis:

The Euclidean distance between two cases is calculated by finding the square root of the sum of the squared differences between the two cases for each of the three variables (crime rate; per capita GDP; level of democracy):

222jijiji zzyyxx

Page 18: The design of your study will help you identify causality

MALAWI: Nigeria has a per capita GDP nearly 3 times that of Malawi, but Nigeria has a substantially higher reported victimization rate (Nigeria had the highest reported attacks of the 14 African cases and Malawi had the lowest). Both countries were given a score of 4 for political rights and civil liberties by Freedom House in 2005.

CHILE & NICARAGUA: Both Chile and Nicaragua have modest crime rates compared to the rest of Central and South America, but differ greatly on GDP and level of democracy. Chile has a per capita GDP almost 7 times higher than Nicaragua and Chile has a score of 1 for both political rights and civil liberties while Nicaragua scored a 3 for both.

Page 19: The design of your study will help you identify causality

THE 4 CASES: The results seem to make intuitive sense. There is substantial variation in GDP across cases, they vary in crime rates from low to moderate to high levels of victimization, and levels of political freedom and civil liberties varied from the highest (score of 1) to moderate/low (score of 4).

Page 20: The design of your study will help you identify causality

Operationalization

Dependent Variables: Attitudes toward Democracy Support for Democracy • Which of these three statements is closest to your

own opinion? • A: Democracy is preferable to any other kind of

government. • B: In some circumstances, a non-democratic

government can be preferable. • C: For someone like me, it doesn’t matter what kind

of government we have. • Recoded so that the value on this variable is 1 if the

response was A, and 0 otherwise.

Page 21: The design of your study will help you identify causality

• Satisfaction with Democracy • Overall, how satisfied are you with the way

democracy works in __________.? Are you:• 0=Tanzania/Nigeria is not a democracy,

1=Not at all satisfied, 2=Not very satisfied, 3=Fairly satisfied, 4=Very satisfied.

Chile and Nicaragua:• In general, would you say that you are very

satisfied, fairly satisfied, not very satisfied or not at all satisfied with the way democracy works in (country)? 4=Very satisfied - 1=Not at all satisfied

Page 22: The design of your study will help you identify causality

Independent Variables

• Experience with Crime

• Comparative assessments of safety from crime and violence

• Crime Performance

Page 23: The design of your study will help you identify causality

Control Variables

• Demographic variables: gender, level of poverty, age, education

• Other variables: political engagement, identification with the party in power, and subjective assessments of satisfaction with life and the economy and government service delivery

Page 24: The design of your study will help you identify causality

TABLE 7: ATTITUDES TOWARD DEMOCRACY II Model 7 Model 8 Support

Democracy (Chile)

Support Democracy (Nicaragua)

Satisfaction with Democracy

(Chile)

Satisfaction with Democracy (Nicaragua)

Est. Method

Logit

Logit

Ordered Logit

Ordered Logit

War on Crime

0.3019** (0.1336)

0.3320*** (0.1180)

0.4495*** (0.1365)

0.3620*** (0.1107)

Victim of Crime

-0.0176 (0.1896)

-0.2076 (0.1929)

-0.0202 (0.1776)

0.1336 (0.1672)

Discuss Politics

0.4759*** (0.1216)

0.3663*** (0.1002)

0.0278 (0.1278)

0.0183 (0.0856)

TV-News

-0.0327 (0.0568)

-0.0232 (0.0679)

0.0366 (0.0507)

0.0227 (0.0581)

Service Delivery

0.0182

(0.0648)

-0.1179** (0.0561)

0.2107*** (0.0662)

0.1414*** (0.0526)

Economy

0.4059*** (0.1296)

0.0094 (0.1179)

0.7059*** (0.1353)

0.4070*** (0.0937)

Quality of Life

0.2248* (0.1226)

-0.0108 (0.1104)

0.4065*** (0.1146)

0.2658*** (0.1028)

Education

0.0384 (0.0309)

-0.0092 (0.0234)

0.0484 (0.0305)

0.0066 (0.0195)

Poverty

0.1871 (0.1437)

-0.0438 (0.1099)

0.2208 (0.1464)

0.0152 (0.0908)

Female

0.0260 (0.1852)

-0.2137 (0.1839)

-0.0736 (0.1750)

-0.6451 (0.1668)

Age

0.0080 (0.0062)

0.0143** (0.0065)

0.0149*** (0.0057)

0.0114* (0.0064)

Cons

-4.1139*** (0.7574)

-0.9637 (0.6937)

# obs 1000 604 1018 640 Pseudo R2 0.0903 0.0538 0.1159 0.0548 LR chi2 74.65 32.55 133.45 73.55 Prob > chi2 0.0000 0.0006 0.0000 0.0000

Page 25: The design of your study will help you identify causality

Results

Attitudes toward Democracy

Feeling Safe

• Nigerians who report feeling better off in terms of safety from crime and violence are significantly more likely to:

– Support democracy

– Be satisfied with the way democracy is working

Page 26: The design of your study will help you identify causality

Crime Victimization

• Nigerians and Malawians who report a higher number of experiences with crime victimization have significantly lower levels of support for democracy.

• There appears to be no relationship between crime victimization and attitudes toward democracy in the Chile and Nicaragua countries.

Page 27: The design of your study will help you identify causality

Conclusion and Implications

• Perceptions of safety and government performance in the area of crime affect attitudes toward democracy.

• Crime victimization generally does not appear related to satisfaction with democracy.

Page 28: The design of your study will help you identify causality

Independent Sample T-test Formula

21

21

21

222

211

221 NN

NN

NN

sNsNs xx

t =

21

21

XXs

XX

Page 29: The design of your study will help you identify causality

Independent-samples t-tests

• But what if you have more than two groups?

• One suggestion: pairwise comparisons (t-tests)

Page 30: The design of your study will help you identify causality

Multiple independent-samples t-tests

• That’s a lot of tests!

# groups # tests2 groups = 1 t-test3 groups = 3 t-tests4 groups = 6 t-tests5 groups = 10 t-tests...10 groups = 45 t-tests

Page 31: The design of your study will help you identify causality

Inflation of familywise error rate

• Familywise error rate – the probability of making at least one Type I error (rejecting the Null Hypothesis when the null is true)

• Every hypothesis test has a probability of making a Type I error ().

• For example, if two t-tests are each conducted using = .05, there is a .0975 probability of committing at least one Type I error.

Page 32: The design of your study will help you identify causality

• The formula for familywise error rate:

# groups # tests nominal alpha familywise alpha

2 groups 1 t-test .05

3 groups 3 t-tests .05

4 groups 6 t-tests .05

5 groups 10 t-tests .05

...

10 groups 45 t-tests .05

Inflation of familywise error rate

1 1c

11 1 1 .95 .05

c

31 1 1 .95 .14

c

61 1 1 .95 .26

c

101 1 1 .95 .40

c

451 1 1 .95 .90

c

Page 33: The design of your study will help you identify causality

Analysis of Variance: Purpose

• Are there differences in the central tendency (mean) of groups?

• Inferential: Could the observed differences be due to chance?

Page 34: The design of your study will help you identify causality

Assumptions of ANOVA

• Normality – scores should be normally distributed within each group.

• Homogeneity of variance – scores should have the same variance within each group.

• Independence of observations – observations are randomly selected.

Page 35: The design of your study will help you identify causality

Logic of Analysis of Variance

• Null hypothesis (Ho): Population means from different conditions are equal– m1 = m2 = m3 = m4

• Alternative hypothesis: H1

– Not all population means equal.

Page 36: The design of your study will help you identify causality

Lets visualize total amount of variance in an experiment

Between Group Differences(Mean Square Group)

Error Variance (Individual Differences + Random Variance) Mean Square Error

Total Variance = Mean Square Total

F ratio is a proportion of the MS group/MS Error.The larger the group differences, the bigger the FThe larger the error variance, the smaller the F

Page 37: The design of your study will help you identify causality

Logic--cont.

• Create a measure of variability among group means– MSgroup

• Create a measure of variability within groups– MSerror

Page 38: The design of your study will help you identify causality

Loves Statistics Hates Statistics Indifferent

9 4 8

7 7 4

6 5 5

11 6 6

12 3 7

Example: Quiz Scores and Attitudes on Statistics

Page 39: The design of your study will help you identify causality

• Find the sum of squares between groups

• Find the sum of squares within groups

• Total sum of squares = sum of between group and within group sums of squares.

22totaltotalgroupgroupbetween XNXNSS

22groupgrouptotalwithin XNXSS

22totaltotaltotaltotal XNXSS

Page 40: The design of your study will help you identify causality

• To find the mean squares: divide each sum of squares by the degrees of freedom (2 different dfs)

• Degrees of freedom between groups =

• k-1, where k = # of groups

• Degrees of freedom within groups = n-k

• MSbetween= SSbetween/dfbetween

• MSwithin= SSwithin/dfwithin

• F = MSbetween / MSwithin

• Compare your F with the F in Table D