SW388R6 Data Analysis and Computers I Slide 1 Chi-square Test of Independence Reviewing the Chi-square Test of Independence Sample Homework Problem Solving

SW388R6Data Analysis

and Computers I

Slide 1

Chi-square Test of Independence

Reviewing the Chi-square Test of Independence

Sample Homework Problem

Solving the Problem with SPSS

Logic for Chi-square Test of Independence


and Computers I

Slide 2

Chi-square Test of Independence

The chi-square test of independence is one of the most frequently used hypothesis test in the social sciences because it can be used with variables at any level of measurement.

In this set of problems, we will use the chi-square test of independence to evaluate group differences when the dependent variable is nominal, dichotomous, ordinal, or grouped interval.

The chi-square test of independence can be used for any variable; both the group (independent) and the test variable (dependent) can be nominal, dichotomous, ordinal, or grouped interval.


and Computers I

Slide 3

Chi-square test of independence vs. other statistical tests comparing groups -

1 Tests of means (t-tests and Anova) compare the

central tendency of groups so that we can say whether one group tends to score higher or lower on the dependent variable.

The chi-square test of independence presumes that all variables are nominal, and tests for differences within individual categories of the dependent variable.

For example, a chi-square test of independence will tell us whether there were more males in the satisfied category than we would have expected by chance, but this does not imply that males were generally more satisfied than females.


and Computers I

Slide 4

Chi-square test of independence vs. other statistical tests comparing groups -

2 To demonstrate, look at the following table:

Males FemalesRow Total

1 = Dissatisfied 20 5 25

2 = Neutral 10 35 45

3 = Satisfied 30 20 50

Column Total 60 60 120

Since the total number of males and females is the same (60), we would expect the 50 subjects in the Satisfied category to be evenly split (25 males and 25 females). Having 30 males instead of 25 implies that males are over-represented in this group.

Yet, females have a higher mean score than males (2.25 vs. 2.17) as an indication of overall trend.


and Computers I

Slide 5

Independence Defined

Two variables are independent if, for all cases, the classification of a case into a particular category of the independent variable has no effect on the probability that the case will fall into any particular category of the dependent variable.

When two variables are independent, there is no relationship between them. We would expect that the frequency breakdowns of the dependent variable to be similar for all groups.


and Computers I

Slide 6

Independence Demonstrated

Suppose we are interested in the relationship between gender and attending college.

If there is no relationship between gender and attending college, and 40% of our total sample attend college, we would expect 40% of the males in our sample to attend college and 40% of the females to attend college.

If there is a relationship between gender and attending college, we would expect a higher proportion of one group to attend college than the other group, e.g. 60% to 20%.


and Computers I

Slide 7

Displaying Independent and Dependent Relationships

Independent Relationship between Gender and College

40% 40% 40%

0%

20%

40%

60%

80%

100%

Males Females TotalPo

po

rtio

n A

tte

nd

ing

Co

lleg

e

Dependent Relationship between Gender and College

60%

20%

40%

0%

20%

40%

60%

80%

100%

Males Females TotalPo

po

rtio

n A

tte

nd

ing

Co

lleg

e

When the variables are independent, the proportion in both groups is close to the same size as the proportion for the total sample.

When group membership makes a difference, the dependent relationship is indicated by one group having a higher proportion than the proportion for the total sample.


and Computers I

Slide 8

Expected Frequencies

Expected frequencies are computed as if there is no difference between the groups, i.e. both groups have the same proportion as the total sample in each category of the dependent variable.

Since the proportion of subjects in each category of the independent variable can differ, we take group category into account in computing expected frequencies as well.

To summarize, the expected frequencies for each cell are computed to be proportional to both the breakdown for the dependent variable and the breakdown for the independent variable.


and Computers I

Slide 9

Expected Frequency Calculation

The data from “Observed Frequencies for Sample Data” is the source for information to compute the expected frequencies. Percentages are computed for the column of all students and for the row of all GPA’s. These percentages are then multiplied by the total number of students in the sample (453) to compute the expected frequency for each cell in the table.


and Computers I

Slide 10

Expected Frequencies vs. Observed Frequencies

The chi-square test of independence plugs the observed frequencies and expected frequencies into a formula which computes how the pattern of observed frequencies differs from the pattern of expected frequencies.

Probabilities for the test statistic can be obtained from the chi-square probability distribution so that we can test hypotheses.

The chi-square test of independence is a test of the influence or impact that a subject’s score on the independent variable has on the same subject’s score for the dependent variable.


and Computers I

Slide 11

Level of measurement and sample size requirements

The chi-square Test of Independence can be used for any level variable, including interval level variables grouped in a frequency distribution. It is most useful for nominal variables for which we do not another option.

Sample size requirement: No cell has an expected frequency less than 1. No more than 20% of cells have an expected

frequency less than 5.

If these requirements are violated, the chi-square distribution will give us misleading probabilities.


and Computers I

Slide 12

Hypotheses

The research hypothesis states that the two variables are dependent or related. This will be true if the observed counts for the categories of the variables in the sample are different from the expected counts.

The null hypothesis is that the two variables are independent. This will be true if the observed counts in the sample are equal to the expected counts.

The decision rule for the chi-square test of independence is the same as our other statistical tests: reject the null hypothesis if the probability of the test statistic is less than or equal to alpha.


and Computers I

Slide 13

Sampling distribution and test statistic

To test the relationship, we use the chi-square test statistic, which follows the chi-square distribution.

If we were calculating the statistic by hand, we would have to compute the degrees of freedom to identify the probability of the test statistic. SPSS will print out the degrees of freedom and the probability of the test statistic for us.


and Computers I

Slide 14

Computing the Test Statistic

Conceptually, the chi-square test of independence statistic is computed by summing the difference between the expected and observed frequencies for each cell in the table divided by the expected frequencies for the cell.

We identify the value and probability for this test statistic from the SPSS statistical output.


and Computers I

Slide 15

Decision and Interpretation

If the probability of the test statistic is less than or equal to the probability of the alpha error rate, we reject the null hypothesis and conclude that our data supports the research hypothesis. We conclude that there is a relationship between the variables.

If the probability of the test statistic is greater than the probability of the alpha error rate, we fail to reject the null hypothesis. We conclude that there is no relationship between the variables, i.e. they are independent.


and Computers I

Slide 16

Which Cell or Cells Caused the Difference

One of the problems in interpreting chi-square tests is the determination of which cell or cells produced the statistically significant difference. Examination of percentages in the contingency table and expected frequency table can be misleading.

The residual, or the difference, between the observed frequency and the expected frequency is a more reliable indicator, especially if the residual is converted to a z-score and compared to a critical value equivalent to the alpha for the problem.

Like the post-hoc tests in one-way Anova, we only conduct post-hoc tests If the result of the chi-square test of independence is statistically significant.


and Computers I

Slide 17

Standardized Residuals

SPSS prints out the standardized residual (converted to a z-score) computed for each cell. It does not produce the probability or significance.

Without a probability, we will compare the size of the standardized residuals to the critical values that correspond to an alpha of 0.05 (+/-1.96) or an alpha of 0.01 (+/-2.58). The problems will tell you which value to use. This is equivalent to testing the null hypothesis that the actual frequency equals the expected frequency for a specific cell versus the research hypothesis of a difference greater than zero.

There can be 0, 1, 2, or more cells with statistically significant standardized residuals to be interpreted.


and Computers I

Slide 18

Interpreting Standardized Residuals

Standardized residuals that have a positive value mean that the cell was over-represented in the actual sample, compared to the expected frequency, i.e. there were more subjects in this category than we expected.

Standardized residuals that have a negative value mean that the cell was under-represented in the actual sample, compared to the expected frequency, i.e. there were fewer subjects in this category than we expected.


and Computers I

Slide 19

Interpreting Cell Differences in a Chi-square Test - 1

A chi-square test of independence of the relationship between sex and marital status finds a statistically significant relationship between the variables.


and Computers I

Slide 20


Researcher often try to identify try to identify which cell or cells are the major contributors to the significant chi-square test by examining the pattern of column percentages.

Based on the column percentages, we would identify cells on the married row and the widowed row as the ones producing the significant result because they show the largest differences: 8.2% on the married row (50.9%-42.7%) and 9.0% on the widowed row (13.1%-4.1%)


and Computers I

Slide 21


Using a level of significance of 0.05, the critical value for a standardized residual would be -1.96 and +1.96. Using standardized residuals, we would find that only the cells on the widowed row are the significant contributors to the chi-square relationship between sex and marital status.

If we interpreted the contribution of the married marital status, we would be mistaken. Basing the interpretation on column percentages can be misleading.


and Computers I

Slide 22

Chi-square test of independence: APA Style - 1

A chi-square test of independence was performed to examine the relation between religion and college interest. The relation between these variables was significant, Χ² (2, N = 170) = 14.14, p <.01. Catholic teens were less likely to show an interest in attending college than were Protestant teens.

Χ² (2, N = 170) = 14.14, p <.01

Source: depts.washington.edu/psywc/handouts/pdf/stats.pdf

Degrees of freedom (from SPSS output)

Value of statistic

Significance of statistic

Number of valid cases


and Computers I

Slide 23

Chi-square test of independence : APA Style – Example 2

A chi-square test of independence indicated that there is a significant relationship between the size of the group and helping behavior, Χ² (2, N=52) = 7.91, p<.05. As show in Table 1, as

group size increased, helping behavior decreased.

Table 1 The percentage of observers who assisted at each group size

Observer Behavior

Group Size Helped Did not Help Total

(n = 31) (n = 21)

_________________________________________________

2 35% 9% n = 13

3 52% 48% n = 26

6 13% 43% n = 13 _________________________________________________

Source: www.ithaca.edu/faculty/alynn/Chi square independence.pdf


and Computers I

Slide 24

This problem uses the data set GSS2000R.Sav to compare the breakdown for the variable "should marijuana be made legal" [grass] for groups of survey respondents defined by the variable "general happiness" [happy]. Using a chi-square test of independence, is the following statement true, true with caution, false, or an incorrect application of a statistic? Use .05 as alpha and 1.96 as the critical value for the post hoc test.

Within the group of survey respondents who said that overall they were not too happy, there were significantly more who thought the use of marijuana should be made legal than would have been expected based on the breakdown of "should marijuana be made legal" by "general happiness".

o Trueo True with cautiono Falseo Incorrect application of a statistic

Sample homework problem: Chi-square test of independence

This is the general framework for the problems in the homework assignment on the chi-square test of independence. The description is similar to findings one might state in a research article.


and Computers I

Slide 25




Sample homework problem: Data set, variables, and criteria

The first paragraph identifies:

• The data set to use, e.g. GSS2000R.Sav• The independent variable that defines the

groups to be compared in the analysis • The dependent variable that is compared

across groups• The alpha level to use in the hypothesis test• The critical value used for the post hoc test


and Computers I

Slide 26




Sample homework problem: Specifications for the test

The second paragraph states the finding that we want to verify with the chi-square test of independence. The finding identifies:

• The specific group of the independent variable that is over or under represented for the stated category of the dependent variable

• The direction of the relationship, i.e. over-representation (significantly more) or under-representation (significantly fewer).


and Computers I

Slide 27




Sample homework problem: Choosing an answer

The answer to a problem will be True if the test of independence supports the finding in the problem statement. The answer to a problem will be False if the test of independence does not support the finding in the problem statement.

The answer to a problem will Incorrect application of a statistic if the Test of Independence violates the sample size requirement, i.e. no expected frequencies less than 1 and no more than 20% of the cells have expected frequencies less than 5.

Since it is legitimate to use ordinal variables in the chi-square test of independence, True with caution is not used for these problems.


and Computers I

Slide 28

Solving the problem with SPSS:Level of measurement

In the chi-square test of independence, the level of measurement for the independent and the dependent variable can be any level that defines groups (dichotomous, nominal, ordinal, or grouped interval). "Should marijuana be made legal" [grass] is dichotomous and "general happiness" [happy] is ordinal, so the level of measurement requirements are satisfied.


and Computers I

Slide 29

Solving the problem with SPSS: The chi-square test of independence- 1

To chi-square test of independence is computed for cross-tabulated tables.

Select Descriptive Statistics > Crosstabs… from the Analyze menu.


and Computers I

Slide 30


First, move the dependent variable grass to the Row(s) list box.

Second, move the independent variable happy to the Column(s) list box.

The finding we are trying to verify is:Within the group of survey respondents who said that overall they were not too happy, there were significantly more who thought the use of marijuana should be made legal than would have been expected based on the breakdown of "should marijuana be made legal" by "general happiness".

We first create a cross-tabulated table with the independent variable in the columns and the dependent variable in the rows.

Third, click on the Statistics button to add the chi-square test.


and Computers I

Slide 31


Second, click on the Continue button to close the dialog box.

First, mark the check box for Chi-square in the Crosstabs: Statistics dialog box.'.


and Computers I

Slide 32


When we return to the Crosstabs dialog box, we click on the Cells button to specify what we wanted to include in the cells of the crosstabs table.


and Computers I

Slide 33


First, we want the cells to show both the Observed and Expected counts for each cell, so we mark these check boxes.

Second, we mark the check box for Standardized residuals, which we will use for the post hoc test.

Third, we click on the Continue button to close the dialog box.


and Computers I

Slide 34


Having completed the specifications for the test, we click on the OK button to generate the output.


and Computers I

Slide 35

Solving the problem with SPSS: Sample size requirement

The sample size requirement for the chi-square test of independence states that none of the cells should have an expected frequency less than 1.0 and no more than 20% of the cells should have an expected frequency less than 5.

The information for verifying this requirement is found in the footnote to the chi-square tests table.

The minimum expected frequency in any cell was 5.24, which is larger than the minimum requirement of 1. None of the cells had an expected frequency less than 5.

The sample size requirements for the chi-square test are satisfied.

If the fail to meet the sample size requirements, the probability for the chi-square statistic may be inaccurate, so it would be an inappropriate application of the statistic.


and Computers I

Slide 36

Solving the problem with SPSS: Overall significance of the test

The chi-square test of independence supported the existence of a relationship between "general happiness" [happy] and "should marijuana be made legal" [grass], Chi-square (2, N = 169) = 17.14, p < .01.


This requires that the chi-square statistic for the table be statistically significant.

If the overall test is not significant, we do not examine the tests of individual cells.


and Computers I

Slide 37

Solving the problem with SPSS: Significance of the specific relationship

The standardized residual for the cell for survey respondents who said that overall they were not too happy and who thought the use of marijuana should be made legal was 3.00, which was greater than the critical value of 1.96.

The specific relationship satisfies the requirement for statistical significance.


This requires that the standardized residual is greater than the critical value.


and Computers I

Slide 38

Solving the problem with SPSS: Direction of the specific relationship - 1

Since the sign of the standardized residual was a plus, the statement that "there were significantly more who thought the use of marijuana should be made legal than would have been expected based on the breakdown of "should marijuana be made legal" by "general happiness"" is correct.


This requires that the standardized residuals is greater than the critical value.


and Computers I

Slide 39


We can verify the direction of the relationship by comparing the Count to the Expected Count.

Consistent with the proportion of subjects for General Happiness and Should Marijuana be Made Legal, we would have expected 5.2 in the specific combination for this problem. We actually had 12 subjects for this combination, more than we would have expected.


This requires that the standardized residuals is greater than the critical value.

The answer to the problem is True.


and Computers I

Slide 40


Alternatively, the problem could have been framed as: Within the group of survey respondents who said that overall they were not too happy, there were significantly fewer who thought the use of marijuana should not be made legal than would have been expected based on the breakdown of "should marijuana be made legal" by "general happiness“, since the standardized residual for that cell was -2.2. It is usually less awkward to state what people do think, rather than what they don’t think, though both statements would be statistically correct.

The finding stated in the problem was: Within the group of survey respondents who said that overall they were not too happy, there were significantly more who thought the use of marijuana should be made legal than would have been expected based on the breakdown of "should marijuana be made legal" by "general happiness".


and Computers I

Slide 41

Logic for homework problems: Chi-square Test of Independence - 1

Inappropriate application of

a statistic

Compute chi-square test of independence, requesting

standardized residuals

All expected frequencies ≥ 1?

Yes

No

Inappropriate application of

a statistic

No more than 20% of cells have

expected frequencies < 5?

Yes

No

Since SPSS output includes the sample size information, we conduct the test at our initial step.


and Computers I

Slide 42

Logic for homework problems: Chi-square Test of Independence - 2

Probability of the chi-square statistic ≤ alpha?

No

Yes

False

Standardized residual for target

cell ≥ critical value?

No

Yes

False

Correct interpretation of relationship (more

or less than expected)?

Yes

True

NoFalse

Documents

SW388R6 Data Analysis and Computers I Slide 1 Chi-square Test of Independence Reviewing the Chi-square Test of Independence Sample Homework Problem Solving