Week 6 ETEC 668 Quantitative Research in Educational Technology
Dr. Seungoh PaekFebruary 19, 2014
Tonight’s Agenda
Introduction to RStudio Continuing with SPSS – Cross-tabulation & Measures of Association
for Nominal & Ordinal Variables– Chi-Square & Other Nonparametric Tests
Introduction to Akamai Scenario Group Discussion for Research Paper
Agenda
Determining Research Design Breakout into Teams Cross-tabulation & Measures of Association
for Nominal & Ordinal Variables Chi-Square & Other Nonparametric Tests PSPP
A TASTE OF RSTUDIO
R
R is a free software environment for statistical computing and graphics.
Function
f(x) = y
RStudio
RStudio is a free and open source integrated development environment (IDE) for R, a programming language for statistical computing and graphics.
Review of Week 5
Probability Samples and Populations The Normal Curve Z-Score Hypothesis (Null Hypothesis vs. Research
Hypothesis) Hypothesis Testing
HYPOTHESIS TESTING
Hypothesis Testing
All events have a probability associated with them
p = your guess of chancep < .05
– .05 or 5% in Education and Psychology– 5% likelihood of results occurring by chance
alone
Error types
Type I– Reject H0 when you should not
Type II– Fail to reject H0 when you should
Error Table
The Real Situation
(Unknown to investigator)
Investigator’s Decision
H0 is True H0 is False
Reject H0 Type I errorCorrect decision
Do Not reject H0Correct decision
Type II error
Which error is better?
NASA engineers examine an electronic circuit.
A criminal court makes a decision as to whether or not Person A is guilty of murder.
SIGNIFICANCE
Statistical
Based on probability Research was technically successful H0 was rejected
P value– Education p < .05 = 5% chance– Medical p < .01 or .001 = 1% or .1% chance
Practical
Does it mean anything to the population?– Is that new treatment worth the cost?– Are my students really doing that much better?
Research Questions
vs
Research Hypotheses
Research Questions in Qualitative Research
Preferred when little is known about a phenomenon
Used when previous studies report conflicting results
Used to describe phenomena
Research Hypotheses for Quantitative Research
Educated guess or presumption based on literature
States the nature of the relationship between two or more variables
Predicts the research outcome Research study designed to test the
relationship described in the hypothesis
Null Hypotheses
Implicit complementary statement to the research hypothesis
States no relationship/difference exists between variables
Statistical test performed on the null Assumed to be true until support for the
research hypothesis is demonstrated
Alternative Hypotheses
Directional hypothesis– Precise statement indicating the nature and
direction of the relationship/difference between variables
Nondirectional hypothesis– States only that relationship/difference will
occur
Assessing Hypotheses
Simply stated? Single sentence? At least two variables? Variables clearly stated? Is the relationship/difference precisely
stated? Testable?
Types of Variables
Variable – Element that is identified in the hypothesis or
research question– Property or characteristic of people or things
that varies in quality or magnitude – Must be identified as independent or dependent
Independent Variables (IV)
Manipulation or variation of this variable is the cause of change in other variables
Technically, independent variable is the term reserved for experimental studies– Also called antecedent variable, experimental
variable, treatment variable, causal variable, predictor variable
Dependent Variables (DV)
The variable of primary interest Research question/hypothesis describes,
explains, or predicts changes in it The variable that is influenced or changed
by the independent variable– In non-experimental research, also called
criterion variable, outcome variable
Intervening or Mediating Variables
Intervening/Mediating variable– Presumed to explain or provide a link between
independent and dependent variables– Relationship between the IV and DV can only be
explained when the intervening variable is present
– E.g. effect of study prep on test scores– Organization of study ideas into a framework
(intervening/mediating)
Control Variables
Special type of IV that can potentially influence the DV
Use statistical procedures (e.g. analysis covariance) to control for these variables
May be demographic or personal variables that need to be “controlled” so that true influence of IV on DV can be determined
Confounding Variables
Confounding variable– Confuses or obscures the effect of independent
on dependent– Makes it difficult to isolate the effects of the
independent variable – Typically cannot be directly measured or
observed– Researchers comment on the influence after
study is completed
Relationship Between Independent and Dependent Variables
Cannot specify independent variables without specifying dependent variables
Number of independent and dependent variables depends on the nature and complexity of the study
The number and type of variables dictates which statistical test will be used
Model for Writing Descriptive Questions & Hypotheses
Identify IV, DV & any intervening/moderating variables
Specify descriptive questions for each IV, DV & intervening variable
Write inferential questions that relate variables or compare groups
Scenario
A researcher wants to study the relationship of critical thinking skills to student achievement in science classes for 8th-graders in a large metropolitan school district. The researcher controls for the effects of prior grades in science classes and parents’ educational attainment.
Step 1: Identify variables
What is the IV?
Step 1: Identify variables
What is the IV?- Critical thinking skills (measured on an
instrument)
Step 1: Identify variables
What is the DV?
Step 1: Identify variables
What is the DV?- Student achievement (measured by grades)
Step 1: Identify variables
What are the control variables?
Step 1: Identify variables
What are the control variables?– Prior grades in science class– Educational attainment of parents
Descriptive Questions
How do the students rate on critical thinking skills?
What are the students’ achievement grades in science classes?
What are the students’ prior grades in science classes?
What is the educational attainment of the parents of the 8th graders?
Inferential Questions
Does critical thinking ability relate to student achievement?
Does critical thinking ability relate to student achievement, controlling for the effects of prior grades in science and the educational attainment of the 8th-graders’ parents?
Cross-tabulation & Measures of Association for Nominal & Ordinal Variables
Cross-tabulation
Thus far, weʻve looked at univariate stats Descriptive stats - summarizes the distribution of
a single variable (central tendency/dispersion) Time for bivariate analysis of nominal/ordinal
variables - explore relationship between two categorical variables
Cross-tab – a table or matrix that shows the distribution of one variable for each category of a second variable
Let’s investigate
What’s the relationship between race (race) & view on capital punishment/death penalty for murder (cappun)?
SPSS commands
Open “DEMO.sav” file Analyze → Descriptive Statistics → Crosstab
Recommendation:- Choose IV as column variable (race)- Select DV as row variable (cappun)
SPSS Output
Let’s add %
SPSS Output
Let’s investigate
What’s the relationship between race (race) & view on gun permits (gunlaw)?
PSPP
PSPP commands
Analyze → Descriptive Statistics → Crosstab
Recommendation:– Choose IV as column variable (race)– Select DV as row variable (cappun)
PSPP Output
Where are
they coming
from?
Missing Data
Go to “Variable View” Find the row for “cappun.” Specify values for missing data
PSPP Output
Measures of Association
Measures of association – summarizes the strength of association between 2 variables
When to use which test…
Level of Measurement
Statistics for Measuring
Nominal Ordinal I/R
Central Tendency
Mode Median Mean
Dispersion/Variability
- RangeVarianceStd. Dev.
Association Lambda Gamma Pearson r2
Tests of Significance
Chi-Square Chi-SquareT-TestANOVA
Chi-Square & Other Nonparametric Tests
Introduction
Parametric statistics have certain assumptions– Variances of each group are similar– Sample is large enough to represent the
population Nonparametric statistics donʻt require the
same assumptions– Allow data that comes in frequencies to be
analyzed…they are “distribution free”– Allow nominal/ordinal data to be analyzed
One-Sample Chi-Square
Chi-square allows you to determine if what you observe in a distribution of frequencies is what you would expect to occur by chance.– One-sample chi-square (goodness of fit test) only has
one dimension– Two-sample chi-square has two dimensions – to test
differences between frequencies (nominal data), i.e. how likely is the observed differences between 2 groups were created by random sampling errors
Computing Chi-Square
What do those symbols mean?
More Hypotheses
Null hypothesis
H0: P1 = P2 = P3
Research hypothesis
H1: P1 ≠ P2 ≠ P3
Computing Chi Square
Category O E D (O-E)2 (O-E)2/2
For 23 30 7 49 1.63
Maybe 17 30 13 169 5.63
Against 50 30 20 400 13.33
Total 90 90 x2 = 20.6
So How Do I Interpret…
x2(2) = 20.6, p < .05– x2 represents the test statistic– 2 is the number of degrees of freedom– 20.6 is the obtained value– p < .05 is the probability
ONE-SAMPLE CHI SQUARE
SPSS Commands
One One-Sample Chi-Square using SPSS Analyze Nonparametric Tests Legacy
Dialogs Chi-Square
SPSS OUTPUT This is your n total
Size. This is your observed
test statistic. This is your degrees of
freedom (df). This is your
significance level. You will report this number as part of your statistical decision in the results section.
TWO-SAMPLE CHI-SQUARE
SPSS commands
Open “DEMO.sav” file Analyze → Descriptive Statistics → Crosstab
Recommendation:- Choose IV as column variable (race)- Select DV as row variable (cappun)
One more step!
SPSS Output
PSPPChi-Square
PSPP Commands
One-Sample Chi Square using PSPP Analyze → Nonparametric Tests → Chi-Square
PSPP commands
Two-Sample Chi Square using PSPP Analyze Descriptive Statistics Crosstab
PSPP Output(Two-Sample Chi-Square)
PSPP Output(Two-Sample Chi-Square)
Other Nonparametric
Tests
RESEARCH GROUP WORK
7-part Model for Conceptualizing Quantitative Ed Tech Research
1. Select a Topic2. Identify the Research Problem3. Conduct a Literature Review4. State the Research questions and hypotheses5. Determine the Research Design6. Determine the Methods7. Identify Data Analysis Procedures
What to do Week 6
1. Do the required readings for Week 07.– Salkind, N. J. Chapter 15. Predicting Who’ll Win the Super Bowl: Using
Linear Regression
2. Continue the group discussion on the final research paper.
3. Do Akamai Consulting Scenario Task 1 (more information TBA)