View
283
Download
2
Tags:
Embed Size (px)
Citation preview
Errors in measurement Reliability
If you measure the same thing twice do you get the same values?
Validity Does your measure really measure what it is supposed to measure??
reliablevalid
reliable
invalid
unreliable invalid
Reliability
True score + measurement error A reliable measure will have a small amount of error
Multiple “kinds” of reliability• Test-retest• Internal consistency• Inter-rater
Reliability
Test-restest reliability Test the same participants more than once• Measurement from the same person at two different times
• Should be consistent across different administrationsReliable Unreliable
Reliability
Internal consistency reliability Multiple items testing the same construct
Extent to which scores on the items of a measure correlate with each other•Cronbach’s alpha (α)•Split-half reliability
• Correlation of score on one half of the measure with the other half (randomly determined)
Reliability
At least 2 raters observe behavior
Inter-rater reliability
Extent to which raters agree in their observations• Are the raters consistent?
Requires some training in judgment
Validity
Does your measure really measure what it is supposed to measure? There are many “kinds” of validity
VALIDITY
CONSTRUCT
CRITERION-ORIENTED
DISCRIMINANT
CONVERGENTPREDICTIVE
CONCURRENT
FACE
INTERNAL EXTERNAL
Many kinds of Validity
VALIDITY
CONSTRUCT
CRITERION-ORIENTED
DISCRIMINANT
CONVERGENTPREDICTIVE
CONCURRENT
FACE
INTERNAL EXTERNAL
Many kinds of Validity
Construct Validity
Usually requires multiple studies, a large body of evidence that supports the claim that the measure really tests the construct
Face Validity At the surface level, does it look as if the measure is testing the construct?
“This guy seems smart to me, and
he got a high score on my IQ measure.”
Internal Validity
Did the change in the DV result from the changes in the IV or does it come from something else?
The precision of the results
Threats to internal validity
History – an event happens the experiment Maturation – participants get older (and
other changes) Selection – nonrandom selection may lead to
biases Mortality – participants drop out or can’t
continue Testing – being in the study actually
influences how the participants respond
The precision of the results
External Validity
Are experiments “real life” behavioral situations, or does the process of control put too much limitation on the “way things really work?”
External Validity Variable representativeness
Relevant variables for the behavior studied along which the sample may vary
Setting representativeness Are the properties of the research setting similar to those outside the lab (Ecological validity)
Subject representativeness Characteristics of sample and target population along these relevant variables
Sampling
Why do we do we use sampling methods? Typically don’t have the resources to test everybody, so we test a subset
Sampling
Population
Everybody that the research is targeted to be about
The subset of the population that actually participates in the research
Sample
Sampling
Sample
Inferential statistics used to generalize back
Sampling to make data collection manageable
Population
Sampling Why do we do we use sampling methods?
Goals of “good” sampling:– Maximize Representativeness:
– To what extent do the characteristics of those in the sample reflect those in the population
– Reduce Bias:– A systematic difference between those in
the sample and those in the population
Sampling Methods Probability sampling
Simple random sampling
Systematic sampling Stratified sampling
Non-probability sampling Convenience sampling Quota sampling
Have some element of random selection
Susceptible to biased selection
Simple random sampling Every individual has a equal and independent chance of being selected from the population
Quota sampling Step 1: identify the specific subgroups Step 2: take from each group until desired number of individuals