More Validity

And some reliability

Today’s class

• Check in

• Validity

• In class exercise

• Reliability

Check in

• Quiz

• 15 questions

• Highest score was 14 (2 of those) so all scores are out of 14

• I added one point to all quizzes to adjust for difficulty

• E.g. if you had 9/15 in raw points your quiz score would be 10/14

• Article 2/Review exercises due Thursday October 13th

• Midterm exam Tuesday October 18th

– Multiple choice– Short answer– Abstract exercise

More validity

• Construct validity

• Content validity

• Face validity

• Concurrent validity

• Predictive validity

• Discriminant validity

• Convergent validity

Construct validity gone awry

• Homosexuality– Classified as a mental disorder by APA until

1973– Freud believed it was an immature stage in

sexual development (not an illness)• Due to overidentification with the mother

– Other psychoanalysts argued it was an illness due to overinvolved mother and rejecting father behavior

More validity

• Content validity– Does the test or measure have the content

that represents the construct– Does it have content that doesn’t represent

the construct

• Content validity of a measure of autism?– What should it include?

Concurrent validity

• Is the measure associated with other related qualities or events that would occur simultaneously

• Concurrent validity of the SAT?– What would you expect it to be related to?

Predictive Validity

• Does the measure predict, at a later date, to something that should be related?

• Your SAT scores and…?

• What are your college grades a measure of?

• What sort of predictive validity will they have?

Convergent Validity

• Is the measure correlated with other similar measures?

• If it’s a measure of racism, will it be correlated with other measures of prejudice and discrimination?

Discriminant Validity

• Does the measure NOT correlate with things it should NOT correlate with?

• Not correlate is not the same as negatively correlated

• Can be hard to establish• E.g. a measure of math skills should be

uncorrelated with a measure of verbal skills– If we believe these are unrelated constructs

Face Validity

• Does it seem to measure what it is measuring

• This can be good or bad– Good if it helps people zero in on the

questions– Bad if measuring something people don’t

want to be honest about-like racist attitudes

• Good face validity does not mean a measure is good

In Class exercise

• Come up with a research idea for the topic area you have been given—anything you want!!

• Identify two variables or more

• Say how you might measure them

Reliability

• Is a measure or instrument consistent?– Across time– Across observers/raters– Across questions

• Without reliability there is no validity

Test-Retest Reliability

• Is it consistent over time?

• Does NOT mean scores should be “identical” over time

• Means relative rank order of each person in a group should be similar across time points

Test-retest reliability• E.g.

• The Hare Psychopathy checklist– A group of people takes it twice– We see if their scores are related over time

• The Beck depression inventory– A group of people takes it twice– We see if their scores are related over time

• How long a time?– Depends on the construct

Test-retest reliability

• Measured with the correlation coefficient R

• R is a measure of the relationship between two continuous (or rank ordered) quantitative variables– E.g. Height and weight

The correlation coefficient R

• A number between -1 and +1

• Closer to 1 or -1 is stronger relationship

• Positive correlation: if one variable has high scores, the other also tends to have high scores– And if it has low scores, the other also does

• Negative correlation: when one has high scores, the other has low scores

Positive Correlation

Negative Correlation

No correlation

Test –retest reliability

• Is calculated using the correlation coefficient R

• If someone has high scores on a depression inventory one week, we expect the same thing again two weeks later– Relative to other people who took it– If the measure has reliability over time

Test-retest reliability

• Key for most surveys of mental illness, personality traits, intelligence, etc.

• We expect some stability in these qualities

• If we don’t expect stability, test retest reliability not necessary or useful

• But if we expect stability and don’t have it– Then our measure is bad

• How much stability?– Usually at least .70 or higher, depending on

time length