Psych 818Psych 818Psychological Psychological MeasurementMeasurement

DeShon - IntroductionDeShon - Introduction

2

PreliminariesPreliminaries

Who am I?Who am I?

Your job - Ask questions and find a way to Your job - Ask questions and find a way to become intellectually engagedbecome intellectually engaged Self-handicapping is unproductiveSelf-handicapping is unproductive You have a huge role in the success of the classYou have a huge role in the success of the class

All work is your own!All work is your own!

Syllabus and GradingSyllabus and Grading

3

PurposePurpose

Measurement TheoryMeasurement Theory Measure DevelopmentMeasure Development Measure UseMeasure Use Measure InterpretationMeasure Interpretation

Exposure to Measurement thinkingExposure to Measurement thinking Exposure to Measurement papersExposure to Measurement papers Exposure to Measurement StandardsExposure to Measurement Standards

4

Responsibility!Responsibility!

As a Psychologist As a Psychologist YOUYOU will likely have will likely have substantial effects on the course of many substantial effects on the course of many lives through the use of tests and lives through the use of tests and measuresmeasures Clinical DiagnosisClinical Diagnosis Policy Making & Risk FactorsPolicy Making & Risk Factors Educational OpportunitiesEducational Opportunities Work OpportunitiesWork Opportunities

Example: Police promotion & expert witnessExample: Police promotion & expert witness

You will have the power to help or hurt You will have the power to help or hurt many people (and their families!)many people (and their families!)

Measurement & PolicyMeasurement & Policy

Measurement often starts as an Measurement often starts as an academic investigationacademic investigation

Then, over time, transfers to policy Then, over time, transfers to policy and decision makingand decision making

Examples:Examples: The Kilogram conundrumThe Kilogram conundrum MPG vs GPMMPG vs GPM

6

Standards for MeasurementStandards for Measurement

APA standards for educational and APA standards for educational and psychological testing and psychological testing and assessmentassessment This is your professional bible.This is your professional bible. You will be held professionally You will be held professionally

accountable to these standardsaccountable to these standards Therefore, you better know them!Therefore, you better know them!

7

Beginning QuestionsBeginning Questions

Why do we measure?Why do we measure?

What is measurement?What is measurement?

8

Why do we measure?Why do we measure?

Decision makingDecision making

PredictionPrediction

ResearchResearch

9

What is Measurement?What is Measurement?

S.S. StevensS.S. Stevens The assignment of numerals to objects or events The assignment of numerals to objects or events

according to a rule.according to a rule. Warren TorgersonWarren Torgerson

The assignment of numbers to The assignment of numbers to perceived perceived attributesattributes of objects or events according to a of objects or events according to a rule.rule.

Lyle JonesLyle Jones Measurement is the determination of the Measurement is the determination of the

magnitude of a specified attribute of an object or magnitude of a specified attribute of an object or event in terms of a unit of measurement.event in terms of a unit of measurement.

Classification (including ordering) is not Classification (including ordering) is not measurement.measurement.

10

What is Measurement?What is Measurement?

Cliff’s definitionCliff’s definition GoodGood measurement is the measurement is the

assignment of numbers to perceived assignment of numbers to perceived attributes of objects or events attributes of objects or events according to rules that according to rules that are easily understood; are easily understood; are easily used; are easily used; yield numbers that are as simply related yield numbers that are as simply related

as possible to as many other sets of as possible to as many other sets of measurements as possible. measurements as possible.

11

Measurement QuestionsMeasurement Questions

Fundamental measurement questionsFundamental measurement questions What is our best estimate of a What is our best estimate of a

psychologically relevant quantitypsychologically relevant quantity ValidityValidity

How precise is our estimate or...How precise is our estimate or... How much error is there in our estimateHow much error is there in our estimate

ReliabilityReliability

12

Analogy with statisticsAnalogy with statistics

Measurement questions look a lot like Measurement questions look a lot like statistical questionsstatistical questions What's our best estimate of a population What's our best estimate of a population

meanmean How precise is our estimate (e.g., How precise is our estimate (e.g.,

standard errors and confidence intervals)standard errors and confidence intervals)

13

History of MeasurementHistory of Measurement

Most early measurements occurred in Most early measurements occurred in physics, chemistry, and much later in physics, chemistry, and much later in biology (e.g., bacteria)biology (e.g., bacteria)

Much attention paid to measuring Much attention paid to measuring specific quantities such as specific quantities such as temperature, pressure, pH, etc...temperature, pressure, pH, etc... But no serious attention given to a But no serious attention given to a

general theory of measurementgeneral theory of measurement Also, little attention paid to a general Also, little attention paid to a general

model of reliability and validitymodel of reliability and validity

14

Two Physical Measurement Two Physical Measurement Examples Examples

TemperatureTemperature Fundamental to virtually all future Fundamental to virtually all future

discoveries in physicsdiscoveries in physics

PHPH Fundamental to the understanding of the Fundamental to the understanding of the

basic elements (i.e., hydrogen) and basic elements (i.e., hydrogen) and chemistrychemistry

15

TemperatureTemperature

In the beginning there was the PERCEPTION of In the beginning there was the PERCEPTION of hot and coldhot and cold

Aristotle’s fundamental qualitiesAristotle’s fundamental qualities Hot and cold were conceived of as opposing qualities Hot and cold were conceived of as opposing qualities

of objectof object Virtually all measurements are rooted in subjective Virtually all measurements are rooted in subjective

perceptionperception Perceptions are fallible!Perceptions are fallible!

200BC Philo’s air thermoscope200BC Philo’s air thermoscope

16

Temperature (cont.)Temperature (cont.)

1592 Galileo reinvents 1592 Galileo reinvents Philo’s (200BC) air Philo’s (200BC) air thermoscope (instrument thermoscope (instrument or indicator)or indicator) This is an indicator with no This is an indicator with no

scalescale Not a measurement?Not a measurement?

Very sensitive to Very sensitive to temperature changestemperature changes

But also open to the air so But also open to the air so sensitive to other variables sensitive to other variables (e.g., Air Pressure)(e.g., Air Pressure)

validity problemvalidity problem

17

Problems with Air ThermoscopesProblems with Air Thermoscopes

No scale – so a qualitative instrumentNo scale – so a qualitative instrument Construct contamination - ValidityConstruct contamination - Validity

open to the air so sensitive to other open to the air so sensitive to other variables (e.g., Air Pressure)variables (e.g., Air Pressure)

What liquid to use? (water, spirits, What liquid to use? (water, spirits, quicksilver)quicksilver) State changes, sensitivity and linearity of State changes, sensitivity and linearity of

expansion ratesexpansion rates No standardized construction so other No standardized construction so other

instruments are not equivalent instruments are not equivalent Inaccurate conceptualization of heatInaccurate conceptualization of heat

18

Fix the validity problemsFix the validity problems

Seal the thermoscopeSeal the thermoscope Duke Ferdinand II (1632)Duke Ferdinand II (1632) Pascal (1663) proved that air pressure Pascal (1663) proved that air pressure

caused errors in open air thermometerscaused errors in open air thermometers Are measurement errors reliability or validityAre measurement errors reliability or validity

Use quicksilver (i.e., mercury)Use quicksilver (i.e., mercury) FareinheitFareinheit

19

Fix the scaling problemsFix the scaling problems Many individuals developed scales to assign numbers to the Many individuals developed scales to assign numbers to the

level of liquid in the level of liquid in the thermometerthermometer (now a thermo – meter)! (now a thermo – meter)! Newton (1700)Newton (1700)

""zeroth degree of heat" as melting snow and "33 degrees of zeroth degree of heat" as melting snow and "33 degrees of heat" as boiling waterheat" as boiling water

Rømer (1708)Rømer (1708) RéaumurRéaumur (1731) (1731)

freezing point of water is 0 degrees, the boiling point 80 freezing point of water is 0 degrees, the boiling point 80 degrees degrees

Fareinheit (1724)Fareinheit (1724) 0 as the melting point of an equal mixture of ice and salt and 0 as the melting point of an equal mixture of ice and salt and

96 as blood heat96 as blood heat Delisle (1732)Delisle (1732)

2400 graduations: 0 degrees as the boiling point and 150 2400 graduations: 0 degrees as the boiling point and 150 degrees as the freezing point of water. degrees as the freezing point of water.

Celcius (1742)Celcius (1742) freezing point of water at 0 °C and the boiling point at 100 °C freezing point of water at 0 °C and the boiling point at 100 °C

20

Proliferation of ScalesProliferation of Scales

21

UnitsUnits

These scales result in “degrees” or These scales result in “degrees” or unitsunits

What are some psychological units?What are some psychological units?

Check out these physical units!Check out these physical units! http://www.unc.edu/~rowlett/units/http://www.unc.edu/~rowlett/units/

22

Comparison of temp. scalesComparison of temp. scales

290944211823−814010440998055265800The surface of the Sun

8831334550−23523494303416681941Titanium melts

6080330671.67212100373.15Water boils (at standard pressure)

26.92529.612.2194.5557.998.236.8310.0Average human body temperature

7.500150491.67320273.15Water freezes (at standard pressure)

−1.83−14.22−5.87176.67459.670−17.78255.37Fahrenheit's ice/salt mixture

−135.90−218.52−90.14²559.7250−459.67−273.150Absolute zero

RømerRéaumurNewtonDelisleRankineFahrenheitCelsiuskelvin

23

Transformations to Equate scalesTransformations to Equate scales

[K] = 373.15 − [°De] · 2/3[°De] = (373.15 − [K]) · 3/2Delisle

[K] = ([°Rø] − 7.5) · 40/21 + 273.15[°Rø] = ([K] − 273.15) · 21/40 + 7.5Rømer

[K] = [°N] · 100/33 + 273.15[°N] = ([K] − 273.15) · 33/100Newton

[K] = [°Ré] · 5/4 + 273.15[°Ré] = ([K] − 273.15) · 4/5Réaumur

[K] = [°Ra] · 5/9[°Ra] = [K] · 9/5Rankine

[K] = ([°F] + 459.67) · 5/9[°F] = [K] · 9/5 − 459.67Fahrenheit

[K] = [°C] + 273.15[°C] = [K] − 273.15Celsius

24

Fix the standardization or Fix the standardization or calibration problemscalibration problems

The technology of glass blowing The technology of glass blowing made it very difficult to construct made it very difficult to construct exactly equal instruments (the size of exactly equal instruments (the size of the liquid resivoir, the amount of the liquid resivoir, the amount of liquid, the bore of the tube, etc…all liquid, the bore of the tube, etc…all must be identical to yield identical must be identical to yield identical readings.readings.

Instead, calibrate or equate using Instead, calibrate or equate using fixed reference pointsfixed reference points

Royal Society thermometersRoyal Society thermometers

25

Theoretical UnderstandingTheoretical Understanding

The various scales of measurement and The various scales of measurement and the calibration of thermometers to fixed the calibration of thermometers to fixed points allowed systematic investigationspoints allowed systematic investigations

That resulted in improved understanding That resulted in improved understanding of what thermometers measure (heat)of what thermometers measure (heat)

Kelvin’s scale of temperature (1848)Kelvin’s scale of temperature (1848) Rational scale with absolute zeroRational scale with absolute zero Based on gas laws and carnot cyclesBased on gas laws and carnot cycles Adopted in 1954!Adopted in 1954!

26

Example 2: pH - Acids & BasesExample 2: pH - Acids & Bases

In the beginning there was … In the beginning there was … perceptionperception Greeks attempted to characterize natureGreeks attempted to characterize nature One fundamental character of nature One fundamental character of nature

was taste.was taste. SaltySalty Sour Sour

the greek word for sour is the base of our word the greek word for sour is the base of our word for acidfor acid

BitterBitter SweetSweet

27

Acids and BasesAcids and Bases

Next, it turned out that sour tasting Next, it turned out that sour tasting substances had other regular substances had other regular propertiesproperties changes the color of litmus (a dye changes the color of litmus (a dye

extracted from a lichen commonly extracted from a lichen commonly known as dyer's weed.) known as dyer's weed.)

Corrodes metalsCorrodes metals

No real understanding of acidsNo real understanding of acids

28

AcidsAcids

Huge proliferation of acid types (named by Huge proliferation of acid types (named by the method used to form the acid)the method used to form the acid) Muratic acid, acetic acid, sulphuric acid, Muratic acid, acetic acid, sulphuric acid,

hydrofluoric acid, phosphoric acid, pyruvic acidhydrofluoric acid, phosphoric acid, pyruvic acid

No clear picture of the cause for the No clear picture of the cause for the common functioning of acidscommon functioning of acids LavoisierLavoisier (1776) thought it was oxygen that (1776) thought it was oxygen that

caused substances to be acidiccaused substances to be acidic Greek word Greek word oxeinoxein meaning 'sour' and meaning 'sour' and gennangennan

meaning 'to generatemeaning 'to generate

29

AcidsAcids

Leibig (1850s) noticed that all acids Leibig (1850s) noticed that all acids have hydrogen in commonhave hydrogen in common This insight required knowledge of the This insight required knowledge of the

elements, molecules, and weights of elements, molecules, and weights of molecules (another measurement molecules (another measurement problem)problem)

Sorenson (1909) – developed the Sorenson (1909) – developed the scale for indexing the activity of scale for indexing the activity of hydrogen in a substancehydrogen in a substance

30

AcidsAcids

pH = (power of hydrogen ) is a pH = (power of hydrogen ) is a measure of the activity of hydrogen measure of the activity of hydrogen ions (H+) in a solutionions (H+) in a solution

Log of number of HLog of number of H+ molecules in a molecules in a substance substance

pH value has no unit but it is not an pH value has no unit but it is not an arbitrary scalearbitrary scale

The number arises from a definition based The number arises from a definition based on the activity of hydrogen ions in the on the activity of hydrogen ions in the solution – has an absolute zerosolution – has an absolute zero

31

Measurement of pHMeasurement of pH

Many ways now exist to measure pH.Many ways now exist to measure pH. Early measurements were based on the Early measurements were based on the

color of a substance resulting from a color of a substance resulting from a chemical reactionchemical reaction

pH indicatorpH indicator A A pH indicatorpH indicator is a is a halochromichalochromic chemical chemical

compoundcompound that is added in small amounts to a that is added in small amounts to a solutionsolution so that the so that the pHpH of the solution can be of the solution can be determined easily. Hence a pH indicator is a determined easily. Hence a pH indicator is a chemicalchemical detector for detector for protonsprotons ( (HH+). Normally, +). Normally, the indicator causes the the indicator causes the colorcolor of the solution to of the solution to change depending on the pH.change depending on the pH.

-Wilkpedia -Wilkpedia

32

pH MeasurementpH Measurement

yellow11.4-13.0blueIndigo carmine

orange-red10.1-12.0yellowAlizarin Yellow R

blue9.4-10.6colorlessThymolphthalein

pink8.2-10.0colorlessPhenolphthalein

blue8.0-9.6yellowThymol blue

red6.6-8.0yellowPhenol red

blue4.5-8.3redLitmus (Azolitmin)

yellow4.2-6.3redMethyl red

red3.0-5.2blueCongo red

yellow2.9-4.0redMethyl yellow

yellow1.2-2.8redThymol blue

blue-violet0.0-1.6yellowMethyl violet

Color - high pH

Transition pH rangeColor - low pH

Indicator

33

pH MeasurementpH Measurement

Adding the indicator does not result Adding the indicator does not result in an absolute shift from one color to in an absolute shift from one color to the other.the other.

Instead, it is a gradual shift and the Instead, it is a gradual shift and the resulting color must be compared to resulting color must be compared to a SCALE to determine the pH.a SCALE to determine the pH.

34

Scaling pH IndicatorsScaling pH Indicators

35

pH and IRT similaritiespH and IRT similarities

The colormetric approach to pH has a The colormetric approach to pH has a great deal of overlap with current great deal of overlap with current Item Response approaches to Item Response approaches to measurement – IRTmeasurement – IRT

Keep this in mind when we get to IRTKeep this in mind when we get to IRT

36

Causality and MeasurementCausality and Measurement

Measurement requires causal Measurement requires causal inferenceinference Changes in the level of the latent entity Changes in the level of the latent entity

cause changes in the level of the cause changes in the level of the indicator variable(s).indicator variable(s).

This inference requires the same This inference requires the same scientific method as any causal research scientific method as any causal research question.question.

Must show that the latent variable is the Must show that the latent variable is the only cause of the indicator (construct only cause of the indicator (construct validity)validity)

37

ConstructsConstructs

What is a construct?What is a construct? Constructs become better defined via Constructs become better defined via

researchresearch Temperature, pH, bacteria, gas pressure, Temperature, pH, bacteria, gas pressure,

electricityelectricity 1. An unobserved 1. An unobserved causecause of variation of variation

in an observable variable(s)in an observable variable(s) 2. A label used to describe a pattern 2. A label used to describe a pattern

of observed covariancesof observed covariances

38

Model of MeasurementModel of Measurement

Construct

Construct = an unobservable but real causal variable(aka latent variable)

Indicator / Instrument

Indicator = an observable variable thatis solely caused by the construct andsensitive to changes in the level of the construct

Scale: the rule for assigning numbers to the levels of theindicator or instrument

39

Basic Measurement ModelBasic Measurement Model

Construct

Measure 1 Measure 2 Measure 3

Error 1 Error 2 Error 3

40

Some quotes to think aboutSome quotes to think about "I often say that when you can measure what you are "I often say that when you can measure what you are

speaking about, and express it in numbers, you know speaking about, and express it in numbers, you know something about it; but when you cannot measure it, something about it; but when you cannot measure it, when you cannot express it in numbers, your when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind: it knowledge is of a meagre and unsatisfactory kind: it may be the beginning of knowledge, but you have may be the beginning of knowledge, but you have scarcely, in your thoughts, advanced to the stage of scarcely, in your thoughts, advanced to the stage of science, whatever the matter may be.“ - Sir William science, whatever the matter may be.“ - Sir William Thomson, Lord Kelvin. 1889 Thomson, Lord Kelvin. 1889

““Whatever exists at all, exists in some amount. To Whatever exists at all, exists in some amount. To know it thoroughly involves knowing its quality as know it thoroughly involves knowing its quality as well as its quantity” - Thorndike, 1918well as its quantity” - Thorndike, 1918