Scaling DeShon - 2005. What is Scaling? Young’s (1984) definition Young’s (1984) definition Scaling is the branch of measurement that involves the construction

ScalingScaling

DeShon - 2005DeShon - 2005

What is Scaling?What is Scaling?

Young’s (1984) definitionYoung’s (1984) definition Scaling is the branch of Scaling is the branch of

measurement that involves the measurement that involves the construction of an instrument that construction of an instrument that associates qualitative constructs with associates qualitative constructs with quantitative metric units. quantitative metric units.

Ex: Attitude toward ImmigrationEx: Attitude toward ImmigrationThe assignment…

…of objects… …to numbers…

Would you let immigrants to live in your country?

Would you let immigrants to live in your neighborhood?

Would you let your child to marry an immigrant?

…according to a consistent rule.

Scaling and the RuleScaling and the Rule

Scaling is the method for determining the Scaling is the method for determining the rulerule

A response scale (e.g., true-false, 1-5 A response scale (e.g., true-false, 1-5 agree scale,etc…) is not scaling.agree scale,etc…) is not scaling. A response scale is just a way to collect data A response scale is just a way to collect data

from people using an instrumentfrom people using an instrument This just attaches a response to an object or This just attaches a response to an object or

statementstatement No consistent rule is usedNo consistent rule is used

scaling involves procedures that you do scaling involves procedures that you do independent of the respondent so that you can independent of the respondent so that you can come up with a numerical value for the object. come up with a numerical value for the object.

Major Scaling MethodsMajor Scaling Methods

Thurstone or Equal Appearing Thurstone or Equal Appearing IntervalsIntervals

Likert or Summative ScalingLikert or Summative Scaling Guttman or Cummulative ScalingGuttman or Cummulative Scaling Mokken or Probabilistic Cummulative Mokken or Probabilistic Cummulative

ScaleScale

Thurstone ScalingThurstone Scaling

Actually created 3 different ways of Actually created 3 different ways of producing a unidimensional scaleproducing a unidimensional scale Equal-Appearing intervals (easiest)Equal-Appearing intervals (easiest) Successive intervalsSuccessive intervals Paired comparisonsPaired comparisons

Example adapted from Example adapted from (http://www.socialresearchmethods.net/kb/scalthur.htm)(http://www.socialresearchmethods.net/kb/scalthur.htm)

Equal Appearing IntervalsEqual Appearing Intervals

StepsSteps1.1. State focus of scaling attempt and State focus of scaling attempt and

definition of the focal variable (e.g., definition of the focal variable (e.g., attitudes toward individuals with AIDS)attitudes toward individuals with AIDS)

2.2. Generate potential scale items used to Generate potential scale items used to elicit attitudeselicit attitudes

- Large set of initial items- Similar wording

Item examplesItem examples People get AIDS by engaging in immoral behaviorPeople get AIDS by engaging in immoral behavior AIDS is the wrath of GodAIDS is the wrath of God AIDS is an epidemic that affects us allAIDS is an epidemic that affects us all People with AIDS are badPeople with AIDS are bad People with AIDS are real peoplePeople with AIDS are real people I treat everyone the same, regardless of whether or not they have AIDSI treat everyone the same, regardless of whether or not they have AIDS AIDS costs the public too muchAIDS costs the public too much AIDS is a death sentenceAIDS is a death sentence AIDS is preventableAIDS is preventable People who contract AIDS deserve itPeople who contract AIDS deserve it AIDS doesn't have a preference, anyone can get it.AIDS doesn't have a preference, anyone can get it. AIDS is the worst thing that could happen to you.AIDS is the worst thing that could happen to you. AIDS is helps control the population.AIDS is helps control the population. People with AIDS do not deserve our helpPeople with AIDS do not deserve our help AIDS will never happen to meAIDS will never happen to me People with AIDS deserve what they gotPeople with AIDS deserve what they got AIDS distracts from other diseases that deserve our attention moreAIDS distracts from other diseases that deserve our attention more People with AIDS deserve our understanding, but not necessarily special treatmentPeople with AIDS deserve our understanding, but not necessarily special treatment People with AIDS should be quarantined to protect the rest of societyPeople with AIDS should be quarantined to protect the rest of society Everyone affected with AIDS deserves it due to their lifestyleEveryone affected with AIDS deserves it due to their lifestyle Someone with AIDS could be just like meSomeone with AIDS could be just like me People infected with AIDS did not have safe sexPeople infected with AIDS did not have safe sex People with AIDS should be treated just like everybody else.People with AIDS should be treated just like everybody else.


Steps (cont)Steps (cont)1.1. Participants rate the items in terms of the Participants rate the items in terms of the

favorability of the attitude toward favorability of the attitude toward individuals with AIDS.individuals with AIDS.

- Usually based on a - Usually based on a 1-11 scale 1-11 scale (1=unfavorable, (1=unfavorable, 11=favorable)11=favorable)


StepsSteps1.1. Graph the distribution of ratings for each Graph the distribution of ratings for each

itemitem

2.2. Compute the median and interquartile range Compute the median and interquartile range for each itemfor each item

Item statsItem stats

451348

451333

1.52.51243

1.52.51236

242244

231224

2.53.51242

2.53.51216

341245

451227

011151

121139

121134

121112

12118

1.52.51123

Interquartile Range

Q3Q1MedianStatement

Number

Final stepFinal step

Select final scale itemsSelect final scale items select statements that are at equal select statements that are at equal

intervals across the range of medians intervals across the range of medians (e.g., one for each rating level, 1-11)(e.g., one for each rating level, 1-11)

try to select the statement that has the try to select the statement that has the least amount of variability across judges least amount of variability across judges (smallest Interquartile Range )(smallest Interquartile Range )

Final scaleFinal scale For instance:For instance:

People with AIDS are like my parents (6)People with AIDS are like my parents (6) Because AIDS is preventable, we should focus our Because AIDS is preventable, we should focus our

resources on prevention instead of curing (5)resources on prevention instead of curing (5) People with AIDS deserve what they got. (1)People with AIDS deserve what they got. (1) Aids affects us all (10)Aids affects us all (10) People with AIDS should be treated just like everybody People with AIDS should be treated just like everybody

else. (11)else. (11) AIDS will never happen to me. (3)AIDS will never happen to me. (3) It's easy to get AIDS (5)It's easy to get AIDS (5) AIDS doesn't have a preference, anyone can get it (9)AIDS doesn't have a preference, anyone can get it (9) AIDS is a disease that anyone can get if they are not AIDS is a disease that anyone can get if they are not

careful (9)careful (9) If you have AIDS, you can still lead a normal life (8)If you have AIDS, you can still lead a normal life (8) AIDS is good because it helps control the population. (2)AIDS is good because it helps control the population. (2) I can't get AIDS if I'm in a monogamous relationship. (4)I can't get AIDS if I'm in a monogamous relationship. (4)

Details…Details… The value in parentheses after each The value in parentheses after each

statement is its scale value. statement is its scale value. Random order of the statements with Random order of the statements with

respect to scale values. respect to scale values. Notice that there is no item with scale Notice that there is no item with scale

value of 7 and that we have two with value of 7 and that we have two with values of 5 and of 9 (one of these pairs will values of 5 and of 9 (one of these pairs will average out to a 7).average out to a 7).

You now have a scale -- a yardstick you You now have a scale -- a yardstick you can use for measuring attitudes towards can use for measuring attitudes towards people with AIDS. people with AIDS.

A person's total scale score is the average A person's total scale score is the average the scale scores of all the items that the scale scores of all the items that person agreed with person agreed with

Thurstone’s Paired ComparisonsThurstone’s Paired Comparisons

Very similar processVery similar process Difference is in how the initial ratings Difference is in how the initial ratings

are collectedare collected method of paired comparisons requires method of paired comparisons requires

each judge to make a judgement about each judge to make a judgement about each pair of statements (Uuugh!) each pair of statements (Uuugh!)

57 statements require 1,596 unique 57 statements require 1,596 unique comparisonscomparisons

Likert ScalesLikert Scales

Same initial process (define Same initial process (define construct, generate a large list of construct, generate a large list of items/stimuli)items/stimuli)

group of judges rate the items - 1-group of judges rate the items - 1-to-5 rating scale where: to-5 rating scale where:

1.1. = strongly unfavorable to the concept = strongly unfavorable to the concept

2.2. = somewhat unfavorable to the concept = somewhat unfavorable to the concept

3.3. = undecided = undecided

4.4. = somewhat favorable to the concept = somewhat favorable to the concept

5.5. = strongly favorable to the concept = strongly favorable to the concept

Likert ScalesLikert Scales

Select the itemsSelect the items compute the intercorrelations between compute the intercorrelations between

all pairs of items, based on the ratings all pairs of items, based on the ratings of the judges. of the judges.

Discard items that have a low item-total Discard items that have a low item-total correlationcorrelation

Administer the scaleAdminister the scale Each respondent is asked to rate each item on Each respondent is asked to rate each item on

some response scale. some response scale. 1.1. = strongly disagree = strongly disagree 2.2. = disagree = disagree 3.3. = undecided = undecided 4.4. = agree = agree 5.5. = strongly agree = strongly agree

Can also use a forced-choice response scale Can also use a forced-choice response scale with an even number of responses (no middle with an even number of responses (no middle neutral)neutral)

Total score for the respondent on the scale is Total score for the respondent on the scale is the sum of their ratings for all of the items (this the sum of their ratings for all of the items (this is why this is sometimes called a "summated" is why this is sometimes called a "summated" scale). scale).

Guttman ScalesGuttman Scales

also known as also known as cumulative scalingcumulative scaling or or scalogram analysisscalogram analysis. .

Goal: a set of items or statements Goal: a set of items or statements such that a respondent who agrees such that a respondent who agrees with any specific question in the list with any specific question in the list will also agree with all previous will also agree with all previous statements. statements.

GuttmanGuttman

Same as before…generate the itemsSame as before…generate the items Judges rate the statements or items Judges rate the statements or items

in terms of how favorable they are to in terms of how favorable they are to the concept of immigration the concept of immigration (Yes=favorable; No=unfavorable)(Yes=favorable; No=unfavorable)

The key to Guttman scaling is in the The key to Guttman scaling is in the analysis. analysis.

Guttman ScalingGuttman Scaling

construct a matrix that shows the construct a matrix that shows the responses of all the judges on all of responses of all the judges on all of the itemsthe items

sort this matrix so that respondents sort this matrix so that respondents who agree with more statements are who agree with more statements are listed at the top and those agreeing listed at the top and those agreeing with fewer are at the bottom. with fewer are at the bottom.

The resulting scale should be nearly The resulting scale should be nearly cummulativecummulative

Guttman MatrixGuttman Matrix

NNNNNN9

NNYNNY4

NNNNYY15

NNYNYY22

NNNNYY32

NNNYYY17

NNYYYY6

NNYYYY2

NYNYYY8

…YYYYY5

Item kItem 3Item 2Item 9Item 4 Item 7Judge

Exceptions are circled

Guttman ScalingGuttman Scaling

Can do this for a small number of Can do this for a small number of itemsitems

if there are many items, then a if there are many items, then a scalogram analysisscalogram analysis is used to do is used to do the same thingthe same thing determines the subsets of items from determines the subsets of items from

our pool that best approximate the our pool that best approximate the cumulative property cumulative property

many analyses used to determine how many analyses used to determine how well the items approximate a true well the items approximate a true cummulative scalecummulative scale

Guttman ScaleGuttman Scale INSTRUCTIONS:INSTRUCTIONS: Place a check next to each statement you Place a check next to each statement you

agree with.agree with. _____ I would permit a child of mine to marry an immigrant._____ I would permit a child of mine to marry an immigrant. _____ I believe that this country should allow more _____ I believe that this country should allow more

immigrants in.immigrants in. _____ I would be comfortable if a new immigrant moved _____ I would be comfortable if a new immigrant moved

next door to me.next door to me. _____ I would be comfortable with new immigrants moving _____ I would be comfortable with new immigrants moving

into my community.into my community. _____ It would be fine with me if new immigrants moved _____ It would be fine with me if new immigrants moved

onto my block.onto my block. _____ I would be comfortable if my child dated a new _____ I would be comfortable if my child dated a new

immigrant. immigrant. Each item has a scale value associated with it (obtained from the scalogram analysis). A respondent's scale score is the sum the scale values of every item they agree with.

Mokken ScalingMokken Scaling

Just like Guttman scalesJust like Guttman scales Individual's scale score is the rank of Individual's scale score is the rank of

the highest item endorsed or the total the highest item endorsed or the total number of items endorsednumber of items endorsed

Key differenceKey difference Mokken scales are probabilistic Mokken scales are probabilistic A respondent answering an item A respondent answering an item

positively will have a large probability of positively will have a large probability of answering a less difficult item in a answering a less difficult item in a positive way as wellpositive way as well

Multidimensional ScalingMultidimensional Scaling

Many interesting concepts are not Many interesting concepts are not unidimensionalunidimensional (e.g., attitude toward immigration)(e.g., attitude toward immigration)

MDS maps the distances between MDS maps the distances between points in a high dimensional space points in a high dimensional space into a lower dimensional space into a lower dimensional space without too much loss of information.without too much loss of information.

Almost always 2-DAlmost always 2-D


The “points” that are represented in The “points” that are represented in multidimensional space can be just multidimensional space can be just about anything.about anything.

These objects might be people, in These objects might be people, in which case MDS can identify clusters which case MDS can identify clusters of people who are “close” versus of people who are “close” versus “distant” in some real or “distant” in some real or psychological sense.psychological sense.


The objects might be physical objects The objects might be physical objects (e.g., cities, cars, books) or (e.g., cities, cars, books) or psychological or cultural objects psychological or cultural objects (e.g., personality traits, religions, (e.g., personality traits, religions, political parties).political parties).

The only requirement is that some The only requirement is that some basis exist for rating or ranking the basis exist for rating or ranking the objects in terms of similarityobjects in terms of similarity


As long as the “distance” between As long as the “distance” between the objects can be assessed in some the objects can be assessed in some fashion, MDS can be used to find the fashion, MDS can be used to find the lowest dimensional space that still lowest dimensional space that still adequately captures the distances adequately captures the distances between objects.between objects.

OOnce the number of dimensions is nce the number of dimensions is identified, must identify the meaning identified, must identify the meaning of the dimensions.of the dimensions.


Basic data representation in MDS is a Basic data representation in MDS is a dissimilarity matrixdissimilarity matrix that shows the that shows the distance between every possible pair of distance between every possible pair of objectsobjects

The goal of MDS is to faithfully represent The goal of MDS is to faithfully represent these distances with the lowest possible these distances with the lowest possible dimensional spacedimensional space

We seek a reference system that can We seek a reference system that can capture the basic information in a data setcapture the basic information in a data set—in this case a data set characterized by —in this case a data set characterized by distances between objects.distances between objects.


This table lists the distances between European This table lists the distances between European cities. A multidimensional scaling of these data cities. A multidimensional scaling of these data should be able to recover the two dimensions should be able to recover the two dimensions (North-South x East-West) that we know must (North-South x East-West) that we know must underlie the spatial relations among the cities.underlie the spatial relations among the cities.

Athens Berlin Dublin London Madrid Paris Rome WarsawAthens 0 1119 1777 1486 1475 1303 646 1013Berlin 1119 0 817 577 1159 545 736 327Dublin 1777 817 0 291 906 489 1182 1135London 1486 577 291 0 783 213 897 904Madrid 1475 1159 906 783 0 652 856 1483Paris 1303 545 489 213 652 0 694 859Rome 646 736 1182 897 856 694 0 839Warsaw 1013 327 1135 904 1483 859 839 0


In the cities data, the meaning is In the cities data, the meaning is quite clear. quite clear.

The dimensions refer to the North-The dimensions refer to the North-South x East-West surface area South x East-West surface area across which the cities are dispersed. across which the cities are dispersed.

Expect MDS to faithfully recreate the Expect MDS to faithfully recreate the map relations among the cities.map relations among the cities.


Common to evaluate the fit of a Common to evaluate the fit of a model using a “stress” statisticmodel using a “stress” statistic

Excellent.02

Good.05

Fair.10

Poor.20

FitStress


Stress StatisticStress Statistic


The stress for the one-dimensional model The stress for the one-dimensional model of the cities data is .31, clearly a poor fitof the cities data is .31, clearly a poor fit

The analyses proceeds by trying higher The analyses proceeds by trying higher dimensional models, seeking the minimum dimensional models, seeking the minimum number of dimensions necessary to get an number of dimensions necessary to get an acceptable fitacceptable fit

A 2-D model fits very well (stress value A 2-D model fits very well (stress value =.009) indicating an exceptional fit. This is =.009) indicating an exceptional fit. This is no great surprise for these data.no great surprise for these data.


-3.00

-2.00

-1.00

.00

1.00

2.00

3.00

-3.00 -2.00 -1.00 .00 1.00 2.00 3.00

Berlin

Warsaw

Rome

Paris

Madrid

LondonDublin

Athens


Rarely do social sciences have metric data Rarely do social sciences have metric data on which to compare the objects. More on which to compare the objects. More typically, the “distances” are on typically, the “distances” are on psychological dimensions for which ordinal psychological dimensions for which ordinal relations are the best scaling possiblerelations are the best scaling possible

Use similarity ratings insteadUse similarity ratings instead Similarity is a slippery slopeSimilarity is a slippery slope Can be very time consuming because they Can be very time consuming because they

require the rating of each pair of objectsrequire the rating of each pair of objects

Shepard & Chipman: Shapes of StatesShepard & Chipman: Shapes of States

Illinois

Nevada

Minn.

Missouri

Alabama

OregonColo.

Nebraska

Maine

La.

Fla.

S.C. W.Va.

Idaho

OklahomaVertical/Irregular

Horizontal/Rectangular

Bend/Handle

Smallish/Wiggly

Beer MDSBeer MDS

Inexpensive

High Quality

High Alcohol

High Class

Good Taste

Refreshing

Dim I is“Flavor”

Dim II is“Image”

BUDWEISER

AMSTELLIGHT

HEINEKEN GUINESS

MILLERLITE

Documents

Scaling DeShon - 2005. What is Scaling? Young’s (1984) definition Young’s (1984) definition Scaling is the branch of measurement that involves the construction