Match line A to one of the three on the left

Match line A to one of the three on the left

What percentage of people went along?

Percentage conforming decisions

0

5

10

15

20

25

30

35

40

alone group situation

% c

on

form

ing

dec

isio

ns

As number of confederates went beyond 3, conformity remained constant

With addition of even one supporter, it reduced drastically

Conformity as group size increased

0

5

10

15

20

25

30

35

40

1 2 3 4 8 OnesupporterNumber of opposing people

% c

on

form

ing

dec

isio

ns

Sample Experimental Design:

Readability of devices•Palmtop•Rocket Book•Laptop

The Effect of Screen Size on Readability Using Three Different Portable Devices

Chung, Kolatch, Sculimbrene, & Wen (2000)

From SHORE Online Experiments, Ben Shneiderman’s class

Independent Variables: 3x 2 designPortable Device (3) x Text length (2)

Palm RocketBook LaptopNo scrollingScrolling

Dependent Variable: ReadabilityMeasures:(1) Time it took to read document(2) Answer to question about document(3) User Satisfaction Survey

Possible Confounding Variables:Font and size of text itself (constant)Contrast of screen (constant)

What are variables?Properties of objects and events that can take on different values are referred to as variables.

For example: Monitor size is a variable in a study of readability.

Kinds of variables:Independent, Dependent, & Confounding variables

Other Categorizations: Discrete variables & continuous variables

Discrete variables:Discrete variables can only take on a few possible values: number of siblings, marital status etc, number of items than can be retrieved by a search engine at any time.

Continuous variables:Continuous variables are those, which can take on (at least in theory) many different values between the lowest and the highest possible point: for example, visual contrast on a monitor

Note. Often whether a variable is discrete or continuous depends on how it is measured

Independent (IV) & Dependent variables(DV):A researcher is often interested in studying the effects of an IV on some other variable, i.e., DV.

An IV is manipulated by, measured or selected by the researcher in order to observer its relationship to some other variable of interest (the DV).

AN IV is an antecedent condition to the observed behavior. Therefore an IV has levels, while a DV can have different measures.

Confounding VariablesSuch variables could affect the relationship between IV and DV thus presenting a confound in your study.

For example, individual differences between people can often confound the relationship between an IV and DV.

Mean reading time for

multiple pages0

5

10

15

20

25

30

35

Palm Pilot Rocket Book Laptop

Tim

e

0

5

10

15

20

25

30

35

RocketBook (Palm likesettings)

RocketBook (orginalsettings)

Tim

eHypothetical Data: Mean

reading time for multiple pages

Effect of text and graphic links on navigation time

IV: Kinds of links

Levels

•Text

•Graphic

•Text&Graphic

DV: Time to find item

Confounding Variable: Differential download time

A SHORE Online study

Hypothetical download times for different kinds of pages

Text Site Graphic SiteText and Graphic

SitePage1 15 28 12Page2 9 22 7Page3 5 9 8Page4 6 12 15Page5 15 13 2Page6 12 18 9Mean 13 15.86 9.71SD 8.1 7.15 4.68

Solution: Use download time as a covariate in the analysis

Note. this is not real data!

Disadvantages of measurement

Problem of realism/ecological validity: It is a reductionist approach, necessarily reduces a phenomenon to a skeletal format. Has it even remained the same phenomenon?

Measuring incorrectly: Measuring incorrectly is

worse than not measuring (?)

Training: Often requires training to do right.

Advantages of measurement

when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind: it may be the beginning of knowledge but you have scarcely, in your thoughts, advanced to the degree of science.

William Thomson, Lord Kelvin, 1891, p.80-quoted by Peter Wright @ British Computer Society

-as quoted by Jakob Nielsen, 1990

Advantages of measurement•By assigning numbers to events, we try to be specific and concrete.

•Language is inherently subjective. Numbers can often add some degree of objectivity (though numbers can be subjectively assigned as well!)

•Promotes a common analytical language, encourages communication

•Helps in dealing with engineers who like numbers!

Levels of Measurement

Nominal scale• Nominal measurement consists of

assigning items to groups or categories. • No quantitative information is conveyed

and no ordering of the items is implied.• Nominal scales are therefore qualitative

rather than quantitative. • Examples: Religious preference, race, and

gender are all examples of nominal scales• Statistics: Sum, Frequency Distributions

Ordinal Scale• Measurements with ordinal scales are ordered:

higher numbers represent higher values. • However, the intervals between the numbers

are not necessarily equal. • There is no "true" zero point for ordinal scales

since the zero point is chosen arbitrarily. • For example, on a five-point rating scale

measuring attitudes towards downloading music, the difference between 2 and 3 may not represent the same difference as the difference between 4 and 5.

• Also, lowest point was arbitrarily chosen to be 1. It could just as well have been 0 or -5.

Interval Scale• On interval measurement scales, one unit

on the scale represents the same magnitude on the trait or characteristic being measure across the whole range of the scale.

• For example, on an interval scale of anxiety, a difference between 10 and 11 would represent the same difference in anxiety as between 50 and 51.

(No interval scale of anxiety exists!)

Ratio Scale• Ratio scales are like interval scales except they

have true zero points. • For example: Kelvin scale of temperature. This

scale has an absolute zero. Thus, a temperature of 300 Kelvin is twice as high as a temperature of 150 Kelvin.

• Very hard to achieve!• Don’t need to worry about the distinction between

interval and ratio

Scales of measurement in usability

Physics Envy (Peter Wright, British Computer Society)

Problem with measurement in usability: “a

measurement, say, 12% decrease in tasktime might mean 12% better productivity, but couldjust as well mean 12% more time wasted on otheractivities or 12% more stress for the users. So wedo not necessarily have a true ratio scale forusability measurements.” Jakob Nielsen

Scales of measurement in usability

Physics Envy (Peter Wright, British Computer Society)

Problem with measurement in usability: “a

measurement, of say12% decrease in task time might mean

>>12% better productivity, but could just as well mean >>12% more time wasted on other activities or >>12% more stress for the users.

So we do not necessarily have a true ratio scale forusability measurements.” Jakob Nielsen

Can we acheive nominal scales?

Require us only to distinguish if twoproblems are the same or not.

Unfortunately, it is hard to know to whatextent two observations are really cases of the sameUnderlying usability problem, and we do not evenhave a good enumeration of usability problems withwhich to match observations.

Can we ever dream of ordinal scales?

Using the somewhat more ambitious ordinal scales to measure usability problems, we face the counting problem.

We would like to know whether one interface is better than another by counting the frequencies of various usability problems in the two systems.

Problem A Problem B Problem CSystem 1 1 1 8System 2 2 1 1

If severity is a constant, we can have ordinal mesurement

Severity rated on a scale of 1 to 5

Problem A Problem B Problem C3 4 1

Importance = Frequency x Severity

Problem A Problem B Problem CFrequency Severity Frequency Severity Frequency Severity

System 1 1 3 1 4 8 8System 2 2 6 1 4 1 1

As you go from nominal to interval scales, you get more information about thing being measured.

Nominal Scales:DO you use CNN for online news?

Yes/No

Ordinal Scales:Dhow many times do you use CNN in a day?

(a) 0 times a day(b) 1-5 times a day(c) more than 5 times a day Yes/No

Interval/Ratio Scales:How many times do you use CNN in a day?

_____ times a day