SamplingSampling

Samples and populations

• Sample:– the participants actually included in a study

• Population:– the larger group from which the sample is

drawn– the group to which the experimenter hopes to

generalize

• Sampling frame:– the members of the population who actually

could be selected for the sample

Election 2000: The limits of sampling

• Going into the presidential election in 2000, polls revealed the election was too close to call

• In the swing state of Florida Bush won by only 537 votes

• The margin of victory was smaller than the margin of error in the vote counting process.

• Newsweek: Gore 46%, Bush 49%

• Harris: Gore 47%, Bush 47%

• CBS: Gore 48%, Bush 47%

• ABC/Washington Times: Gore 46%, Bush 49%

The ideal sample: A truly valid sample must be:• Random• Representative• Sufficient• Note: in practical terms, it is not easy to

draw a random sample. The only factor operating when a given subject is selected must be chance.

The Ideal Sample: A truly valid sample must be

• random: – equal likelihood principle: each and

every member of the larger population has the same chance of being included in the study

– selection bias: potential participants are systematically excluded from the experiment by the researcher

– response bias: potential participants exclude themselves from the experiment

Representative

• representative: possessing the same traits, characteristics, and qualities, and in the same proportions, as the larger population– sampling error: the degree to

which the value of an estimate calculated from a sample deviates from the actual population

• (example: political polls, + or - 3 points)

Sampling error is based on three factors

• Sample size: larger samples are more representative of the larger population– Size matters; all else being

equal, a larger sample is better than a small sample

• Variability in the population: the more homogenous the population, the more the sample will reflect it.

• The sampling method: Some types of samples have greater or less sampling error than others.

problems with using the proverbial college freshman• May not be typical or

representative of the larger, “normal” adult population.– More idealistic– More educated– Less income– Lifestyle differences

(drinking behavior, musical tastes, etc.)

Sufficient

• sufficient: a large sample produces a smaller sampling error than a small sample

• difficult to determine number of subjects needed.

– magnitude of effects

– sensitivity of dependent measures

• small samples have low statistical power– differences have to be

whopping to be statistically significant

– no less than 5-10 subjects per condition for factorial designs

– social sciences “rule of thumb,” 30 subjects per condition

– 1500 subjects enough for a national sample

– need for “power” tests

Probability sampling: Types of Random Samples

• simple random sample• systematic random sample: • stratified sample• cluster sample

Simple random sample

• Drawing names at random out of a hat

• California lottery—numbered balls are selected at random

• Selecting participants at random by assigning every potential subject a number, then drawing numbers at random

• Seldom used in actual practice

Systematic sample

• Every 10th name, 79th name or nth name in a complete list of the population is chosen for inclusion in the sample.

• The starting number is usually selected at random, to prevent experimenter bias– Referred to as a systematic

sample with a random start

Stratified sample

• Increases representativeness by decreasing sampling error

• Reproduces the correct percentages or ratios of demographic characteristics found in the larger population

• Researcher must decide the criteria on which stratification will be based

• Stratification can lead to an infinite regress

Cluster sample

• Also known as multi-stage cluster sampling

• Involves taking a sub-sample of a sub-sample of a sub-sample– a sub-set of a population is

selected at random, then a sub-set of that sample is selected, and so on…

• More efficient, practical than other methods of random sampling, but also less accurate

Non-Random Samples

• convenience sample:– whoever is available– example: mall intercept

• volunteer sample:– soliciting participants– example: college students

• purposive sample:– seeking out people with

certain characteristics or attributes

– example: parents of children whose were abducted and murdered

Types of Non-Random Samples--continued

• quota sample (like a stratified sample, but nonrandom):– based on a quota matrix– example: elderly of a certain

age, in a certain income bracket• network sample (snowball

sample):– based on referrals– example: children with rare

disorders