Exam….

Sampling Basics

Jeremy Kees, Ph.D.

Conceptually defined…

Sampling provides a means of gaining information about the population without the need to examine the population in its entirety

Sampling is far more important for descriptive/causal research than exploratory research

• A quality sample allows us to use a sample statistic to tell us something about a population parameter

• In other words, we can draw confident conclusions about how the larger population will behave without surveying everyone in that population!

Why sample?

• Usually not practical to collect try and collect data from the entire population of interest– But sometimes it is!

• Workforce

• Villanova

• B2B customers

In context….

• Sampling is one “piece of the puzzle” to getting valid data

• The sample could be great, but if your measures are bad, the data will be bad

• The composition of your sample directly impacts the robustness of your learnings– Sampling strategy relates to the internal or

external validity of your findings??

External Validity

• The degree to which the conclusions in our study would hold for other persons in other places and at other times

Threats to External Validity

• The population you want to generalize to is different from your sample

• The setting/place you want to generalize to is different from the place you conducted your study

• The time period when you conducted your study was peculiar

How do we strengthen external validity??

• Use random selection from the population of interest

• Replicate your study with different samples, in different settings, and over time

Sample Design

• Includes method of selection, sample structure, and analysis plans

• The goal is to achieve a balance between– Required precision– Available resources

Step 1: Define the population

• Specify the target population as clear and complete as possible– This is your “sampling frame”

• List of all units in your population

• Not always available– In fact, it’s rarely available for me

Procedure for Drawing a Sample

Select the Sample Elements

Define the Target Population

Identify the Sampling Frame

Select a Sampling Method

Determine the Sample Size

Collect the Data from the Designated Elements

Step 1

Step 2

Step 4

Step 3

Step 5

Step 6

Defining the Population (Step 1)

• Who are we interested in generalizing to?

Define the Sample Frame

• In some cases, we can identify all possible cases and then randomly select our sample

• However, in social science research, this is not typically the case

• We do our best to chose a sampling frame that will give us our best shot and obtaining a sample that is representative of the population of interest

Select a Sampling Method

• So we have decided who we’re going to sample from, now we have to decide HOW to go about doing it

Sampling Methodologies: Probability Sampling

• Simple Random Sampling– Systematic Random Sampling

• Stratified Random Sampling

• Cluster Random Sampling

Sampling Methodologies: Nonprobability Sampling

• Convenience Sampling

• Modal Instance Sampling

• Expert Sampling

• Quota Sampling

• Heterogeneity Sampling

• Snowball Sampling

Simple Random Sampling

• Every member of the population has the same chance of being selected

• Advantages– Simple– Easy calculation of error

• Limitations– Need complete and accurate

population listing

Cluster Sampling

• Clusters are formed and then a random sample is taken from each cluster– Clusters are heterogeneous

– Geography

• Advantages– Quick/easy (relative to other

forms of random sampling)

– Don’t need complete population info

Stratified Sampling

• Strata are formed and then a random sample is taken from each strata– Stratas are homogenous segments

• Advantages– Ensures underrepresented groups in the

population are represented in the sample

Quota Sampling

• Stratify the population an important variable and then sample each stratum

• Advantages– Quick/easy (relative to other forms of random

sampling)

– Ensures representation from minorities in the population

• Use of quotas to get an approximate representative sample (CWL example)

Convenience Sampling

• Using those who are willing to volunteer (or forced to participate)

• Advantages– Quick/easy (relative to other forms of

random sampling)

– Inexpensive

• Limitations– Generalization problems

Sample Size

• Sample size is a function of…– Effect size– Alpha level (precision)– Statistical power

• Sometimes difficult to estimate– Sample size calculator

Just to get us thinking…

• TED Talk

• Analysis Issues to consider– Collect as much data as you can reasonably

afford– It is often useful to look at segments

Panel Companies

• Qualtrics• Research Now• CriticalMix• Others…

• Even professional marketing research firms need accountability

• Attention screeners are highly recommended

Data Costs

• Cost by Incidence Rate (IR)

• Gen Pop (100% IR) = $4-5– Mturk = $0.50 or less

• Specialized Sample (<5% IR) = $20 - 25– Adolescent smokers

• Physicians = $150+

Sampling Challenges

• Adolescent Smokers– Challenges??

Sampling Challenges

• Veterinarians– Challenges??

Sampling Challenges

• Pet owners– 65% female

– Age 32-49

– Urban/suburban

– $75k+ HHI

– Have visited the vet in the past year

– Pet sleeps indoors

– Must be a “conscientious” owner• Eg, celebrate pets bday; purchase holiday gifts for

pet; feel guilty about leaving the pet alone, etc.

• Challenges??

Sampling Challenges

• Any research that will inform very important decisions that will have major impact– Food/Tobacco policy decisions

(warnings/disclosures)

– Industry initiatives (FOP labeling)

• Challenges??– As more $$ or human wellbeing is at stake,

sampling becomes very important

Team Assignment #3

** Don’t start on this assignment until you’ve read Fowler (CH 6-7)

• Based on your research design1. Write a paragraph about what your

measurement instrument is supposed to accomplish

2. Make a list of what should be measured to accomplish the goals of the study

3. Develop your measurement instrument

Team Assignment #3

• Deliverables include:1. A very clean, polished version that you

could use to actually collect data• This means you will need to carefully think

through all of the issues we covered tonight (e.g., set-up, ordering, length, multi-item scales, etc.)

2. Intro paragraph and variable list (see previous slide)

(Note: Don’t worry about defining your sample--you’ll have your chance to do that next week)

Assignment #4

• Develop a sampling plan1. Discuss your target market and define the

sampling frame

2. Select a sampling method and explain why it is most appropriate for your study

3. Determine your sample size

4. Explain your data collection strategy (e.g., how you would go about getting your sample to participate)