The Logic of Sampling Week 2 Day 2 DIE 4564 Research Methods

The Logic of SamplingWeek 2 Day 2

DIE 4564 Research Methods

http://www.youtube.com/watch?v=be9e-Q-jC-0&list=TLFnRyKtISo3c

http://www.youtube.com/watch?v=be9e-Q-jC-0&list=TLFnRyKtISo3c

Types of Research Populations

At least four different types of populations must be considered when preparing to collect data:• The results of the study should be applicable to

the target population • The source population is a well-defined subset

of individuals from the target population• The sample population is the individuals from

the source population who are asked to participate

• The study population is the members of the sample population who actually participate in the study

Target Populations

• A well-defined study question identifies a target population to which the results of the study should apply.

• A target population might be quite narrow (like one wing of a long-term acute care hospital) or relatively large (like a whole country).

• Unless the target population is very small, measuring the entire target population or even randomly sampling from it may be impossible.

Source Populations

A source population (sometimes called a sampling frame) consists of an enumerated list of population members. For example: • All women with a breast cancer diagnosis in the

past 2 years who are indexed in a particular cancer registry

• All members of a professional sports league• All households within 2 miles of a particular

nuclear power plant

Sample Populations

A source population is often much larger than the sample size required for a study. In this situation, only a portion of the source population is selected to serve as a sample population.

Sampling methods can be categorized as probability-based or non-probability.

Examples of Types of Probability Sampling

Example of a non-probability-based sample

A non-probability-based convenience population can be selected based on the ease of access to those individuals, schools, or communities.

Caution- Convenience samples may not be representative!

Study Populations

• The study population will consist of the members of the sample population who can be located, who consent to participation, and who meet all eligibility criteria.

• A 100% participation rate is extremely rare. • A low response rate may result in nonresponse

bias if the members of the sample population who agree to be in the study are systematically different from nonparticipants.

What % response would you think is “representative” of a population?

Study Populations

A less than 100% participation rate is usually not a problem as long as the researcher:• Uses suitable and carefully explained sampling

methods• Takes appropriate steps to maximize the

participation rate• Recruits an adequately large sample size

Cross-Sectional Surveys

• The goal of most cross-sectional surveys is to describe a specific target population accurately.

• Convenience samples rarely result in a study population that is representative of the target population.

• Ideally, the researcher needs some way to confirm that the source population is similar to the target population and that the sample population is similar to the source population.

Case-Control Studies

• All cases must have the same disease, disability, or other health-related condition.

• The controls must be similar to the cases in every way except for their disease status, so cases and controls should be drawn from populations with similar demographics.

Cohort Studies

• Longitudinal cohort studies: the participants should be representative of the source and target populations The requirements for longitudinal studies are similar to those for cross-sectional studies, since both study designs recruit population-based samples.

• Prospective / retrospective cohort studies: the exposed and unexposed should be drawn from similar populations The recruitment of exposed and unexposed for cohort studies is like the recruitment of the cases and controls for case-control studies.

Experimental Studies

• Experimental studies require a source population that is reasonably representative of the target population.

• Safety is always the top priority in designing an experimental study. The risk of harm to participants can be reduced by selecting an appropriate source population and defining strict inclusion and exclusion criteria.

Vulnerable Populations

• Vulnerable populations in health research include some people with poor health, some people with limited decision-making capacity, and members of some socially marginalized groups, among others.

• Despite the potential risks of including members of these populations in research studies, including them is the only way to study health issues in these groups. Example: The health of prisoners can only be

studied by conducting research in prisons.

Vulnerable Populations

• Research conducted with members of vulnerable populations requires extra consideration of the potential risks of research to participants.

• The ability of every participant to provide informed consent free from coercion must be assured.

• Concerns about the increased risks of adverse effects from study participation must be addressed.

Community Involvement

• Some studies benefit from or require the participation and/or support of whole geographic, cultural, or social communities and their leaders.

• Community-based studies often work best when they use methods such as those developed for Community-Based Participatory Research.

Nonprobability Sampling*

• Nonprobability Sampling – any technique in which samples are selected in some way not suggested by probability theory. Reliance on available subjects

(convenience) Purposive or judgmental sampling Snowball sampling Quota sampling

Nonprobability Sampling

• Reliance on Available Subjects Convenience sampling Does not allow for control over

representativeness. Only justified if less risky methods are

unavailable. Researchers must be very cautious about

generalizing when this method is used. When might this method be appropriate?


• Purposive or Judgmental Sampling – a type of nonprobability sampling in which the units to be observed are selected on the basis of the researcher’s judgment about which ones will be the most useful or representative. Small subsets of a population Two-group comparison Deviant cases When might this method be appropriate?


• Snowball Sampling – a nonprobability sampling method whereby each person interviewed may be asked to suggest additional people for interviewing. Often used in field research, special

populations

When might this method be appropriate?


• Quota Sampling – a type of nonprobability sampling in which units are selected into a sample on the basis of pre-specified characteristics, so that the total sample will have the same distribution of characteristics assumed to exist in the population being studied.

Similar to probability sampling, but has problems: quota frame must be accurate, selection of sample elements may be biased

When might this method be appropriate?


• Selecting Informants Informant – someone who is well versed in

the social phenomenon that you wish to study and who is willing to tell you what s/he knows about it.


• Review Question A researcher studying college success

knows that a particular university’s student body is 40% first years, 25% second years, 20% third years, and 15% fourth years. The researcher selects cases to match this distribution. What kind of nonprobability sampling technique has the researcher used?


• Review Question Because the researcher is sampling in

order to match the population distribution, the quota sampling technique is being used.

The Theory and Logic of Probability Sampling

• Probability Sampling – the general term for samples selected in accord with probability theory. Often used for large-scale surveys.

If all members of a population were identical in all respects there would be no need for careful sampling procedures. However, this is rarely the same.

A sample of individuals from a population must contain the same variations that exist in the population.


• Conscious and Subconscious Sampling Bias Bias – those selected are not typical nor

representative of the larger population.


• Representativeness and Probability of Selection

Representativeness – the quality of a sample of having the same distribution of characteristics as the population from which it was selected.

Samples might not need not be representative in all respects, yet must be representative in all aspects relevant to the research question.


• Representativeness and Probability of Selection A sample will be representative of the

population from which it is selected if all members of the population have an equal chance of being selected in the sample.

EPSEM (Equal Probability of Selection Method) - method to create a random sample


• Representativeness and Probability of Selection Advantages of Probability Sampling1. Probability samples are typically more

representative than other types of samples because biases are avoided.

2. Probability theory permits researchers to estimate the accuracy or representativeness of the sample.


• Key terms: Element – that unit of which a population is

composed and which is selected in a sample. Population – the theoretically specified

aggregation of the elements in a study. Study Population – a sampling method in which

each element has an equal chance of selection independent of any other event in the selection process.


• Key terms:• Random Selection – each element has an

equal chance of selection independent of any other event in the selection process.

• Sampling Unit – that element or set of elements considered for selection in some stage of sampling.

• Parameter – a summary description of a given variable in a population

Ex: Sampling Distribution of Ten Cases


• Sampling Distribution and Estimates of Sampling Error Statistic – the summary description of a

variable in a sample, used to estimate a population parameter.


• Sampling Distribution and Estimates of Sampling Error Sampling Error – the degree of error to be

expected of a given sample design.

Sample Size

• Sample size determination is the act of choosing the number of observations to include in a statistical sample.

• Sample size must be large enough to make inferences about a population from a sample.

Ways to determine sample size*

• Expedience based on what is doable or affordable (convenience)

• Use a target variance for an estimate to be derived from the sample eventually obtained.

• Use a target for the power of a statistical test to be applied once the sample is collected.

Importance of Sample Size

An adequate number of study participants is required

to achieve valid and significant results.

Importance of Sample Size

The goal is to recruit just the right number of participants based on statistical estimations of how many people are required to answer the study question with a specified level of certainty.

• If more participants are recruited than are statistically required, resources are wasted.

• If too few participants are recruited, the whole study will be almost worthless because there will not be enough statistical power to answer the study question.

Bigger Samples Are Better

Large samples from a population are usually better than small ones at yielding a sample mean close to the true population value.

Sample Size and Means

Bigger Samples Are Better

• When the sample size is small, the sample mean may be quite far from the mean in the total population from which the sample was drawn. This is represented by a wide confidence interval that reaches far from the sample mean.

• When the sample size is large, the sample mean is expected to be close to the population mean, and the confidence interval will be narrower.

Larger Samples from a Population Have a Narrower 95% Confidence Interval Than Smaller Samples

Sample Size Estimation

A sample size calculator – more accurately called a sample size estimator – should be used to identify an appropriate sample size goal.

Sample size estimators suggest an appropriate minimum sample size based on a series of “best guesses” the researcher makes about the expected characteristics of the sample population.

When in doubt, err on the size of a larger sample!

FIGURE 17- 3 Examples of Sample Size Calculation

Power Estimation

Another way to check for sample size requirements is to work backward from the number of participants likely to be recruited to see whether that sample size provides adequate statistical power for the study design.

Statistical power is the ability of a statistical test to detect significant differences in a population when differences really do exist.

Power Estimation

Sometimes a sample population does not capture the true experience of the population:• Type 1 errors (α) occur when a study population

yields a significant statistical test result when one does not exist in the source population.

• Type 2 errors (β) occur when a statistical test of data from the study population finds no significant result when one actually exists in the source population.

• Power = 1 – β

FIGURE 17- 4 Power and Errors

Examples of Power Calculation

Be prepared to rethink the study

question, study approach, and/or target

and source populations if the power for

the estimated number of participants is

not sufficient.


• Confidence Levels and Confidence Intervals Confidence Level – the estimated

probability that a population parameter lies within a given confidence interval.

Confidence Interval – the range of values within which a population parameter is estimated to lie.


• Review Question True or False: Regardless of the sample

size, the mean of the sampling distribution will equal the true population mean.


• Review Question True: Regardless of the sample size, the

mean of the sampling distribution will equal the true population mean.

Populations and Sampling Frames

• Sampling Frame – a list of units that compose a population from which a sample is selected. If the sample is to be representative of the

population, it is essential that the sampling frame include all members of the population.


• Review of Populations and Sampling Frames

1. Findings based on a sample represent only the aggregation of elements that compose the sampling frame.

2. Sampling frames do not include all the elements their names might imply. Omissions are inevitable.

3. To be generalized, all elements must have equal representation in the frame.


• Review Question To study college performance, a

researcher obtains a list of all enrolled students at the university from the Registrar’s Office. This list is called what?


• Review Question This list would be called the sampling

frame. The researcher could then select cases from that list to comprise the sample.

Types of Sampling Designs

• Simple Random Sampling• Systematic Sampling• Stratified Sampling• Implicit Stratification in Systematic Sampling


• Simple Random Sampling – a type of probability sampling in which the units composing a population are assigned numbers. A set of random numbers is generated and the units having those numbers are included in the sample. May be time consuming. Used in experimental designs.




• Systematic Sampling – a type of probability sampling in which every kth unit in a list is selected for inclusion in the sample. Easier than simple random sampling. In Social Science may even be considered

more accurate than simple random samples.


• Systematic Sampling Sampling Interval – the standard distance

between elements selected from a population in the sample.

sizesample

sizepopulationIntervalSampling


• Systematic Sampling Sampling Ratio – the proportion of

elements in the population that are selected to be in a sample.

sizepopulation

sizesampleRatioSampling


• Stratified Sampling Stratification – the grouping of units composing a

population into homogenous groups (strata) before sampling.

Per Social Science, slightly more accurate than simple random sampling.

Stratification is a modification to simple random and systematic sample methods.


• Stratified Sampling


• Implicit Stratification in Systematic Sampling Systematic sampling can, under certain

conditions, be more accurate than simple random sampling.Particularly when the arrangement of

the list is implicitly stratified.


• Illustration: Sampling University Students Study population and sampling frame

Stratification

Sample selection

Sample modification


• Review Question When using systematic sampling, the first

unit is selected by __________.


• Review Question When using systematic sampling, the first

unit is selected by random choice.

Multistage Cluster Sampling

• Cluster Sampling – a multistage sampling in which natural groups are sampled initially with the members of each selected group being sub-sampled afterward.

• Used when it is not practical or possible to create a list of all elements that compose the target population.

• Highly efficient, but less accurate.


• Multistage Designs and Sampling Error


• Stratification in Multistage Cluster Sampling Stratification can take place at each level

of sampling.


• Probability Proportionate to Size (PPS) Sampling – a type of multistage cluster sample in which clusters are selected not with equal probabilities but with probabilities proportionate to their sizes—as measured by the number of units to be sub-sampled. A more sophisticated form of cluster

sampling.


• Disproportionate Sampling and Weighting Weighting – assigning different weights to

cases that were selected into a sample with different probabilities of selection.

Probability Sampling in Review

• Probability sampling remains the most effective method for the selection of study elements for two reasons. Probability sampling avoids researchers’

conscious or subconscious biases in element selection.

Probability sampling permits estimates of sampling error.

The Ethics of Sampling

• Because probability sampling always carries a risk of error, the researcher must inform readers of any errors that might make results misleading.

The Ethics of Sampling

• Sometimes, nonprobability sampling methods are used to obtain the breadth of variations in a population. In this case, the researcher must ensure that readers do not confuse variations with what’s typical in the population.

Quick Quiz

Chapter 7 Quiz

One of the most visible uses of survey sampling lies in _____.

A. political polling

B. probability sampling

C. core sampling

D. nonprobability sampling

Chapter 7 Quiz

Answer: A.

One of the most visible uses of survey sampling lies in political polling.

Chapter 7 Quiz

_____ sampling occurs when units are selected on the basis of pre-specified characteristics.

A. Snowball

B. Quota

C. Purposive

D. Probability

Chapter 7 Quiz

Answer: B.Quota sampling occurs when the units are selected on the basis of pre-specified characteristics.

Chapter 7 Quiz

_____ describes a sample whose aggregate characteristics closely approximate the aggregate characteristics of the population.

A. Exclusion

B. Probability sampling

C. EPSEM

D. Representativeness

Chapter 7 Quiz

Answer: D.

Representativeness describes a sample whose aggregate characteristics closely approximate the aggregate characteristics of the population.

Chapter 7 Quiz

A _____ is the list of elements from which a probability sample is selected.

A. confidence level

B. confidence interval

C. sampling frame

D. systematic sample

Chapter 7 Quiz

Answer: C.

A sampling frame is the list of elements from which a probability sample is selected.

Chapter 7 Quiz

_____ is the general term for samples selected in accord with probability theory.

A. Nonprobability analysis

B. Correlation

C. Probability sampling

Chapter 7 Quiz

Answer: C.

Probability sampling is the general term for samples selected in accord with probability theory.

Chapter 7 Quiz

A _____ population is the aggregation of elements from which a sample if actually selected.

A. theoretical

B. small

C. large

D. concept

E. study

Chapter 7 Quiz

Answer: E.

A study population is the aggregation of elements from which a sample if actually selected.

Documents

The Logic of Sampling Week 2 Day 2 DIE 4564 Research Methods