Introduction to Survey Research

Pathways to Careers in Clinical and Translational Research (PACCTR)

Curriculum Core

Harry Truman displays a copy of the Chicago Daily Tribune newspaper that erroneously reported the election of Thomas Dewey in 1948. Truman's narrow victory embarrassed pollsters, members of his own party, and the press who had predicted a Dewey landslide.

Introduction to Survey Introduction to Survey ResearchResearch

Overview of Sections

•One: Basics of Survey Research

•Two: Measurement Levels and Types of Data

•Three: Elements of Item Construction

•Four: Assessing Reliability and Validity

•Five: Determining the Coding System

•Six: Deciding How to Enter and Store the Data

•Seven: Data Entry, Data Checking and Quality Control

•Resources

Section One

Basics of Survey Research

Section 1Basics of Survey Research

• Objectives

– Describe what surveys are used to measure – Give examples of how surveys can assess change – Define "needs", "assets", "behavior", "opinions",

"attitudes", "beliefs"

Key Terms:

Survey, questionnaire, descriptive research, causal explanation, prediction, evaluation, single sample, successive samples, panel study

What do surveys measure?

• Behavior, attitudes and beliefs

• Repeated observations.– Example: if a researcher were interested in exercise patterns

in obese patients, we could follow them every time they perform any physical activity to determine frequency, duration and intensity. Similarly, if we were interested in dietary habits, we could sit at their table in the morning and record what they ate.

• This is not feasible and there are many behaviors that cannot be directly observed such as attitudes, beliefs and opinions.

What do surveys of people measure?

• So, what are researchers to do?

Do a Survey (Questionnaires)

• Surveys are a systematic way of asking people to volunteer information about their attitudes, behaviors, opinions and beliefs. The success of survey research rests on how closely the answers that people give to survey questions matches reality that is, how people really think and act.

• The first problem that a survey researcher has to tackle is how to design the survey so that it gets the right information.

– Is this survey necessary? – Is the purpose of the survey to evaluate people or programs? – Can the data be obtained by other means? – What level of detail is required?

• The second problem is how accurate does the survey have to be?

– Is this a one-time survey or can the researcher repeat the survey on different occasions and in different settings?

– How will the results be used? – How easy is it to do the survey?

Why do a survey?

• The survey is an appropriate means of gathering information under three conditions:

– when the goals of the research call for quantitative and qualitative data,

– when the information sought is specific and familiar to the respondents and;

– the researcher has prior knowledge of the responses likely to emerge.

Use of Surveys by study design

• Descriptive research – Describe phenomena and summarize them. – The goal of using surveys for descriptive research is to

get a precise measurement of specific concepts i.e. depression, quality of life.

• Causal explanation – Measure associations i.e. chemotherapy and quality of life.– The data from surveys can provide a causal explanation to

phenomena such as why teens become pregnant or why teens do drugs.

• Evaluation – Efficacy of a program

• Prediction – Predict future events

Basic Survey Designs

• Cross-sectional surveys:– Data collected at one point in time selected to

represent a larger population

• Longitudinal Surveys:– Trend:

• Surveys of sample population at different time points

– Cohort:• Study of sample population each time data are collected but samples studied maybe different

– Panel:• Data collection at various time points with the same sample of respondents

Section Two

Measurement Levels and Types of Data

Section TwoMeasurement Levels and Types of Data

• OBJECTIVES

– Define and give examples of levels of measurement

– Define and give examples of scales – Contrast the types of measurement scales – Compare and contrast interval versus ordinal

scales

Key Terms Categorical, ordinal, scale, attitudes, opinions, beliefs, behavior, attributes

Selecting the types of data to collect

•The type of data you collect and how you collect them affects how you can use the data.

•Therefore, before you start data collection you must decide what will be measured and how it will be measured.

Types of data

• Categorical data: numbers or words are used to group things– Examples: gender, race, religion, food group, or place

of residence.

• Ordinal data: When the numbers are used to order a list of things– The ranking of football or basketball teams is done

using ordinal numbers. A list of things to do would also be ordinal data.

• Interval data.– Scale or interval data would be things like height,

weight, age.

Types of data collected

• There are six basic types of data that you might collect:

– Attitudes – Opinions – Beliefs – Behavior – Attributes (demographic characteristics) – Preferences

Basic types of survey questions

• The way a question or statement is worded and the response options offered determine the nature of the data received.

• Types of survey questions include:

– Open-ended response– Closed response – Semantic differential scales – Agreement and rating scales – Ranking scales – Checklists

Basic types of survey questions

• Open-ended question– Respondent writes response in own words– Considerations for using open-ended questions:

• Need to enter data by hand• Develop a coding scheme for responses• Content analysis?

– Frequently used in exploratory studies to facilitate better understanding of a concept

• Suggestion: it’s a good idea to always include an open-ended question giving the respondent the opportunity to add any additional comments.

Basic types of survey questions

• Open-ended question– Advantages:

• Allows the respondent to answer the question with few limitations

• Report more information than with discrete answers

– Disadvantage:• Need qualitative methods or coding system to analyze the responses

• Require subjective judgements

– Example:• What habits increase a person’s risk for being overweight?

• Describe the pain you experience with walking?

Basic types of survey questions

•Closed response: – These are the "multiple-choice" variety where a person has to choose among several possible answers.

– There are two types of closed response questions:•ordered answer choices represent points along a continuum. – Pain on a scale of 0 (none) to 10 (worst pain ever)

•unordered answer choices with each choice is an independent answer. – Examples: ethnicity and marital status

Basic types of survey questions •Closed response:

– Advantages:•Quicker and easier to answer•Easier to tabulate and analyze•List of possible responses helps participant understand the meaning of the question

•Suitable to multi-item scales designed to provide a single score

– Disadvantages:•Do not allow participants to express their own answers

•Set of answers may not be exhaustive•Must be clear about selection of items, one or as manay as applicable

Basic types of survey questions

•Closed response: – When question allows more than one answer it does

not force respondent to consider each answer- Not ideal.

– Better to use yes no and multiple items– Example:

Which of the following do you believe increases your risk for stroke?

Yes No Don’t know• Smoking• Overweight• Stress• Drinking alcohol

Hulley et al., (2007) Designing Clinical Research, 3rd Edition pg. 242

Basic types of survey questions

• Partially open response – Require specific, short answers that do not

encourage free expression. – are a compromise between closed response and open

response forms. – provide an “Other” category where a person can

provide additional information. – Example: blank spaces provided for the questions

on racial background and persons living with you.

Modes of Survey Administration

• Personal (face-to-face)

• Telephone

• Mail

• Web

• Combination of methods

How do you decide on the mode ofdata collection?

Population+

Characteristics Of The Sample+

Types of Questions+

Question Topic+

Response Rate+

$$ Cost $$+

Time

Modes of Survey AdministrationPersonal (face-to-face)

• ADVANTAGES: – Generally yields highest cooperation and lowest

refusal rates

– Allows for longer, more complex interviews – High response quality – Takes advantage of interviewer presence – Multi-method data collection

• DISADVANTAGES: – Most costly mode of administration – Longer data collection period – Interviewer concerns

Modes of Survey AdministrationTelephone

• ADVANTAGES: – Less expensive than personal interviews – RDD samples of general population – Shorter data collection period than personal interviews – Interviewer administration (vs. mail) – Better control and supervision of interviewers (vs.

personal) – Better response rate than mail for list samples

• DISADVANTAGES: – Biased against households without telephones, unlisted

numbers– Nonresponse – Questionnaire constraints – Difficult to administer questionnaires on sensitive or

complex topics

Modes of Survey AdministrationMail

• ADVANTAGES: – Generally lowest cost – Can be administered by smaller team of people (no field

staff)– Access to otherwise difficult to locate, busy populations – Respondents can look up information or consult with

others

• DISADVANTAGES: – Most difficult to obtain cooperation – No interviewer involved in collection of data – Need good sample – More likely to need an incentive for respondents

– Slower data collection period than telephone

Comparison of Modes of Survey Administration

Variable Mail Phone F/F

Cost Cheapest Moderate Costly

Speed Moderate Fast Slow

Response rate Low to moderate

Moderate High

Sampling need Address Telephone number Address

Burden on respondent

High Moderate Low

Control participation Of others

Unknown High Variable

Length of Questionnaire

Short Moderate Long

Sensitive questions

Best Moderate Poor

Lengthy answer choices

Poor Good Best

Open-ended responses

Poor Good Best

Complexity of Questionnaire

Poor Good Best

Possibility of interviewer bias

None Moderate High

Modes of Survey AdministrationWEB SURVEYS

• ADVANTAGES: – Lower cost (no paper, postage, mailing, data entry costs) – Can reach international populations – Time required for implementation reduced – Complex skip patterns can be programmed – Sample size can be greater

• DISADVANTAGES: – Approximately 40% of homes own a computer; 30% have home e-

mail – Representative samples difficult - cannot generate random

samples of general population – Differences in capabilities of people's computers and

software for accessing Web surveys – Different ISPs/line speeds limits extent of graphics that can

be used

Section Three

Elements of Item Construction

Section ThreeElements of Item Construction

• OBJECTIVES

– Identify types of bias in questionnaires – Identify and state how to correct common

wording problems – Define types of response sets

Key Terms Social desirability, acquiescence, bias,

midpoint, operational definition, response set

How Bias affects response?

Need we say more?

Writing questions

– Good Questions•Are clear and use simple language•Are concise•Are specific•Are possible to answer•Are relevant to the respondent•Do not use negatives•Avoid bias terms•Have only one part (not two parted question)

Writing questions

• Common Wording Problems – Writing questions for a particular survey means doing them

for • a particular population, • a particular purpose and • for placement next to other questions in the survey. • Words that are too difficult for some to understand may be perfectly acceptable for others. A question that is fairly vague may satisfy the objectives of one study but not the ones of another.

– So that every respondent will understand a question, it is important to keep the reading level at or below the average reading level of the population. Complex words may be replaced by simpler ones or ones more easily understood. If you are giving a survey to a particular group, you would want to use words that are common to the group.

Response Options

• Should reflect concepts you are trying to measure, and fit with the wording of the question– Avoid simple “yes” or “no” answers and attempt

to measure intensity if possible

• Mutually exclusive (select only one answer)

• Exhaustive (all possible answers are listed, including other or not applicable or don’t know)

Question Order

• Be attentive:– Initial questions affect answers to subsequent

ones

• Start with easy, salient, non-threatening questions near the end

• Cluster questions addressing the same topic or concept together.

• Avoid redundancy

Questionnaire Format

• As short as possible

• Visually attractive and nicely reproduced

• Readable (consider font size)

• Uncluttered

• Broken into logical sections if possible

• Clear skip patterns for contingency questions

• Clear spaces for respondents to mark answers– Boxes– Parentheses (X)– Or numbers to circle 1…2…3– Avoid lines to put check___

• Amount of space provided for open-ended questions will determine amount respondent will write.

Questionnaire FormatExample

• The following questions are about Instructions from the Medical outcomes study

Pain measures the pain or pains you experienced in the past 4 weeks. If you had more than one pain, answer the questions by describing your feelings of pain in general.

1. How much bodily pain have you generally had during the past 4 weeks?

(Circle one)None……………………………………………….1Very mild……………………………………….2Mild………………………………………………….3Moderate……………………………………….4Severe…………………………………………….5Very Severe………………………………….6

Questionnaire Format

•Helpful Hint…– Get consultation on questions and response options from statisticians and data management people.

Instructions to Respondents

• General instructions should be provided at the beginning of the self-administered survey– Brief explanation includes:

• Purpose• Significance of the answers• How to answer the questions• Stress: CONFIDENTIALITY OF RESPONSES

– Provide specific instructions to questions as needed

Instructions to RespondentsExample

• The following questions are about Instructions from the Medical outcomes study

Pain measures the pain or pains you experienced in the past 4 weeks. If you had more than one pain, answer the questions by describing your feelings of pain in general.

Pre-test (pilot)

• Be sure to pilot test the survey instrument before the actual administration, especially if it is a new instrument that you constructed or has not been used previously in your population of interest.

– Helpful Hint:• Also good idea to test your sample design, data collection methods, data processing and analysis if possible.

Pre-test (pilot)

• Use a similar population or identical population you will be measuring.

• Testing:– Question clarity

• Failure to answer?• Multiple answers?• “Other” answers and how to handle them in data entry• Qualified answers

– Questionnaire format• Instructions?• Flow?• Layout?

– Variance in responses• Consistency in responses among respondents

Pre-test (pilot)

• Revise instrument as needed

• Repeat pre-test– Ideally with some of the original pre-test

respondents and new respondents

Data collectionHints

•Mail survey should include:– Survey instrument– Return envelope

•Self-addressed and stamped

Data collectionHints

• Self-administered survey should include:– Cover letter

• On letterhead• Signed by PI or most significant person possible• Brief explanation of

– Purpose– How results will be used– Why respondent was selected– Why answers are important– Emphasize confidentiality (if applicable)– Provide contact information for questions– Describe/explain any incentives for participation or

consequences or not for non-participation

Monitor survey returns

• Open surveys as they are returned

• Assign unique ID number (if not already assigned)

• Stamp date on survey

• Track number returns daily– Used to inform timing of follow-up reminders

Monitor survey returns

•Optimal response occurs with 2 reminders– Timing

•Based on returns (1-2 weeks after original)

– Materials (mail survey)•Reminder card to have new survey sent or•Send entire packet again

Section Four

Assessing Reliability and Validity

Section fourAssessing Reliability and Validity

• OBJECTIVES – Compare and contrast types of reliability – Compare and contrast types of validity

Key Terms

Face validity, content validity, concurrent validity, discriminant validity, construct validity, test-retest reliability, internal consistency, stability.

Reliability and validity

•In order for information to be useful, it has to be:– consistent, – dependable,– accurate and, – most of all, true.

•Too often, we are presented with information that fails on one or more of these criteria.

•In research, these criteria are represented by the concepts of reliability and validity.

Reliability and validity

• Reliability: – Expect to obtain the same information time after

time.• Assessed by correlation coefficient • The concept of reliability can be applied to sampling • If we repeatedly draw random samples of equal size from a population, we can expect to get the same sample values each time (plus or minus a certain amount due to sampling error).

• Validity:– Measures the concept intended to measure

• Instrument is presented or used in the way for which it was intended

• An IQ test is valid only if it is used to measure intelligence it is not valid if it used to assign individuals to groups.

• A psychological test that is a valid measure of anxiety is not a valid measure of stress.

• Face validity – Information collected appears to be what was expected. (Face

value)– A question that asked Do you smoke? would appear to have

face validity as a measure of smoking behavior.

• Content validity – A question adequately reflects the underlying behavior or

body of knowledge. – Content validity is established by having a panel of experts

evaluate and agree on the relevance of the test items.

• Concurrent validity – One instrument or question is comparable to another that has

been shown to validly measure the same content or construct. – Concurrent validity is established by correlating one

question with another that has previously been validated.

Types of validity

• Discriminant validity – A question or survey that is able to discriminate between group

differences. – Example

• Depressions scales have discriminate validity if individuals who are depressed score differently from those who are not clinically depressed.

• Predictive validity – A question can be used to predict behavior. – Example:

• Can you walk 5 blocks?

• Construct validity– A construct is a theoretical dimension like self-esteem that is measured

by having several questions that all relate to how people view themselves. – Self-esteem does not exist by itself but is represented by how people

respond to these questions. – In this example, construct validity measures the extent to which these

responses can be called, self-esteem.

Types of validity

Types of reliability

• Test-retest reliability – Obtained by administering the same test on two or more

successive occasions and then correlating the scores. – Statistic that reflects reliability is correlation

coefficient, higher is better.

• Internal consistency– Obtained by correlating the scores on several questions that

pertain to the same content to the sum total of the scores. – The average item-total correlation is a measure of how

consistently people respond to related items on a test. – For example: if a math test had several sets of items that

required people to multiply two and three digit numbers, you would expect persons who could not correctly multiply two digit numbers to be unable to multiply three digit numbers. Likewise, you would expect persons who can multiply three digit numbers to be able to multiply two-digit numbers. These expectations should be consistent from person to person and would be borne out by high item-total correlations of the items.

Types of reliability

• Stability – How much variation exists in scores upon repeated

administrations of the instrument.– Stable measures will reproduce the same score on

repeated administrations of the instrument. – This concept is similar to test-retest except

that in test-retest situations there is no assumption that the absolute value of each persons test score will stay the same.

– Considerations:• Time between administration, if too close may be remembering answers

• Learning effect on repeated administrations

Section Five

Determining the Coding System

Section Five

Determining the Coding System • OBJECTIVES

– Develop a coding scheme that reduces a data to quantifiable terms.

– Compare and contrast different types of coding schemes

– List coding requirements of various computer programs

Key Terms coding, alphanumeric, missing values, fixed field, free field, recoding, reconversion

The fine art of coding

– Coding:• The process of converting information into a quantifiable format (usually numerical) so that a systematic analysis of the information can be done.

• The coding process is made easier by precoding responses and enabling circled or checked numbers to be entered directly into a data set.

• The codebook is like a foreign language dictionary in that it translates English responses into numerical or categorical values.

• The coding system used should model or be easily convertible to the format required by the computers software.

– If the information is categorical in nature, then, the choice of whether to represent a data value as a number or a label depends upon the analyses that will be done.

– If you are only going to measure frequency counts and percentages, then either a number or label is acceptable.

The fine art of coding

• Statistical analyses requires numbers.

• Labels cannot be manipulated mathematically.

• Qualitative data requires content analysis– data elements might be entered as words and phrases

instead of numbers.

• Missing values– When data cant be collected on one or more variables from

one or more source. – Statistical computer programs deal with missing values in

different ways. – Generally, they give you the option of identifying what

values are to be considered Missing, or allowing the computer to assume that zeroes or blank spaces represent missing values.

– Should you assign a number to represent missing values? – Yes, if you intend on either analyzing the missing values

or replacing them with other values. – You must identify or define your missing values before

doing any analyses.

Data organization

• When data is entered into a computer, it will generally be organized in a spreadsheet orientation where the rows represent different cases and the columns represent different variables.

• If you leave spaces blank instead of typing in a value, you may not know if the space represents a missing value or a missing data entry.

• By entering values in every space provided, you will know exactly what data has or has not been collected.

Section Six

Deciding How to Enter and Store the Data

Section Six Creating the database structure

•OBJECTIVES – Define the elements of a database – Describe the steps in creating a database – Describe how to use a form to enter data

Key Terms database, table, records, fields

Creating the database structure

•Generally, the first column of a data table will have an unique ID number that will identify the survey.– This is to allow the researcher to go back and check the data on the survey with the data entered on the computer.

– The next columns of the table will usually contain the data from the survey in the order of the questions on the survey.

Creating the database structure

To make it easier to identify and manipulate the data, each column is given a unique short name that refers to the question number or subject on the survey.

o If the questions on a survey are numbered, then an easy way to label the columns would be Q1, Q2, etc

Section Seven

Data Entry, Data Checking and Quality Control

Section Seven:Data Entry, Data Checking and Quality

Control•OBJECTIVES

– Define the variables of interest in the study – Demonstrate procedures for developing a questionnaire

– Define the major methods of data collection in quantitative studies and qualitative studies

– Demonstrate how to determine if responses are random and representative

Key Terms variable, coding scheme, tabulation, missing data, non-response bias

Data entry and quality control

• Data entry is the process of transferring or transcribing the information from coding sheets, field notes, questionnaires and surveys onto a computer. – The process of data entry can be facilitated

by use of machine-readable sheets or by image scanners and optical character recognition software • manual data entry is not required.

– Important to check the accuracy of the data– Analyses hinges on the accuracy of the data,

good quality control procedures are needed for error checking and entry consistency.

Data entry and quality control

• There are computer programs that facilitate the quality control function of data entry directly by checking for acceptable values at the time they are entered. – Example:

• if you recording ages of adults in a study, values lower than 18 would be errors.

• Manual checking of the data may or may not spot the error.

– The data entry program would immediately reject any entered value that does not fall within the accepted range or is not of the proper form.

Data coding problems

• Missing data handling – Prepare for nonresponse as treating it as one of the

categories (example, a No response). – Other situations, you might want to assign the

average value to retain the rest of the data for analyses that require it.

– Individual item analysis, drop the missing data adding constants dont change the overall results.

– For statistical purposes, to keep as much of the original sample size as possible.

Data coding problems

• Another solution: Regression– can be used to estimate missing data using answers to other

responses. – The problem may be avoided by having more than one question dealing

with a topic. – Not everything you ask will be of equal importance some items you

may afford to lose data. – The main problem with nonresponse is that you dont know if someone

skipped a question or decided not to respond.– Including a non-response category will ensure that people respond. – Remind them both at the beginning of the questionnaire and at the

end to answer ALL times. – Dont give people a way not to respond give them the opportunity to

respond. – You can identify respondents who essentially are non-participants:

people who mark the middle response for all items. Only slightly better than nonresponders.

– Do you keep them in the data bank? No, since their responses will

skew the data

Avoiding bias from non-respondents

• Its important to know whether those who did not respond differ greatly from those who responded. – Relatively few refusals provide the

theoretical potential for introducing considerable error into estimates of the sample characteristics.

– The extent of the differences between respondents and nonrespondents can seldom be determined.

– Indirect methods may be used.

Avoiding bias from non-respondents

• Mail questionnaires are also susceptible to other factors that affect response rate. – Interest in the subject of the questionnaire may

affect ones answers and also determine if the questionnaire will be completed and returned.

– Another consideration is the ability of the respondent to provide written responses.

– People with less education are likely to be underrepresented, partly because of lower educational attainment, but also because of more difficulties with their seeing and writing capabilities.

Web Resources:

• http://gsociology.icaap.org/methods/surveys.htm

• http://athome.harvard.edu/dh/vsr.html

• http://www.socialresearchmethods.net/kb/survtype.php

• http://www.hcp.med.harvard.edu/statistics/survey-soft/

• http://gking.harvard.edu/projects/survey.shtml

• http://www.whatisasurvey.info/

• Http://www2.acs.ncsu.edu/UPA/surveyt/uapr.survey_research/

• http://www.srl.uic.edu

Resources

http://gsociology.icaap.org/methods/surveys.htm

• Hulley, Cummings, Browner, Grady, Newman (2007). Designing Clinical Research, 3rd Edition. Philadelphia. Lippincott, Williams & Wilkins.

• ESSENTIALS OF SURVEY RESEARCH AND ANALYSIS A WORKBOOK FOR COMMUNITY RESEARCHERS © 1998 Developed by Dr. Ronald Jay Polland for the Adolescent Pregnancy Prevention Grant, Duval County Health Department.

• Babbie, Earl (2002), The Basics of Social Research 2nd ed. Wadsworth Thomson Learning: CA.

• Dillman, D.A. (2000), Mail and Internet Surveys 2nd ed. Wiley: NY.

• Fink, Arlene and Jacqueline Kosecoff (1998), How to Conduct Surveys. Sage: CA.

• Fowler, Floyd J. (1993), Survey Research Methods. Sage: CA.

• Fowler, Floyd J. (1995), Improving Survey Questions. Sage: CA.

• Sudman, Seymour and Bradburn, Norman (1982), A Practical Guide to Questionnaire Design. Jossey-Bass: San Francisco.

• Tourangeau, R., and Smith, T.W. (1996), “Asking Senstive Questions: The Impact of Data Collection Mode, Question Format, and Question Context, “ Public Opinion Quarterly, 60:275-304.

• Also see American Association of Public Opinion Research, “Best Practices for Survey and Public Opinion Research.” http://www.aapor.org/ethics/best.html

Resources

http://gsociology.icaap.org/methods/surveys.htm

• Aday, L. A. (1996). Designing and Conducting Health Surveys, 2nd ed. San Francisco: Jossey-Bass.

• Biemer, P., Groves, R., Lyberg, L., Mathiowetz, N., & Sudman, S. (eds.) (1991). Measurement Errors in Surveys. New York: Wiley.

• Dillman, D. (1978). Mail and Telephone Surveys: The Total Design Method. New York: Wiley.

• Dillman, D. (2000). Mail and Internet Surveys: The Tailored Design Method. New York: Wiley & Sons.

• Fink, A., & Kosecoff, J. (1985). How to Conduct Surveys: A Step-by-step Guide. Beverly Hills, CA: Sage, 1985.

• Fowler, F. J., Jr. Survey Research Methods, 2nd ed. Newbury Park, CA: Sage, 1993.

• Groves, R. (1989). Survey Errors and Survey Costs. New York: Wiley, 1989.

• Groves, R., Biemer, P., Lyberg, L., Massey, J., Nicholls, W., II, & Waksberg, J. (eds.) (1988). Telephone Survey Methodology. New York: Wiley.

Resources

• Lavrakas, P. J. (1993). Telephone Survey Methods: Sampling, Selection, and Supervision. Newbury Park, CA: Sage.

• Lessler, J. T., & Kalsbeek, W. D. (1992). Nonsampling Error in Surveys. New York: Wiley.

• Lyberg, L., Biemer, P., Collins, M., deLeeuw, E., Dippo, C., Schwarz, N., & Trewin, D. (eds.) (1997). Survey Measurement and Process Quality. New York: Wiley.

• Marín, G,. & Marín, B. V. (1991). Research with Hispanic Populations. Newbury Park, CA: Sage.

• Turner, C. F., & Martin, E. (eds.) (1984). Surveying Subjective Phenomena (2 volumes). New York: Russell Sage. Journals: Public Opinion Quarterly and Journal of Official Statistics

PACCTR* Curriculum Core

• Roberta Oka RN, ANP, DNSc, School of Nursing

• George Sawaya MD, School of Medicine

• Rebecca Jackson MD, School of Medicine

• Susan Hyde DDS, MPH, PhD, School of Dentistry

• Jennifer Cocohoba PharmD, School of Pharmacy

• Joel Palefsky MD School of Medicine

* Pathways to Careers in Clinical and Translational Research