© (2015, 2012, 2008) by Pearson Education, Inc. All Rights Reserved Chapter 12: Survey Designs Educational Research: Planning, Conducting, and Evaluating

© (2015, 2012, 2008) by Pearson Education, Inc. All Rights Reserved

Chapter 12: Survey Designs

Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research

Edition 5

John W. Creswell

Creswell, Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, 5th Ed. © (2015, 2012, 2008) by Pearson Education, Inc. All Rights Reserved 12-2

By the end of this chapter, you should be able to:

Define survey research, and describe when to use it, and how it developed

Describe the types of survey designs Identify the key characteristics of survey research Describe how to construct and use a mailed

questionnaire Describe how to design and conduct an interview survey Identify potential ethical issues in survey research List the steps in conducting survey research Identify criteria useful for evaluating survey research


What Is Survey Research?

Survey research designs are procedures in quantitative research in which investigators administer a survey to a sample or to the entire population of people in order to describe the attitudes, opinions, behaviors, or characteristics of the population.


When to Use a Survey

To assess trends To assess opinions, beliefs, and

attitudes of individuals For follow-up analyses For program evaluation


The Development of Survey Research

1817 International Survey of Education Systems 1890 Stanley Hall survey of children 1907 The Pittsburgh Survey examined social problems WWI and WWII sampling techniques improved Wide applications in social sciences-marketing research,

public opinion research, journalism Universities established social science research centers Private organizations such as Gallup, Rand Corporation,

and Roper formed Internet-based surveys widely used


The Types of Survey Designs

Changesin a

subpopulationgroup identified

by a commoncharacteristic

over time

Time of Data Collection

Study Over TimeStudy at One Point in Time

Longitudinal Cross-sectional

Changes in thesame

peopleover time

Trends inthe same

populationover time

Attitudesand

Practices

CommunityNeeds

ProgramEvaluation

TrendCohort

Panel GroupComparisons

NationalSurvey


Key Characteristics of Survey Research Sampling from a population Collecting data through questionnaires

or interviews Designing instruments for data

collection Obtaining a high response rate


Population, Target Population, Sampling Frame, and Sample

The PopulationThe group of individuals having one characteristic that distinguishes them from other groups.

The Target Population or Sampling FrameThe actual list of sampling units from which the sample is selected.

The SampleThe group of participants in a study selectedfrom the target population from which theresearcher generalizes to the target population.


Reducing Error in Survey Research

Reducing coverage error: Have a good sampling frame list on which to select individuals

Reducing sampling error: Select as large a sample from the population as possible

Reducing measurement error: Use a good instrument with clear, unambiguous questions and response options

Reducing nonresponse error: Use rigorous administration procedures to achieve as large a return rate as possible


Forms of Data Collection Based on Who Completes or Records the Data

Who completes or records the data?

Participant Researcher

MailedQuestionnaire

ElectronicQuestionnaire

One on oneOne

To aGroup

OverTelephone

IndividualInterview

Focus GroupInterview

TelephoneInterview


Steps in Designing an Instrument Write different types of questions Use strategies to construct good

questions Perform a pilot test of the questions


Aspects of Survey Instrument Design: Types of Questions Personal Attitudinal Behavioral Sensitive Closed-ended Open-ended


Question Construction Problems Question is unclear because of vague words Two or more questions in one Question is wordy Question contains negatives Question contains jargon Overlapping response options


Question Construction Problems (cont’d)

Unbalanced response options Mismatch between questions and response options Respondent does not have understanding to

answer the question Not all respondents can answer the question—

need branching


Pilot Testing

Test on a small number of individuals in the sample

Ask for written feedback on the questions

Revise the survey based on the written comments

Exclude the pilot participants from the final sample for the study


Strategies to Improve Response Rates Prenotify participants Use follow-up procedures Study a problem interesting to the

population under study Use a brief instrument Consider the use of incentives


Three-Phase Survey Administration Procedure

Step 1:First Mailing

of Survey

Step 2:Second Mailing

of Survey

Step 3:PostcardMailing

2 Weeks 2 Weeks 2 WeeksStart 6 Weeks

Time


Response Bias The responses do not accurately reflect

the views of the sample and the population. Overly positive or negative

Assess response bias, particularly when response rates remain low

Wave analysis


Construct and Analyze a Mailed Questionnaire

Write a cover letter to invite the participants to complete the questionnaire

Form and construct the questionnaire Identify what statistical procedures will be

used to analyze data from the mailed questionnaire


Planning and Designing an Interview Survey

The interviewer should maintain a neutral stance during the interview

Train the interviewers prior to the interview Take good notes of responses or use an

audio recorder For telephone interviews, develop a telephone

interview guide prior to the interview


Potential Ethical Issues in Survey Research

Overstating the benefits of participating in the study

Placing interviewers or participants in unsafe situations

Protecting confidentiality of survey responses Disclosing identity of individuals through data

analysis of a subset Not destroying instruments at the conclusion

of the study


Steps in Conducting Survey Research1. Decide if a survey is the best design

to use2. Identify the research questions or

hypotheses3. Identify the population, the sampling

frame, and the sample4. Determine the survey design and

data collection procedures


Steps in Conducting Survey Research (cont’d)

5. Develop or locate an instrument6. Administer the instrument7. Analyze the data to address the

research questions or hypotheses8. Write the report


Evaluating Survey Research

Describes the target population Identifies and uses a systematic approach

to identifying the sample Identifies the size of the sample and

means for identifying the sample size Identifies the type of survey used Mention survey instrument for data

collection


Evaluating Survey Research (cont’d)

Reports the reliability and validity of past scores on the instrument

Discusses procedures for administering the instrument

Administration procedures provide a discussion about the follow-up procedures to ensure a high return rate

Provides a systematic procedure for analyzing the survey data

Documents

© (2015, 2012, 2008) by Pearson Education, Inc. All Rights Reserved Chapter 12: Survey Designs Educational Research: Planning, Conducting, and Evaluating