Upload
theodore-harrison
View
235
Download
1
Tags:
Embed Size (px)
Citation preview
Getting Started…
“A telephone survey says that 51 percent of college students drink until they pass out at least once a month. The other 49
percent didn't answer the phone. “-Craig Kilborn
The Following Topics will be Discussed:
Theoretical Tradition and Epistemological Paradigm
Most Appropriate Types of Research Questions Major Features of the Method Major Criteria for Rigour Strengths and Weaknesses Designing a Project using the Method
Theoretical and Epistemological Background
In general, a survey involves the collection of information from a large group of people or a population, commonly via:Opinion SurveysPolitical PollsTV Viewing PollsHowever…
Theoretical and Epistemological Background
…our focus is on survey research, which is conducted to advance scientific knowledge or develop theory.
Survey research is frequently utilized within the Social Sciences, including disciplines like Psychology, Marketing, and Organizational Behaviour.
Positivism
Positivism has become a dominant institutional form in social research.
However, during the 1970's and 1980's prominent concerns were raised about the limits of quantitative data and methods often associated with positivism; including survey research designs.Eg: Concerns explained.
Positively More Positivism
Assumption: Objective world which science can “mirror” with privileged knowledge.
Key Focus: Search for contextual and organizational variables which cause organizational actions.
Key Theories in Paradigm: Contingency theory, systems theory, population ecology, transaction cost economics of organizing, dustbowl empiricism, etc.
Goal of Paradigm: Uncover truth and facts as quantitatively specified relations among variables.
Positively More Positivism II
Criteria for Assessing Research: Prediction=Explanation, Rigour; internal/external
validity, reliability. Unit of Analysis: The variable. Research Methods and Types of Analysis:
Experiments, questionnaires, secondary data analysis, quantitatively coded documents.
Quantitative: Regression, Likert scaling, structural equation modelling.
Qualitative: Grounded theory testing.
Major Features of the Method
Variables are often operationalized when researchers ask people questions as a way of getting data for analysis and interpretation (the questions are either asked by an interviewer, or written down and given to respondents for completion)
2 Forms of surveys: Questionnaire: document containing questions and others
types of items designed to solicit information appropriate for analysis
In dept Interviewing: ask people questions in order to gather data
Survey research is especially appropriate for making descriptive studies of large populations
Data collected may be used for explanatory purposes
The Beauty of Questionnaires
It provides a method of collecting Data by either asking people questions or asking them to agree or disagree with statements representing different points of view
Used primarily in survey research, but also in experiments, field research and other modes of observations
Questions can be open ended (respondents supply their own answer-similar to a short answer question on an exam), answers can be in the form of writing on paper or verbally reporting answers to an interviewer
Example: what is your opinion on abortion? It can also be closed ended (select from a list of answers provided)
This type of questionnaire is more common than open-ended ones because they provide greater uniformity of responses, and are more easily processed
Example: From a range of 1-5, how would you rate this hotel? With 1 being poor, and 5 as excellent
A Closer Look at Closed Ended Questions
Two structural requirements for closed ended questions: 1. Response categories provided should be exhaustive (should
include all possible responses that might be expected-even if answers may be obscure to you and me). Usually researchers try to ensure this by adding a category such as “other” and may follow “please specify”
2. The answer category must be mutually exclusive: respondent should not be compelled to select more than one to ensure this, we must carefully consider each combination of categories, could the respondent reasonably choose more than one answer?
If our survey is constructed carefully, we usually do not need to add an instruction to tell respondents to select the “best answer”.
Things to Note for whenAsking Questions
Make Items Clear: questionnaire items should be precise so that the respondent knows exactly what the researcher is asking
Avoid Double-Barrelled Questions: Researchers asking respondents for a single answer to question that actually has multiple parts
Respondents must be competent to answer: When asking respondents to provide information, must keep in mind whether they can do so reliably
Respondents must be willing to answer: Often, we would like to learn things from people that they are unwilling to share with us
Continued…
Questions should be relevant: Questionnaire should be relevant to most respondents
Shorts Items are the best: respondents are often unwilling to study an item in order to understand it, respondent should be able to read something quickly, understand the main point, and select an answer without difficulty, answers should not be misinterpreted
Avoid Negative Items/”Negative” words: appearance of a negation in a questionnaire items paves the way for easy misinterpretation
Avoid Biased Items and Terms: Meaning of someone’s response to a question depends in large part on its wording, some questions seem to encourage particular response more than do other questions
Three Main Methods for Administering Survey Questionnaires
Self administered questionnaires: respondents are asked to complete the questionnaire themselves (example: Mail survey), administer questionnaire to a group of respondents gathered at the same place at the same time.
Surveys administered by interviewers in face-to-face encounters
Surveys conducted by the telephone
Mail Distribution and Return
Research worker can either hand deliver questionnaires, requesting that respondents mail the completed questionnaires to the research office or after questionnaires are mailed, researcher visits homes to pick them up and check for completeness
Completion rate seems to be higher when research worker delivers the questionnaire, picks it up or both.
Basic method of collecting data through mail is to send questionnaire, with a letter of explanation and a self addressed, stamped envelope for retuning (for example, like the surveys we receive in our mailboxes, many often have the “postage paid” indicated in the corner of the envelope)
The Main reason for not returning questionnaires is that it’s TOO MUCH TROUBLE!
Acceptable Response Rates
Response rates: the number of people participating in a survey divided by the number selected in the sample (percentage form)
Inferential statistics used with survey analysis assumes that all members of the initial sample complete and return their questionnaires
Because that is very rare to happen, response bias is a concernresearchers often hopes for the possibility that the respondents look essentially like a random sample of the initial sample, and a somewhat smaller random sample of the total population
Continued…
Overall response rate is one guide to the representativeness of the sample reponsdentshigh response rates means less chance of significant response bias, with low response rates , correspondents are likely to differ from the respondents in ways other than their willingness to participate in the survey
Rough guides (with no statistical basis) to what is a good response rate: 50% is adequate for analysis and reporting, 60% is good and 70% is very good
Interview Surveys
Interviewers ask questions orally and record respondents answers Usually done in a face-to-face encounter, but telephone interviewing
follow similar guidelines Most interviews require more than one interviewers, but small scale
interviews can be done by one person Researchers must assume that a questionnaire item will mean the
same thing to every respondent and every given response must mean the same when given by different respondents
The interview’s presence should not affect a respondent's perception of a question or the answer given
To save time and money, a given interviewer is typically assigned to complete all the interviews in a particular geographical area
General Guidelines for Survey Interviewing
Appearance and Demeanor: interviewers should dress in a fashion similar to that of the people they’ll be interviewing (cleanliness and neatness in modest apparel). Dress and grooming are typically regarded as signs of a person’s attitudes and orientations. Interviewers should be pleasant, must communicate interest in getting to know the respondent without appearing to spy (never too casual or clingy). Interview will be more successful if the interviewer can become the kind of person the respondent is comfortable with, respondents deserve the most enjoyable experience the researcher can provide
Continued… Familiarity with Questionnaire: must be able to read questionnaire items to
respondents without error and stumbling over words, lines must be read as though they are part of a natural conversation
Interviewer must be familiar with the specifications prepared in conjunction with the questionnaire
Some questions will not fit a given respondent's situation, interviewer must determine how the question should be interpreted in that situation
It would be better for the interviewer to leave a question unanswered than to spend a period of time searching though the specifications for clarifications or trying to interpret the relevant instructions
Following Question Wording Exactly Recording Responses Exactly Probing for Responses: Sometimes respondents will give an
inappropriate/incomplete answer, request for an elaboration can be useful (Probe). Probes must be completely neutral, must not affect the nature of subsequent response
Advantages of Doing an Interview Survey
They typically attain high response rates (a properly designed and executed interview survey ought to achieve a completion rate of at least 80-85%)
Presence of and interviewer decreases the number of “don’t knows:” and “no answers”, since clarification can easily be done on the spot
Interviewers can clarify matters if respondent clearly misunderstands the intent of the question
Can observe respondents as well as ask questions
Telephone Surveys:Positive Factors (+)
Saves both money and time May dress anyway you please without affecting the
answers respondents give Respondents may be more honest in giving socially
disapproved answers (no eye to eye contact) Interviewers can communicate a lot about themselves
over the phone, even though they can’t be seen Allow greater control over data collection if several
interviewers are engaged in the project
Telephone Surveys:Negative Factors (-)
Some phone numbers are unlisted (but this has been erased through a technique called random digit dialling)
Bogus surveys-ones that are actually sales campaigns disguised as research
The ease in which people can hang up Answering machines (using machines to screen
calls)however research has showed that this had not yet had a significant effect on the ability of telephone researchers to contact prospective respondents
Comparing theThree Different Methods
Self Administered Questionnaires
Interview Surveys Telephone Surveys
Cheaper and quicker Produce fewer incomplete questionnaires
Cheaper and can be mounted and excited quickly
Costs no more to conduct a national survey than a local one of the same sample size
More effective for complicated surveys
Safer when interviewing high crime areas
Require a small staff Conduct survey based on sample of addresses or phone numbers rather on names
Impact of interviewers on responses is somewhat lessened when they can’t be seen
Computer Assisted Telephone Interviewing (CATI)
Increasingly used by academic, government and commercial survey researchers
Central computer is programmed to select a telephone number at random and dials it
Interviewer introduces study and asks the question displayed on the screen and then types that answer into the computer terminal (depending on the question-open/close ended)
Computer automatically prepares the data for analysis, or researcher can begin analysing data before the interview is complete (gaining and advanced view of how the analysis will turn out)
New Technologies andSurvey Research
CAPI (Computer assisted personal interviewing): face to face interviews rather than over the phone
CASI (Computer assisted self interviewing): respondent reads questions on the computer screen and enters his/her own answers
CSAQ (Computerized self administered questionnaire): respondent receives the questionnaire on a floppy disk, bulletin board, or other means and completes the questions, software then accepts the answers. Respondent then returns the data file
TDE (Touchtone data entry): initiates process by calling a number at the research organization, prompts a series of computerized questions, respondent answers by pressing the keys on the telephone pad
VR (Voice recognition): same as TDE, but the system accepts spoken responses
Internet and world wide web
Do’s and Don’ts for Conducting Online Surveys
DO Consistent wording between the
invitation and the survey Use plain simple language Offer to share selected results from
others who also have completed the survey
Plan the time of day and day of week to mail (when will respondents most likely be reading mail at home?)
Be aware of technical limitations (Will respondents have programs needed to access)
Test incentives, rewards and prize drawings to determine the optimal response
Limit studies to >15 minutes
DO NOT Use terms such as
unique ID number in the invitation, then ask respondents to type “password” when they get to the survey
Force the respondent to scroll down the screen for the URL for the study location
Secondary Analysis
Survey research involves 3 steps: questionnaire construction, sample selection, and data collection (interviewing/self administered questionnaires)
This is a form of research in which the data is collected and processed by one researcher are reanalyzed- often for a different purpose, by another
Example: General Social Survey (GSS) Advantage: cheaper and faster than doing original surveys, benefit
from the work of top flight professionals, enhance possibility of meta analysis (researcher brings together a body of past research on a topic)
Disadvantage: question of validity (have no assurance that data collected will be appropriate for you research interests)
Rigour in Survey Research
1. Internal Validity To be internally valid, the conclusions of the
research must be supported by the data. Internal validity is judged according to the
accuracy with which a description of particular events represents the data.
The essence of internal validity for survey research is complete confidence that your conclusions come from the data.
Reliability
2. Reliability Reliability is generally concerned with
replication: an account is considered to be reliable if the data are reproducible.
If the analytic strategy were repeated by the same or different investigator, then the results should be the same.
Another Type of Validity
3. External Validity External validity or generalizability for survey
research is captured by the question, “How can one determine the extent to which the findings of a particular inquiry have applicability in other contexts or with other subjects?”
Essentially, the extent to which the effect can be generalized to populations, settings, treatment variables, and measurement variables.
External Validity Continued…
This can also refer to “fit” or the degree to which the audience or reader of the report is able to transfer the research findings to contexts outside of the study situation to other settings.
The researcher must supply a substantial amount of clear and detailed information or thick description about the issue/phenomenon studied and the setting in which that issue/phenomenon was found.
The degree of transferability is a direct function of the similarity or “fittingness” between the two contexts.
Strategies for Ensuring Rigour
1. Verification Strategies The most important way to ensure that
research is rigourous is to focus on verification during the study.
Verification is the process of checking, confirming, making sure, being certain.
Doing this as the study is conducted is key as the researcher then can identify and correct threats to reliability and validity as they surface.
Initial Strategies Address:
Investigator Responsiveness Methodological Coherence Sampling Data Analysis Thinking Theoretically
Additional Strategies
There are other strategies that can be used during the research to contribute to rigour, namely:
Prolonged Engagement Participant Checks Journal Writing Peer Review Audit Trail.
Weaknesses for Survey Research
Lacks the context of social life Appear superficial on coverage of complex
topics Subject to artificiality Surveys can also be inflexible because the
initial study design must remain the same Therefore, surveys are weak on validity
Strengths for Survey Research
Describes the characteristics of a large population Allows a large sample of respondents Surveys are flexible, allows flexibility in your analyses Surveys have a strength in measurement Helps determine unemployment rates, and voting
intensions Helps examine official documents such as marriage,
birth, or death records Therefore, surveys are strong on reliability
Survey Design
1. Purpose 2. Respondent Group 3. Questionnaire Construction 4. Administration of Questionnaire 5. Analysis Example: Sample Questionnaire (Handout)Student Opinion Survey on Group Assignments in
University Courses
Additional Resources
Presentation Website:http://members.shaw.ca/kristopher.skinner/soc315
Group Project E-mail: [email protected]