Upload
nash
View
19
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Overview of Research Process. Ethics. Develop Hypothesis. Research Design. Review Research. Communicate Results. Data Analyses. Select Question. Measurement. Current Focus. Research Designs. Max Precision. Max Context. Runkey & McGrath typology. Max Generality. Data Analyses. - PowerPoint PPT Presentation
Citation preview
SelectQuestion
ReviewResearch
DevelopHypothesis Research
Design
Measurement
Data Analyses CommunicateResults
Ethics
Overview of Research Process
Research Designs
Measurement
Data Analyses
Max Precision
Max Context
Scaling
Reliability & Validity
Max Generality
Current Focus
Qualitative
Runkey & McGrath typology
Particular Behavior SystemsUniversal Behavior Systems
ObtrusiveOperations
UnobtrusiveOperations
NaturalSettings
ContrivedSettings
FieldStudies
FieldExperiments
LabExperiments
MaximumContext
MaximumPrecision
MaximumGenerality
Formal Theory
Sample Surveys
SettingIndependent
Behavior notmeasured
ComputerSimulations
Runkel& McGrath, 1972
ExperimentalSimulations
How to do field research?
What is field research?• Examples
– Field Studies• Cross sectional
– Field Experiments• E.g., Longitudinal, prog evaluation
• Similarity and differences from– Other methods of data collection
• Large-scale (Sample) Surveys
– Methods statistical analyses• Correlational
Particular Behavior SystemsUniversal Behavior Systems
ObtrusiveOperations
UnobtrusiveOperations
NaturalSettings
ContrivedSettings
FieldStudies
FieldExperiments
LabExperiments
MaximumContext
MaximumPrecision
MaximumGenerality
Formal Theory
Sample Surveys
SettingIndependent
Behavior notmeasured
ComputerSimulations
Runkel& McGrath, 1972
ExperimentalSimulations
Why do field research?: General reasons
Describe Phenomena How satisfied are the employees
Establish standards How satisfied are our employees compared to another organization
Establish value added by a program
The effect of the new benefits program on employee satisfaction
Make decisions Should we continue with the new benefits program
Validate/test intuitions
Everyone else is using the new benefits program, is it any good?
Identify source of problem & potential solutions
Why are our employees dissatisfied? How to increase their satisfaction?
Describe
Predict
Explain
• Type of organizational change & development• E.g., self & peer evaluation of oral presentation
(Radhakrishnan & Yang, 2006)
• Two-way (symbolic) communication channel between employees & organization via content and conduct – e.g., UT Employee Survey
• Cox, T. Jr (2001). Creating the Multicultural Organization: A Strategy for Capturing the Power of Diversity San Francisco, CA: Jossey Bass
Why do field research?Organization-specific reasons
After deciding why you are doing field research, decide
how you will collect data
• Types of Data Collection Methods – Numerical vs. Non-numerical– Oral/Written vs. Observational– Behavioral vs. non-behavioral
• Each of the above types of data can be collected via all or some of the following– Questionnaires/Surveys– Observation (Archival)– Interviews
Methods of data collection• Bias in any one method is overcome if you
use multiple methods– Cf choosing research designs
• Some methods are better suited for measuring certain kinds of concepts – E.g., willingness & ability should determine
use of self report• Stereotype research
• Amount of resources used by method– Researchers resources– Participants’ resources
Time & resources restrict you to certain methods of data
collection
• Questionnaires • E.g., Field study, cross-sectional data
• Archival data• E.g., Field studies, Sample (large scale) Surveys
– If using, justify measures w/logic & research• e.g., ESL indicators
• Qualitative (non-numerical) data will take too long for collection & analyses
Instructor-Generated Exampleof a Questionnaire
• Hypothesis based on Rode et al., 2005, AOMLE – Additional control variable
• Renner, M. & Mackin R. (2000). A life stress instrument for classroom use in M. Ware & D. Johnson (Eds.) M. Handbook of demosntrations and activities in the teaching of psychology: Vol 1 Lawrence Erlbaum: Marwah, NJ.
Before designing your questionnaire identify
• Research hypothesis
• Predictor, criterion & explanatory variables
• Pre-existing measures of predictor & criterion variable– Bonus if you have measure of explanatory
variable
Why identify pre-existing measures for your questionnaire?• Examples of pre-existing measures
– Found in books on Reserve at CIRHR library– Psycinfo database:
• Search: Measures OR Questionnaires AND your topic keyword
• Why use pre-existing measures– Improves statistical reliability of your study– Improves validity of your study
• Disadvantages of pre-existing measures– E.g., UT study
How pre-existing measures improve validity
• Validity– Content based on definition of concept– Content can be based on qualitative data
generated by potential participants • E.g., critical incidents for ethnic harassment (EH)
measure (Schneider, Hitlan, & Radhakrishnan 2000) but see Swim et al EH measure
– Not all constructs need participant-generated data
• e.g., answers to an exam
How pre-existing measures improve reliability
• Reliability– If measure is tested on samples similar to your
sample, then you can be confident in the measure
• Schneider et al., 2000
– Can reasonably expect hypothesis to be supported if concepts are reliably measured
Pre-existing measures used in the instructor’s example
• Satisfaction measures – cited in Rode et al., 2005
• Performance Measure– Cited in Rode et al, 2005
• Control Variables– Citizenship replaced by primary language
question which is more appropriate– Not feasible to collect IQ measure in context– Stress measure
• Described in Renner & Mackin, 2000 Instructor slightly modified stem based on previous research (Schneider et al., 2000)
After deciding on measures, structure questionnaire
1. Content of Study information Sheet & Consent form
– See methodology assignment guidelines
2. Logic of ordering
3. Assess criterion variable first in cross-sectional study
4. Attractiveness via Visual Layout– Headings, Font size, White Space
More issues to consider when structuring questionnaire
5. Number of control variables & length of survey
– Shortening pre-existing measures is tempting but might damage reliability and validity.
6. Assessing sensitive variables– E.g., Class demonstration survey; UT survey
7. Ease of data analyses– Numbering sections & items– Number of Open-ended questions
Issues the Instructor faced when designing the examplar
questionnaire• Sensitive Variables
– Dropping additional demographic variable due to sample size
• What if the hypothesis is not supported– Restricted range on the GPA variable– Arguments to use stress as a control
variable vs. an antecedent
While or After designing questionnaire develop
sampling plan• Sampling plan depends whether you
want maximum precision, maximum context or maximum generality– E.g., maximum generality then need
random, large, representative sample
Particular Behavior SystemsUniversal Behavior Systems
ObtrusiveOperations
UnobtrusiveOperations
NaturalSettings
ContrivedSettings
FieldStudies
FieldExperiments
LabExperiments
MaximumContext
MaximumPrecision
MaximumGenerality
Formal Theory
Sample Surveys
SettingIndependent
Behavior notmeasured
ComputerSimulations
Runkel& McGrath, 1972
ExperimentalSimulations
Some terms in the area of sampling
• Population:– Group you are interested in obtaining data from
and studying.
• Sample:– Representative number of respondents from the
population that you sample.
• Actual sample:– The actual number of participants from your
sample that complete and return your survey
Types of Sampling You Can Hope vs. Actually do
• Every person in the population has exactly the same probability of being included in the sample to avoid bias.
• Sample is representative of the larger population.
• Representativeness can be checked by comparing the characteristics of a sample to those of the population – e.g., gender, age, tenure
Random Sampling
One Possible Modification of Random Sampling
• Stratification sampling:– Population divided into groups called strata.– Random selection from within groups.– Ensures representation on some critical factor
in the sample (e.g., gender, job category).
A Second Possible Modificationof Random Sampling
• Cluster sampling:– Participants chosen as members of a group
rather than as individuals.– Randomly select work teams, organizations,
factories, plans, facilities, etc.
Convenience Sampling(AKA what you will end up
doing for this course)
• Selection of participants based on easy availability or accessibility.
• Snowball or chain sampling – people who know people.
How to get a good sample size
• Provide incentives before or after.
• Indicate support from stakeholders.
• Convincing reason to complete it.
• Promise of feedback.
• Reminders.
• Personalize correspondence.
• Return envelope with postage / web-survey
What you learned today• Is your study a field study (or field expt)
or a sample survey?• Will you administer the questionnaire
yourself or collect archival data?• For both data collection methods you
need to use data collected with, or collect data with pre-existing valid & reliable measures– How to find reliable & valid measures– Why use them
• How to design a good questionnaire
• What sampling plan you can hope to use– How to get a large enough sample with the
sampling plan you will use
What you learned today