Embed Size (px)
Citation preview
© 2005 The McGraw-Hill Companies, Inc., All Rights Reserved.
Chapter 7
Using Nonexperimental Research
© 2005 The McGraw-Hill Companies, Inc., All Rights Reserved.
Developing Behavioral Categories A behavioral category includes the general and
specific classes of behavior to be observed Categories must be operationally defined Developing behavioral categories may be easy or
challenging Behavioral categories must be clearly defined to
avoid confusion Begin with clear goals for research Clearly define all hypotheses Keep categories as simple as possible Avoid temptation to accomplish too much in one
study
© 2005 The McGraw-Hill Companies, Inc., All Rights Reserved.
Quantifying Behavior in Observational Research
Frequency Method Record the frequency with which a behavior
occurs within a time period Duration Method
Record how long a behavior lasts Intervals Method
Divide the observation period into several discrete time intervals (e.g., ten 2-minute intervals), and record whether a behavior occurs within each interval
© 2005 The McGraw-Hill Companies, Inc., All Rights Reserved.
Coping With Complexity in Observational Research
Time Sampling Scan subjects for a specific period (e.g., 30
seconds), and then record your observations during the next period
Individual Sampling Select a subject and observe behavior for a
given period (e.g., 30 seconds), and then shift to another subject and repeat observations
© 2005 The McGraw-Hill Companies, Inc., All Rights Reserved.
Event Sampling Select one behavior for observation and
record all instances of that behavior It is best if one behavior can be specified as
more important than others Recording
Use a recording device to make a record of behavior for later review
© 2005 The McGraw-Hill Companies, Inc., All Rights Reserved.
Evaluating Interrater Reliability
You must establish reliability of observations from multiple observers (interrater reliability)
Methods for evaluating interrater reliability Percent agreement
Simplest method Percent agreement should be around 70% Percent agreement may underestimate agreement
Cohen’s Kappa Popular method Allows you to determine if agreement observed is due
to chance
© 2005 The McGraw-Hill Companies, Inc., All Rights Reserved.
Pearson Product-Moment Correlation Correlate ratings of multiple observers with Pearson r Simple and easy method to evaluate interrater
reliability Two sets of scores may correlate highly, but may still
differ markedly Intraclass Correlation (ICC)
Extension of Analysis of Variance logic to interrater reliability
A powerful and flexible tool for evaluating interrater reliability
© 2005 The McGraw-Hill Companies, Inc., All Rights Reserved.
Interrater Reliability: Using Cohen’s Kappa
Tabulate frequencies of interrater agreement and disagreement in a CONFUSION MATRIX
Determine the proportion of actual agreement by summing the values along the diagonal of the confusion matrix and dividing by the total number of observations
Find the proportion of expected agreement by multiplying corresponding row and column totals and dividing by the number of observations squared
Enter resulting numbers in the formula for Cohen’s Kappa
A Cohen’s Kappa of .70 or more indicates acceptable interrater reliability
© 2005 The McGraw-Hill Companies, Inc., All Rights Reserved.
Nonexperimental Approaches to Data Collection
Naturalistic Observation Unobtrusive observations of subjects’ naturally
occurring behavior are made Ethnography
The researcher becomes immersed in the behavioral or social system being studied. May be conducted as a participant or non-participant observation study
Sociometry You identify and measure interpersonal
relationships within a group
© 2005 The McGraw-Hill Companies, Inc., All Rights Reserved.
Case History You observe and report on a single case
Archival Research You use existing records (e.g., police records)
as your source of data Content Analysis
You analyze spoken or written records for the occurrence of specific categories of events (e.g., a word or phrase)
Both RECORDING and CONTEXT UNITS are evaluated
© 2005 The McGraw-Hill Companies, Inc., All Rights Reserved.
Issues to Be Considered in Ethnography
Observing as a participant or non-participant
Gaining access to a field setting Gaining entry into the group Becoming invisible Making observations and recording data Analyzing ethnographic data
© 2005 The McGraw-Hill Companies, Inc., All Rights Reserved.
Content Analysis: Defining Characteristics
Used to analyze a written or spoken record for occurrence of specific behaviors or events
Archival sources often used as sources for data
Appears simple, but may be complex Should be used within a clearly developed
study, including hypotheses to be tested Response categories must be clearly defined A method for quantifying behavior must be
defined
© 2005 The McGraw-Hill Companies, Inc., All Rights Reserved.
Performing a Content Analysis
Clearly defined response categories are essential Two units of analysis
Recording unit: Element of the material you are going to record (e.g., instances of a certain word)
Context unit: Context within which material analyzed appears
Observers doing content analysis must be blind so that bias will not enter the analysis
Materials to be analyzed should be chosen carefully to increase generality
Cannot be used to establish causal connections among variables
© 2005 The McGraw-Hill Companies, Inc., All Rights Reserved.
Factors to Include When Meta-Analyzing Literature
Full reference citation Names and addresses of authors Sex of experimenter Sex of subjects used in each experiment Characteristics of subject sample (e.g., how
obtained, number) Task required of subjects and other details
about the dependent variable
© 2005 The McGraw-Hill Companies, Inc., All Rights Reserved.
Design of the study (including any unusual features)
Control groups and procedures included to reduce confoundings
Results from statistical tests that bear directly on the issue being considered in the meta-analysis (effect sizes, values of inferential statistics, p values)
