© 2009 by The McGraw-Hill Companies, Inc. Research Methods in Psychology Observation

© 2009 by The McGraw-Hill Companies, Inc.

Research Methods in Psychology

Observation


Observational Research

Researchers cannot observe• all of a person’s behavior• all people’s behavior

Researchers can observe• samples of individuals• samples of behavior at particular times• samples of different settings and conditions



Goal of samples• represent larger population of

behaviors people settings and conditions



Example:• In a typical week, how many hours of

television do you watch? What is the average number of hours for the

class?

• Is this average representative of the number of hours of TV watched by all students on campus? all college students? all people?



Use data from a sample to represent the population• “generalize” the findings from sample to

population

External validity• extent to which a study’s findings may be

used to describe people, settings, conditions

• beyond those used in the study



Generalize findings• sample must be representative of population

is sample similar to population? do we know characteristics of entire population?

Psychology studies with college student samples• are psychology students representative of

larger population?


Sampling Behavior

Extent to which observations may be generalized (external validity)• depends on how behavior is sampled

Two methods• time sampling• situation sampling

Goal: obtain representative sample of behavior


Sampling Behavior, continued

Time Sampling• choose time intervals for making observations

systematic random

• don’t use time sampling for observing behavior during rare events (e.g., hurricane) event sampling


Sampling Behavior, continued

Situation Sampling• choose different settings, circumstances,

conditions for observations• enhances external validity• use subject sampling to observe some people

within a situation


Classification of Observational Methods

Categories based on intervention by researcher• Observation without Intervention• Observation with Intervention

Categories based on methods for recording behavior• comprehensive record• selected behaviors


Observation without Intervention

Naturalistic Observation• observation in natural (real-world) setting without

attempt to intervene or change situation• use when ethical considerations prevent experimental

manipulation

Goals• describe “normal” behavior, examine relationships

among naturally occurring variables• establish external validity of lab findings


Observation with Intervention

Most psychological research involves intervention

Three methods in natural settings• participant observation• structured observation• field experiment


Observation with Intervention, continued

Participant observation• observer is active participant in the natural

setting he or she observes undisguised: people know they’re being observed disguised: people don’t know they’re being

observed



Problems with participant observation• Reactivity

when people change their usual behavior because they’re being observed

disguised participant observation controls reactivity

• Observers lose objectivity or become too involved in situation

• Observers influence behavior of people they’re observing



Structured observation• set up (structure) specific situation in order to

observe behavior• used when behavior is difficult to observe as it

naturally occurs• researchers use confederates to structure

situations• problems: when observers don’t follow same

procedures across observations



Example of structured observation• Simons and Levin (1998): “change blindness”• Web site:

http://viscog.beckman.uiuc.edu/djs_lab/demos.html go to “A subject in a real-world person change event”

http://viscog.beckman.uiuc.edu/djs_lab/demos.html

http://viscog.beckman.uiuc.edu/djs_lab/demos.html



Field Experiment• manipulate independent variable in natural

setting and observe behavior (dependent variable) two or more conditions to compare (IV) often use confederates to create conditions strive for control in natural setting


Recording Behavior

Comprehensive record• video, audio recordings

Select specific behaviors• checklists, ratings

Method for recording behavior determines how results are• measured, summarized, analyzed, reported


Recording Behavior, continued

Qualitative Records• Narrative records: complete reproduction of

behavior (video, audio, field notes)• made during or soon after behavior occurs• carefully train observers• advantage: can review record often• disadvantage: costly, time-consuming


Recording Behavior, continued

Quantitative Records• Selected behaviors• Requires decision regarding how to measure

behavior (e.g., frequency, duration)• checklists, electronic recording and tracking

Measurement Scales• Four levels for quantifying behavior

nominal, ordinal, interval, ratio


Measurement Scales

Nominal• classify behaviors, events, and characteristics

into categories• checklist examples:

sex (male/female) volunteer to help (yes/no) grade (pass/fail)

• lowest level of measurement


Measurement Scales, continued

Ordinal• rank-order behaviors and events• 1st, 2nd, 3rd, etc.• example: letter grades on a test (A, B, C, …)

Question: What is the average grade? can’t compute averages using ordinal

measurement



Interval• scale allows researcher to specify distance between

observations on a given dimension• distance between points on scale is equal• example: scores on a test• big advantage: compute means and standard

deviations allows easy summary of groups

• question: does someone who scored 90 on a test know twice as much as someone who scored 45?



Ratio• same as interval, but score of zero is

meaningful• example: time

someone who finished test in 50 min took twice as long as someone who finished in 25 min

• Most psychological measurement: no meaningful score of “0” we don’t recognize concepts such as “zero

memory” or “zero intelligence”



Most psychological measurement• interval scales are assumed• example: rating scale of aggression

1---2---3---4---5---6---7---8---9---10not aggressive very aggressive

• rating scales are treated as interval scales but are more accurately described as ordinal

• Is psychological distance between different points on the scale equal?


Analysis of Observational Data

Method for analysis depends on• how data are recorded• measurement scale

Two types of analysis• qualitative• quantitative


Analysis of Observational Data, continued

Qualitative Analysis• comprehensive, narrative records• three steps

code data from narrative record display the data draw and verify conclusions



Example of qualitative data analysis:teacher effectiveness based on videos of teachers and students

(1) Develop coding scheme

classify various teacher and student behaviors

teacher: use of questions, examples, humor

student: ask questions, boredom, note-taking



(2) Display data.

example: Develop a sequence of effectiveness during classroom period in which effective teachers

begin with anecdote

then present an example

then ask a question

then insert humor, etc.

Display this in diagram of nodes from one event to next



(3) Draw and verify conclusions.

example: develop a theory that an effective teacher uses several different types of engagement throughout a class period



Problems with Qualitative Analysis• How to determine if theory is correct?• Validity (truthfulness) of qualitative analyses

often is questioned The data (evidence) to develop theory often are

the same data used to support the theory• Qualitative analyses can be circular

Researchers’ biases can influence which data they examine to support their theory



Quantitative Analysis • selected behaviors• method of data analysis depends on

measurement scale used to record behavior



Nominal• summarize behavior using relative frequency

example: how many people in each category or percentage of people in a category

Interval or Ratio• summarize behavior using central tendency

(mean) and dispersion (standard deviation)



Reliability• refers to consistency• Do two or more observers agree in their

observations? (are they consistent with each other?)

• interobserver reliability example: suppose 2 observers rated “teacher

effectiveness” on a 1–5 scale (not effective–very effective)



• Suppose Observer 1 rated Instructor Z with a “1” Observer 2 rated Instructor Z with a “4”

• Question: Is Instructor Z effective?• Can’t tell because of low interobserver

reliability



Factors that affect interobserver reliability• characteristics of the observers

bored, tired, amount of experience train observers and provide feedback

• clearly define events and behaviors to be observed provide examples



How to assess interobserver reliability• Nominal scale: percentage agreement

number of times 2 observers agree

number of opportunities to agree X 100



• interobserver reliability with nominal scale• example: Is the teacher effective (yes/no)

Observer 1 Observer 2

teacher 1 yes yes agree

teacher 2 yes no disagree

teacher 3 no yes disagree

teacher 4 no no agree

Percent agreement: # of times agree = 2 = .50 X 100 = 50%

# of opportunities 4

50% is low interobserver reliability (< 85%)



• interobserver reliability with interval and ratio scales• correlation coefficient

Obs 1 Obs 2 Obs 1 Obs 2teacher 1 4 3 teacher 11 4 5teacher 2 5 5 teacher 12 4 4teacher 3 2 1 teacher 13 3 3teacher 4 3 3 teacher 14 4 3teacher 5 3 3 teacher 15 5 5teacher 6 4 4 teacher 16 4 4teacher 7 2 2 teacher 17 2 3teacher 8 1 2 teacher 18 2 2teacher 9 1 1 teacher 19 3 2teacher 10 3 3 teacher 20 5 5



correlation between ratings for Observer 1 and Observer 2 = .88

acceptable reliability is correlation greater than .85


Thinking Critically About Observational Research

Problems in observational research• influence of the observer on behavior• observer bias


Thinking Critically About Observational Research, continued

Influence of the Observer• Reactivity: people change their usual behavior

when they know they’re being observed• Researchers want to observe people’s usual

behavior• Demand characteristics: people pay attention

to cues and information in the situation to guide their behavior



Controlling reactivity• conceal observer (ethics: privacy issues)• disguised participant observation (privacy)• use indirect (unobtrusive) observation• Adapt participants to observer

habituation desensitization

Reactivity is a potential problem in all psychological research



Observer bias• Observers often have expectations about

behavior example: expectations based on research

hypotheses

• Expectations can lead observers to look at only particular behaviors

• Observer bias: systematic errors in observation that result from expectations also called expectancy effects



Observer bias• potential problem in all research• hard to eliminate• observers must always be aware that they

may be biased• reduce bias by keeping observers “blind” to

aspects of the study: reasons for observations goals of the study hypotheses

Documents

© 2009 by The McGraw-Hill Companies, Inc. Research Methods in Psychology Observation