The Basics of Experimentation IIFinal Considerations

Participants

Type of participants: 4 things you need to consider when choosing who to test

 Precedent: Increases your likelihood of success BUT limits generalizability Availability: many studies use psy 1010 students... Pros and

cons?  Type of research project:

obviously, if you want to see when children develop social stereotypes... you’ll need to use children!

 

Ethics: adhere to the “justice” principle

Number of participants:

Finances: generally, the more people you test, the more it costs 

Time: same idea, unless you can do group testing

 Availability: How big is the subject pool to begin with?

 Within Group Variability:

The more variability you expect to see within groups, the more people you should test. This will decrease within group variability and increase power

Apparatus: to automate or not to automate?

going high-tech isn’t always good (e.g. RAM & rats, tubes in the win-stay task)

 

Always make sure your equipment is in good working order!

Recording the DV

Follow the KISS principle!

Your DV should dictate the technology, not vs.

Anything that is not accurate or consistent is not reliableand if it’s not reliable, it’s not ________

THE EXPERIMENTER AS THE EXTRANEOUS VARIABLE

Experimenter characteristics Physical and psychological attributes of the experimenter (not to mention his/her skill level) can influence the outcome of the experiment.

Single vs Multiple Experimenters Single = analogous to holding the EV constant … does not become a nuisance or confound...

Effect on power, external and internal validity?? 

Experimenter expectancies (Rosenthal Effect)

If not controlled for, an experimenter’s expectancies (which are different for each group) can produce in results consistent with his/her expectancies... Creates a confound e.g. Maze dull vs maze bright rats 

early data returns effect:

caution when analyzing data as it comes in. Can easily set you up for a Rosenthal effect!  

Controlling experimenter effects

Physiological & Psychological Effects Very difficult to control Standardize everything! … procedures, appearance…

Experimenter Expectancies 1. Write down the instructions to participants

2. Scoring participants needs to be as objective as possible. Get someone who is “blind” to score for you

3. Use instrumentation and automation to remove biases

4. Conduct a single blind experiment: experimenter (the one interacting with and scoring participants) does not know who is in which group  

PARTICIPANT PERCEPTIONS AS EXTRANEOUS VARIABLES

Demand characteristics (DC’s) DC’s refer to EVs (other than what the experimenter says and does that is a part of the research design) that participants could use to try and figure out how they should behave or act, or what your hypothesis is.

If participants figure out which group they are in, this creates a confound because there is now a systematic difference between the groups

If participants don’t figure out which group they are in or what your hypothesis is, each one will still try to guess. Different participants will guess differently within each group, creating a nuisance variable

Controlling participant perception effects Demand Characteristics 

Single blind experiments (keep participants “blind”)Double blind experiments (both parties are “blind”)Confound control but not nuisance control 

Give incorrect info to the participants (do not allow them to guess what the experiment is about on their own)Confound and nuisance control 

Problems include ethical issues with using deception and participants will still be exposed to demand characteristics… only the DC’s will be to the incorrect info

Look at the experimental context or situation to make sure no undesirable cues or DCs are present

Response Bias

a phenomenon whereby people’s answers are influenced by something other than what they truly think, feel, or believe. Yea-saying, nay-saying, and response sets can produce a response bias.

1. Yay-saying and nay-saying makes scores invalid

2. Response sets a “pattern of responding” or a “pattern of behavior” which is not representative of how participants really think, feel, or behave.

DC’s present in the context or situation where the experiment takes place could cause participants to respond or behave a certain way.

If using surveys, the way questions are worded, the sequence of the questions... could create a response set. It could suggest to participants how they should answer or just simply “put them in a rut”

Yea-saying, nay-saying & “falling into a rut”

Use the reverse scoring procedure

randomize the two types of questions (if there are many questions)

use counterbalancing (different subjects get the questions in different orders)

make sure a socially desirable response is not implied