Problems in ResearchProblems in Research

►Poor design decisionsPoor design decisions►Researcher biasResearcher bias►Researcher effects on participantsResearcher effects on participants►Things participants may do regardlessThings participants may do regardless

psyc

hlot

ron.

org.

uk

Poor DesignPoor Design

►Problems that stem from a badly Problems that stem from a badly designed studydesigned study Poor operationalisation of variablesPoor operationalisation of variables Inadequate control of variablesInadequate control of variables

►Threaten the internal validity of the Threaten the internal validity of the studystudy

►Cause reliability problemsCause reliability problems

psyc

hlot

ron.

org.

uk

Poor DesignPoor Design

►Poor operationalisationPoor operationalisation Insufficient care when translating Insufficient care when translating

unobservable psychological processes unobservable psychological processes into observable behavioural onesinto observable behavioural ones

►Can end up measuring the wrong thingCan end up measuring the wrong thing►Can generate false positives or false Can generate false positives or false

negativesnegatives

psyc

hlot

ron.

org.

uk

Poor DesignPoor Design

►Reliability – the extent to which a study Reliability – the extent to which a study or measuring tool produces consistent or measuring tool produces consistent resultsresults Test-retest reliabilityTest-retest reliability Inter-observer reliabilityInter-observer reliability

►Reliability is maximised by (1) careful Reliability is maximised by (1) careful operationalisation; (2) adequate control operationalisation; (2) adequate control of variablesof variables

►NB: a reliable study can still be invalidNB: a reliable study can still be invalid

psyc

hlot

ron.

org.

uk

Poor DesignPoor Design

► Inadequate controlsInadequate controls Extraneous variables should be held Extraneous variables should be held

constantconstant Most are not a problem (random effect)Most are not a problem (random effect)

►Those that have a systematic effect Those that have a systematic effect (e.g. affect one condition more than (e.g. affect one condition more than another) become confounding another) become confounding variablesvariables

psyc

hlot

ron.

org.

uk

Researcher BiasResearcher Bias

►Emphasising some things at the Emphasising some things at the expense of othersexpense of others Self-confirmatory bias – tendency for Self-confirmatory bias – tendency for

researchers to pay more attention to data researchers to pay more attention to data that confirm their expectationsthat confirm their expectations

►Reduce the need for subjective Reduce the need for subjective judgements by (1) careful judgements by (1) careful operationalisation; (2) use of double operationalisation; (2) use of double blindblind

psyc

hlot

ron.

org.

uk

Researchers & ParticipantsResearchers & Participants

►Researcher effects - behaviour of Researcher effects - behaviour of researcher can affect that of researcher can affect that of participants:participants: Intimidate (anxiety)Intimidate (anxiety) Insult (motivation)Insult (motivation) Cue PPs to behave in particular waysCue PPs to behave in particular ways

►Can be reduced through Can be reduced through standardisation of scripts and standardisation of scripts and proceduresprocedures

psyc

hlot

ron.

org.

uk

Researchers & ParticipantsResearchers & Participants

► Interviewer effects – characteristics of Interviewer effects – characteristics of the researcher can affect how PP the researcher can affect how PP relates to them; hence their responsesrelates to them; hence their responses GenderGender Social classSocial class AgeAge Ethnicity…Ethnicity…

►Difficult to predict, impossible to Difficult to predict, impossible to eliminateeliminate

psyc

hlot

ron.

org.

uk

Participants…Participants…

►……are people too; people try to make are people too; people try to make sense of their situation…how they do sense of their situation…how they do this is not always predictable.this is not always predictable. Attempts to please (or displease) the Attempts to please (or displease) the

researcherresearcher Management of self-presentationManagement of self-presentation

psyc

hlot

ron.

org.

uk

ParticipantsParticipants

►Respond to the fact that they are Respond to the fact that they are being studiedbeing studied Evaluation anxietyEvaluation anxiety Feelings about the study & the researcherFeelings about the study & the researcher

►Effects on motivation – please the Effects on motivation – please the researcher or confound the results; researcher or confound the results; depends on how researcher is depends on how researcher is perceivedperceived

psyc

hlot

ron.

org.

uk

ParticipantsParticipants

►May respond to demand characteristics May respond to demand characteristics (cues); these can enter the research:(cues); these can enter the research: Through poor design e.g. allowing PPs to be Through poor design e.g. allowing PPs to be

aware of both conditions in an exptaware of both conditions in an expt Through researcher behaviour e.g. Through researcher behaviour e.g.

unwittingly indicating desired behaviourunwittingly indicating desired behaviour

►Can be reduced through careful choice of Can be reduced through careful choice of design; standardisation; double blinddesign; standardisation; double blind

psyc

hlot

ron.

org.

uk

ParticipantsParticipants

►Generally try to manage how they are Generally try to manage how they are perceived by others (we all do); can perceived by others (we all do); can lead to:lead to: Socially desirable responses Socially desirable responses Other distortions of truth in order to Other distortions of truth in order to

manage impressions (how depends on manage impressions (how depends on who and why)who and why)

►Can be reduced through anonymity, Can be reduced through anonymity, confidentiality but rarely eliminatedconfidentiality but rarely eliminated

psyc

hlot

ron.

org.

uk

General PrecautionsGeneral Precautions

►Start with the aim, not the procedureStart with the aim, not the procedure►Pay attention to choice of design, Pay attention to choice of design,

controls, operationalisation etc.controls, operationalisation etc.►Pilot all procedures and materialsPilot all procedures and materials►Use double blind procedures where Use double blind procedures where

appropriate & possibleappropriate & possible

psyc

hlot

ron.

org.

uk