INTROSPECTIVE RESEARCH:

VERBAL PROTOCOLS

CHAPTER 3

3.1Introducing

Introspective Research

HOW DO YOU FIND OUT WHAT I’M THINKING? Ask!

How do you define/analyze human emotional/mental

states?Introspective techniques

sources of information about other minds

Asking participants to delve into their own states of consciousness and verbally report on cognitive, affective or social aspects of that

consciousness is the technique used in the introspective studies.

HOW DO YOU USE INTROSPECTIVE TECHNIQUES?

1. Set a task.2. Ask participants to report on

what their brains/hearts are processing as they carry out the task.

Example: solving an arithmetic problem…you mutter to yourself as you solve it….

FÆRCH AND KASPER’S INTROSPECTION IN SECOND LANGUAGE RESEARCH (1987)

Test-taking and validation

Translation Reading Problem-

solving Error analysis Motivation etc

Covers introspective studies in:

LOOKING BACK…

20th centuryWilliam James, Titchener, James

Joyce,Marcel Proust and Virginia Woolf

Within a few years: Under attack by Watson ‘’…untrustworthy for scientific purposes…’’

BehaviorismAbandoned for about 50 years

Ericsson and Simon’s Verbal Protocols (1984,1993)

psychological studies needed to be designed in respect to a model of mental process

formulated such a model based in the basis of information processing theory

Critical feature: assumes a multiple memory model comprising both short-term and long-term memory stores and presumed mechanisms by which they are activated

PRINCIPLES FOR INTROSPECTIVE STUDIES Time intervening between mental operations

and report is critical and should be minimized as much as possible

Verbalization places additional cognitive demands on mental processing that requires care in order to achieve insightful results

Verbal reports of mental processes should avoid the usual social conventions of talking to someone

PRINCIPLES FOR INTROSPECTIVE STUDIES (CONT.) There is a lot of information in

introspective reports aside from the words themselves. parallel signals

Verbal reports of automatic processes are not possible. Such processes include visual and motor processes and low-attention, automatized linguistic processes such as the social chat of native speakers

Research should be based on a model of mental processes that allows predictions about how mental operations will be organized under various conditions

THREE LEVELS OF VERBAL REPORT

Report and the task

Information Example

1Talk-alouds

Report is concurrent with the mental task

linguistically encoded, can be directly stated

Talking aloud while thinking on how to spell a word

2Think-alouds

Report is concurrent with mental task

not already linguistically encoded, thus requires linguistic encoding

Describing what a corkscrew looks like

3 Retrospective

studies

Report is subsequent to mental task

consists of selected foci, descriptions, explanations and interpretations

Reporting a route you travelled

POINTS TO NOTE… Solo/ self-report type

o One is both a participant-subject and analyst

o Example: diary studies, Introspective record language learning impressions

Recordingaudio/video –for later analysisSolo – recordBig class: in pairs, students note down what their partner say – language lab

3.2Experiencing Introspective

Research

GENERAL PRINCIPLES Principle 1- Always use a recording

device.

Principle 2 – Think aloud. Don’t talk.

Principle 3 – Do not be too directive in instructions to participants.

ALL TALK AND NO PLAY MAKES TOO SLEEPY

Sit in pairs Anagram - LIPYMS Participant talk aloud solving the anagram Researcher write down. Reverse roles

BRAINWORK… Were there any problems in solving the

anagram? Were the problems clear from the talk aloud

procedures? Did the two participants use different strategies? Would you use a new strategy if you were to do

it again?

3.3Compiling

Introspective Data

Mostly recorded + notes

Transcribe data from the recording and integrate the notes you have made

Segmenting each segment – a short thought unit, which will be coded and analyzed

ANALYZING INTROSPECTIVE DATA

ANALYSIS OF VERBAL PROTOCOL‘Solving Anagrams’ task types:1. Task Type A : (NPEHPA = HAPPEN)2. Task Type B : (ALPHABET = BET,

BEAT, TAB, PEAL, TABLE)3. Task Type C :

(TEAR, SUN, TRUANT, RESTART = RESTAURANT)

4. Task Type D : (RATS + (?) (word meaning begin) =

RATS + (T) (word meaning begin) = START)

STEPS IN PROTOCOL ANALYSIS1. Responses are transcribed, segmented

and arrayed in the order produced.2. Codes are devised.3. Codes are assigned to response

segments.4. Codes sequences and combinations

are examined for patterns.5. Coding combinations are compared.6. Combinations of codes and coding

classes are examined to see if types of participant strategies can be identified.

CODING LANGUAGE DATA

Coding scheme(Ericsson and Simon, 1987)

Constraint Type (C-type) – Responses are those in which

the participant uses her knowledge of possible letter sequences in English as a

guide for constructing longer sentences.

C-types responses often appear as ‘spelling out’ of

possible letter sequences and letter positions.

Alternative Type (A-type) – Responses are those in which the participant pronounces a

sequence in an attempt to find a sound match in lexical

memory. A-type responses often

appear as ‘sounding out’ of possible syllables or words.

DESIGNING YOUR OWN INTROSPECTIVE RESEARCH STUDY.

1. Free-form responses2. Given orally by the participant3. Elicit cognitive process4. Carry out a specific task5. Take place within particular dimensions

and limited duration6. Produced during or shortly following

out of the task

AREAS OF INTROSPECTIVE STUDY1. Solving logic problems2. Solving a maze3. Crossword puzzle working4. Making word associations5. Completing cloze-type passages6. Ordering scrambled

sentences/paragraph

INTERPRETING INTROSPECTIVE RESEARCH1. Review the objections to introspective

studies.2. Check if the researcher has recognized

the potential problems of such studies.3. See if the researcher has tried to

minimize the impact of these problems.

OBJECTION A -INTROSPECTION ON THINKING PROCESS OF ANY TYPE IS EITHER IMPOSSIBLE OR UNFEASIBLE.

Suggestions:1) Give lots of practice2) Exercise care in choosing introspective tasks

that do not require participants to deal with introspections of higher order, highly automized linguistic processes.

OBJECTION B – EVEN IF INTROSPECTION ON MENTAL STATES IS POSSIBLE OF INTEREST TO LANGUAGE EDUCATORS, INTROSPECTIVE REPORTS ARE BOUND TO BE UNRELIABLE

Issue : RELIABILITY1. Participants’ access to the mental processes

that he/she is trying to articulate.2. Researcher’s instructions, examples or

training is bias. 3. Verbal reporting getting in the way of mental

processing and cause it to be something other than what it would be if there was no requirement to verbalise involved.

SIGNIFICANCE OF INTROSPECTION AS A

SCHOLARLY FOCUS

Plato and Aristotle as early practitionersReasoning, thought and truth

William James (father of modern psychology

Introspective observation is what we have to rely on first and foremost and always. The word introspection need hardly be defined – it means, of course, looking into our own minds and reporting what we there discover’ (Jamesb1890: 185)

Played a major role in psychology in the first two decades of the twentieth century.

Was short-lived, held to be of doubtful validity and overtaken by behaviorism.

‘Verbal protocols are never mentioned as a technique for data collection in books on research methodology’ (Cavalcanti 1987)

U-turn :- since that time, introspective research blossomed (admittedly somewhat biased) ‘verbal reports are now generally recognized

as major sources of data on participants’ cognitive process in specific tasks (Ericsson and Simons 1993)

REFLECTING ON INTROSPECTIVE RESEARCH Wide variety of different types. Any verbal protocol study can be

marked as being of one type or another within each category.

Variety of different uses.

Classification n of verbal protocol data types by Fearch and Cassper. (pg 77)

Payoffs (pg76)

SUMMARY Introspective study:- moved from

enthusiastic endorsement to abandonment to enthusiastic resurrection. (rise/fall/rise)

Remind us that:- choice of research paradigms is a matter of fashion as much as of objective analysis.

It is on the rising tide of acceptance and enthusiasm.

Currents studies:- respond to some earlier critismValidity and reliability

Many caution have been issued and research procedures have been prescribed – minimizing slack.

In sum – What for valid and reliable finding? Design studies based on mental process Incorporate proper warm-up activitiesCareful task instructionAppropriate monitoring of participants