Barry HughesDepartment of Psychology

Research Centre for Cognitive NeuroscienceUniversity of Auckland

Terri Hedgpeth Disability Resources Center

Arizona State University

What finger movements reveal about cognitive processing during Braille reading

General hypotheses

Finger movements during Braille reading are useful as a window onto the otherwise invisible processes that underlie Braille reading.

Analyses of finger movements will contribute to • the development of formal models of perceptual-motor coordination during reading• linking functional cognitive models to distribution patterns of neural activity• insights into the learning and teaching of braille

Print reading Braille reading

Perceptual modality

Visual Haptic

Movement types Saccades and fixations

Smooth (?) continuous (?) exhaustive (?) contact

Attention allocation

Processing characteristics

Serial

Parallel

Serial?

Serial?

Movement coordination

Conjugate eye rotation

One-handed and two- handed?

Information uptake Only during fixations Only with contact and movement?

Finger velocity by time

Finger position by time

Potential factors that could influence the left-right velocity of reading finger(s)

Haptic (skin-surface) interactions

Motor control factors

Linguistic processing demands

General Methods

Participants (N = 23)

Age ranged from 22 y to 75 y; mean: 45.8 (+ 17.6) y. Mean experience as fluent readers : 35.5 (+ 18.1) y.

Under normal circumstances, six participants used a single finger to read; 16 read with the index finger of each hand and one read with

the index and middle fingers of each hand.

For 19 readers the right index finger was designated dominant; for four readers the left index was designated dominant.

Movement Recording and Analysis

The grip pen for a high resolution digitizing tablet (Wacom Intuos)

was fitted to a light-weight finger attachment on the reader’s

dominant reading finger.

Study 1: Scanning vs reading Braille

Experiment in two phases, presented in a counterbalanced order

1. A phase in which readers were required to read but merely to move the finger across as (“scan as smoothly as possible”) meaningless strings of Braille cells.

2. A phase in which readers were required to accurately read sentences (either aloud or to repeat verbatim).

All sentences, whether scanned or read, were within a single line of text.All sentences to be read were rendered in Grade 2 (contracted) Braille.

In all cases the sentences took a template form:

article [adjective] noun verb article adjective noun.

Sentences were created by crossing two factors:

Word [noun, verb, adjective] frequency (high or low)Sentence meaning (‘meaning’ or ‘nonsense’)

Four sentences of each type were read twice in a random order.

1. high frequency words, meaningA father [213] imagines [71] his quiet [74] year [678].,A "F imagines 8 quiet year4

2. low frequency words, meaning 3. high frequency words, nonsense

4. low frequency words, nonsense A blurb [1] calculates [1] the bottled [2] walnut [3]. ,A blurb calculates ! bottl$ walnut4

Control Scanning

0

10

20

30

40

0

10

20

30

40

Left-

Rig

ht V

eloc

ity (

cm/s

)

0

10

20

30

40

Left-Right Position (cm)

0 5 10 15 20 25 30

0

10

20

30

40

0 raised dots per cell

1 raised dot per cell

3 raised dots per cell

5 raised dots per cell

Continuous cell repetition

yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy

lllllllllllllllllllllllllllllllllllllllllllllllllll

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

················································

0

10

20

30

40

Left-

Rig

ht V

eloc

ity (

cm/s

)0

10

20

30

40

0 raised dots per cell

0

10

20

30

40

Left-Right Position (cm)

0 5 10 15 20 25 30

0

10

20

30

40

1 raised dot per cell

3 raised dots per cell

5 raised dots per cell

Noncontinuous cell repetition

aaaaa aaaaa aaaaaa aaaaa aaaa aaaaa aaaaa

llll lllll llllll lllll lllll lllll llllll

yyyy yyyyy yyyyyy yyyyy yyyy yyyyyy yyyyyy

0

10

20

30

40 Pseudo-sentence

0

10

20

30

40

2D Graph 1

Lef

t-R

ight

Vel

ocity

(cm

/s)

0

10

20

30

40

2D Graph 2

Left-Right Position (cm)

0 5 10 15 20 25 30

0

10

20

30

40

'A Wwwwww Tttttt aaaaaa F ccccccc

'A Tttttt ((((( h wwwww Nnnnnnn

'A eeee J ttttt n mmmmmm

'A bbbbb cccccccccc y bbbbbb wwwwwwww

Left-

Rig

ht V

eloc

ity (

cm/s

)

-20

-10

0

10

20

-20

-10

0

10

20

-20

-10

0

10

20

Left-Right Position (cm)

0 5 10 15 20 25-20

-10

0

10

20

,A weekly tr"t;t all{s = 3trol4

,! Kmusical s*Eme expla9$ h} answ}4

,! job kept an obvi|s purpose4

,A "F imagines 8 quiet year4

HIGH FREQUENCY MEANING

Reading

-20

-10

0

10

20

Left-

Rig

ht V

eloc

ity (

cm/s

)

-20

-10

0

10

20

-20

-10

0

10

20

Left-Right Position (cm)

0 5 10 15 20 25-20

-10

0

10

20

LOW FREQUENCY NONSENSE

,A blurb calculates ! bottl$ walnut4

,! dr"my valve labels a 4missal4

,A .qua9t ulc} p5n$ ! atroc;y4

,! hangov} hails a <o?/ly lunacy4

Mea

n N

o.

Zer

o-C

ross

ing

s

020406080

100120140160180

Mea

n N

o.

Rev

ersa

ls

0

5

10

15

20

25

30

Silent Reading Oral Reading

HM HN LM LNR1 R2 R1 R2 R1 R2 R1 R2

Mea

n V

elo

city

(cm

/s)

0

1

2

3

4Silent Reading Oral Reading

HM HN LM LNR1 R2 R1 R2 R1 R2 R1 R2

Silent Reading Oral Reading

HM HN LM LNR1 R2 R1 R2 R1 R2 R1 R2

B

C

A

HM HN LM LNR1 R2 R1 R2 R1 R2 R1 R2

HM HN LM LNR1 R2 R1 R2 R1 R2 R1 R2

HM HN LM LNR1 R2 R1 R2 R1 R2 R1 R2

For all dependent variables, ANOVA revealed effects of word frequency (H v. L)re-reading (R1 v. R2)and their interaction

reading mode (silent v. oral)sentence meaning (M v. N)had no significant effect on any dependent variable.

Mean Reading Velocity (cm/s)0 2 4 6 8 10M

ean

No

. Acc

ele

ratio

n Z

ero-

Cro

ssin

gs

0

100

200

300

400

500

Low Frequency words, First reading

0 2 4 6 8 10

Low Frequency words, Second reading

Mean Reading Velocity (cm/s)

0 2 4 6 8 100

100

200

300

400

500

Mea

n N

o. A

ccel

erat

ion

Zer

o-C

ross

ing

s

Mean Reading Velocity (cm/s)0 2 4 6 8 10

Mean Reading Velocity (cm/s)

High Frequency words, First reading High Frequency words, Second reading

A B

C D

Mean Scanning Velocity (cm/s)0 5 10 15 20

0

100

200

300

400

500

Mea

n N

o. A

ccel

erat

ion

Zer

o-C

ross

ing

s

E

There appears to be a strong exponential and inverse relationship between the mean velocity with which a sentence is read and the number of inflections in the velocity trace.

This strengthens the claim that the smoothness of the velocity trace is largely determined by the finger’s mean velocity, rather than by language processing directly.

Study 2

If word frequency has an effect on mean velocity and smoothness and reversals, is this because

(a) the words are more frequently occurring? or (b) the orthographic features of the words are more familiar?

Participants read sentences that had similar openings but concluded with key words, including an adjective and a noun.

These key words involved combinations of word frequency (high or low), and orthogonal familiarity (high or low)*

* Statistics from E-lexicon Project http://elexicon.wustl.edu/ See also Balota et al (2007). Behavior Research Methods, 39 (3), 445-459.

Words High-Orthography High HH ,%e sad %e _h a r1sona# morn+4She said she had a reasonable morning.

Words High-Orthography Low HL ,%e sad %e _h unusual ?"\s4She said she had unusual thoughts.

Words Low-Orthography High LH,%e sad %e _h mangl$ 3te/s4She said she had mangled contests.

Words Low-Orthography Low LL ,%e sad %e _h newf.d idiosyncrasies4She said she had newfound idiosyncrasies.

Left-Right position (cm)

0 2 4 6 8 10 12 14 16 18 20 22

Fin

ger

Vel

ocity

(cm

/s)

-20

-10

0

10

20

30 HH

0 2 4 6 8 10 12 14 16 18 20 22

-20

-10

0

10

20

30 HL

0 2 4 6 8 10 12 14 16 18 20 22

-20

-10

0

10

20

30 LH

0 2 4 6 8 10 12 14 16 18 20 22

-20

-10

0

10

20

30LL

Left-Right position (cm)

0 2 4 6 8 10 12 14 16 18 20 22

Fin

ger

Vel

ocity

(cm

/s)

-20

-10

0

10

20

30HH

0 2 4 6 8 10 12 14 16 18 20 22

-20

-10

0

10

20

30HL

0 2 4 6 8 10 12 14 16 18 20 22

-20

-10

0

10

20

30 LH

0 2 4 6 8 10 12 14 16 18 20 22

-20

-10

0

10

20

30 LL

Words High-Orthography High HH,Ty 3sid}$ x a will# 3nec;n4They considered it a willing connection.

Words High-Orthography Low HL,Ty 3sid}$ x a typical faculty4They considered it a typical faculty.

Words Low-Orthography High LH,Ty 3sid}$ x a 3d5s$ repres.n4They considered it a condensed repression.

Words Low-Orthography Low LL,Ty 3sid}$ x a noxi\s typhoid4They considered it a noxious typhoid.

As expected, there was little difference in kinematic variables during opening segments

In the key word segments, we find

• an increase in velocity (especially for high frequency words);• an increase in acceleration zero-crossings (especially for low frequency words);• an increase in reversals (especially for low frequency words)but • no effects of orthographic familiarity

Study 3: On reversals

Reversals are a distinct category of movement intermittencies in Braille reading.

What causes reversals in Braille reading?

are they regressions to recover from explicit errors of comprehension?

are they movements that serve to synchronise the finger position with the linguistic processing (the ‘finger-mind span’)?

are they the second, return part of fast look ahead movements?

We asked readers to silently read sentences to completion, at which point a symbol would indicate which one of the sentence’s two nouns should be reversed to as fast and as accurately as possible.

The sentences came in three forms:

normal: grammatical and coherent The sickly maid took the pill.The motorists smashed the glowing headlights.

reversed: grammatical but not semantically coherent The tasty eggplants consumed the goat. The aroma smelt the sneaky predators.

scrambled: neither grammatical nor coherent Frail held snapshots the the historian. Costly teens the the admired yacht.

The target noun could be:located in the noun phrase or the verb phrase of a normal

sentence)short or long in length (4 cells or 8 cells in Grade 2 Braille)

Normal sentences:grammatical and coherent

Reversed sentences:grammatical but not coherent

Scrambled sentences:Not grammatical, not coherent

Summary

Kinematic analyses reveal important details of the reading finger

• that “smooth” Braille reading is almost always an illusion;

• that processing language more efficiently results in an increase in velocity and only indirectly in smoother movements;

• that some factors (such as letter combination frequency, mode of reading and sentence meaning) have no measurable effect on reading velocities;

• that Braille readers reverse movement direction more often than print readers –and perhaps for different reasons;

• that reversals are fast but not ballistic movements, with contact maintained with the reading surface.

Research has been supported by grants from

Bilateral Research Assistance Programme of the Royal Society of New Zealand

Faculty of Science Research Development Fund

Collaborators

Arend Van Gemmert, Louisiana State University George Stelmach, Arizona State University

Hans-Leo Teulings, Neuroscript LLC Ashwin Mathur, University of Auckland Phillipa Turner, University of Auckland Mayuri Patel, University of Auckland