Relativity versus Universals: an obsolete dichotomy

Relativity versus Universals: an obsolete dichotomy

Paul Kay Terry RegierInternational Computer Science Institute, U. of Chicago U. C., Berkeley [email protected] [email protected]

Irvine March 2008

Major point of this talk: The opposition of so-called relativism to so-called universalism impedes our understanding of the complex ways in which language both influences perception and is influenced by perception…

or

Half a Whorf is better than no Whorf at all.

Plan of the talk

• Influence of language on color perception/discrimination

(Kay) Moral: Hedged “relativism”

• Color naming similarities and differences across languages

(Regier) Moral: Hedged “universalism”

Previous work, going back as far as 1984, has demonstrated: •(1) “Categorical Perception” (CP) for color, •(2) that CP occurs at boundaries of linguistically encoded basic color terms, and•(3) that when basic color term boundaries differ between languages, color CP occurs at the linguistic boundaries.•(e.g., Kay & Kempton 1984, Levinson, 1996; Lucy, 1996; Roberson et al., 2000; Winawer et al., 2003. For full references: [email protected].).

This part of the talk is about recent work, showing that:• Color CP is lateralized to the right Visual Field (RVF) and therefore to the Left Cerebral Hemisphere (LH).

• BACKGROUND:

(A) Visual fields project to brain contralaterally;

(B) LH is dominant for language. QuickTime™ and a

TIFF (LZW) decompressorare needed to see this picture.

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

Front

Lexical categories influence perception in the RVF. (a) Print-rendered versions of the four colors used. (b) Sample display for the visual search task. Participants were required to press one of two response keys, indicating the side containing the target color. (c) In the no-interference condition, RTs were faster for the between-category pair and slower for the within-

category pairs when targets appeared in the RVF compared with when they appeared in the LVF. (d) Effects were reversed with verbal interference. *, P < 0.05, two-tailed t test, df = 10; ns, nonsignificant. Values are mean SEM

BERKELEY EXP. 1 Fig. 1. Lexical categories influence perception in the RVF. (a) Print-rendered versions of the four colors used. (b) Sample display for the visual search task. Participants were required to press one of two response keys, indicating the side containing the target color. (c) In the no-interference condition, RTs were faster for the between-category pair and slower for the within-category pairs when targets appeared in the RVF compared with when they appeared in the LVF. (d) Effects were reversed with verbal interference. *, P < 0.05, two-tailed t test, df = 10; ns, nonsignificant. Values are mean ± SEM.(Gilbert, A., Regier, T., Kay, P., & Ivry, R. (2006) PNAS)

BERKELEY EXP. 2 Fig. 2. Modulation of color-category effects in the RVF is specific to linguistic demands of the interference task. (a) Trial events. Within a block of trials, the visual search task was interleaved with blank displays, displays containing a color word, or displays containing a spatial grid. (b and c) No-interference and verbal-interference results replicate those obtained in the first experiment. (d) For the nonverbal-interference condition, performance followed a pattern similar to that observed in the no-interference condition. *, P < 0.05, two-tailed t test, df = 10; ns, nonsignificant.(Gilbert, A., Regier, T., Kay, P., & Ivry, R. (2006) PNAS)

Elimination of lateralized Whorf effect with verbal interference argues that linguistic categories are activated on-line in this task (as against the learning of linguistic categories having warped the perceptual space itself). Whether the effect is strictly speaking perceptual or involves immediate post-perceptual processing can’t be answered from these data.

OK, but:

That’s just one study, in one lab,on one color boundary.

Replicated, at University of Surrey, at blue/green, blue/purple, purple/pink

boundaries. (Also: weaker Whorf in LFV,probably due to trans-callosal transfer.)

(Drivonikou, Kay, Regier, Ivry, Gilbert, Franklin & Davies (2007). PNAS.)

SURREY EXP. 1 Fig. 1. The category effect is larger in the RVF than in the LVF in a reanalysis of the data from a color identification task used by Daoutis et al. (16). (a) Stimuli in CIE coordinates. The within-category set contains three hues of green: G1, G2, G3; the across-category set contains a blue (B), a purple (Pu), and a pink (Pi). Perceptual distance is the same for all adjacent pairs across both sets (G1-G2, G2-G3, B-Pu, and Pu-Pi). When a peripheral stimulus (e.g., G1) is the target, it is linearly separable from the distractors (G2 G3). (b) Illustration of a target-present trial with 15 distractors. The target is indicated here by the arrow that, however, was not present in the display itself. (c and d) Target detection times for within- and across-category targets by LVF and RVF: collapsed across linear separability (c) and linearly separable targets alone (d). Error bars show 95% confidence limits. (G. V. Drivonikou, P. Kay, T. Regier, R. B. Ivry, A. L. Gilbert, A. Franklin, and I. R. L. Davies. PNAS 2007;104;1097-1102)

SURREY EXP. 2 Fig. 2. A larger category effect is observed in the RVF on a color detection task. (a) Munsell codes of the stimuli; stimuli varied in hue at constant value and chroma. Hue separations were either five steps (far set) or 2.5 steps (near set). The target was either in the same color category as the background (e.g., 10BG on 5B, both blue) or in the adjacent category (e.g., 10BG on 5BG, blue on green). (b) Illustration of a test frame: white circles show possible target locations around the fixation cross, and the black circle representing the target. (c and d) Blue-green set: The difference in RT between within- and across-category is larger in the RVF (c). Target-background perceptual separation only affects the RVF (d). (e) Blue-purple set: Again, the difference in RT between within- and across-category is larger in the RVF. Error bars are 95% confidence limits. (G. V. Drivonikou, P. Kay, T. Regier, R. B. Ivry, A. L. Gilbert, A. Franklin, and I. R. L. Davies. PNAS 2007;104;1097-1102)

What about split-brain patients? They should also show CP stronger in RVF.

Fig. 3. Lexical categories influence perception in the RVF of a callostomy patient. *, P < 0.05, two-tailed t-test, df = 1; ns, nonsignificant. (Gilbert, A., Regier, T. Kay, P., &

Ivry, R. PNAS. 2003; 100, 489-494)

Fig. 6. Visual search task results from callosotomy patient testing. Error bars show 95% confidence limits. (Gilbert, A., Regier, T., Kay, P., & Ivry, R. Brain and Language, in press)

Is (near) restriction of CP to RVF limited to color?

Answer: No.

(Gilbert, A., Regier, T., Kay, P., & Ivry, R. Brain and Language, in press)

Dog and cat stimuli for extension of COLOR CPoutside of color domain.

Fig. 2. Sample display for the visual search task [as in BERKELEY EXP. 1] with a between-categories stimulus pair. Participants were required to press one of two response keys, indicating the side containing the target. (Gilbert, A., Regier, T., Kay, P., & Ivry, R. Brain and Language, in press)



(Gilbert, A., Regier, T.,Kay, P., & Ivry, R. Brain and Language (2007), doi:10.1016/j.bandl.2007.06.001)

Standard search task with dog and cat stimuliRVF CP observed, except with verbal interference

What about other languages? Two early reports.

Debi Roberson (U. of Essex) has reported that CP for a color category boundary occurring in Korean but not English is present in the RVF and not the LVF (for fastest-responding participants only).



(Source: Roberson, D. et al., Categorical perception of colour in the left visual field is verbally mediated. Cognition (2007), doi:10.1016/j.cognition.2007.09.001.)

Also, Drivonikou, Davies, Franklin & Taylor, (2007, Lateralisation of colour categorical perception: A cross-cultural study" Perception 36 ECVP Abstract Supplement)report, “Greek has two basic blue terms (ble and galazio), and many African languages have one term that includes blue and green. CP was shown by faster detection of targets on different- than same-category backgrounds. However, lateralisation of CP was only found when category boundaries were marked by the language.”

These two studies provide evidence that what’s at work is linguistically encoded categories. Color CP was, as we’ve seen, previously established to vary with linguistic boundaries; Roberson’s and Drivonikou et al.’s recent studies indicate that this holds for lateralized color CP as well.

Moreover…

VF to Brain Hemisphere inference has been confirmed in an Event-Related Potential (ERP) study using the same stimuli as in BERKELEY EXPS. 1 & 2.

+

+

+

Target200 ms

Cross-category LVF deviant

200 ms +

+

+

o

+

+

Fixation[800-1200 ms]

+

Within-category RVF deviant

200 ms

+

Standard200 ms





Standard200 ms

Standard200 ms

++

+

++

Target200 ms

Cross-category LVF deviant

200 ms +

++

+

oo

+

++


+

Within-category RVF deviant

200 ms

++

Standard200 ms





Standard200 ms

Standard200 ms

EEG experiment protocol. Aubrey Gilbert’s Dissertation (Berkeley, 2007).

ŅIn a paradigm similar to ones used in previous visual mismatch negativity (V-MMN) studies, event-related potentials were recorded in response to standard and deviant color stimuli presented lateral to a centralized task. Deviant stimuli were either from the same (within-category) or from a different (cross-category) lexical category than the standard stimuli. Two deviance-related negativities (DRNs) were observed: an early (150-300 ms), left hemisphere-lateralized component at occipital and extrastriate sites, and a late (400-700 ms) bilateral, frontal component that was larger in the left hemisphere. These DRNs were evoked only when the cross-category deviant was presented in the right visual field. Evoked potentials for all other deviant conditions did not differ significantly from those for the standard stimuli.Ó -- Aubrey Gilbert (Berkeley Dissertation. 2007)

Figure 3. Grand-averaged ERPs to standard (in black) and each of the deviant stimuli. The only deviant to evoke ERPs with significant differences from those evoked by standard stimuli was the cross-category deviant (in red) and these differences only occurred when this deviant was presented in the RVF. The significant differences of note are an earlier (~150-300 ms) increased negativity at occipital and extrastriate sites that is lateralized mostly to the LH, and a later (~400-700 ms) increased negativity at frontal sites that is observed bilaterally.



(Tan, L-H, Chang, A H D, Kay P, Khong, P-L, Yip, L K C, & Luke, K-K. PNAS, 105, 4004-4009 (2008)).

EXPERIMENT RT (mean, approx.) LVF CP? Callosotomy 1 (400) NO (transfer impossible) Callosotomy 2 (700) NO (transfer impossible) BERKELEY 1 425 NO BERKELEY 2 440 NO SURREY 2 540 NO/YES* Roberson (fastest Ss) 740 NO SURREY 1 1375 YES (transfer probable) Roberson (slowest Ss) 1420 YES (transfer probable)

Table 1: Approximate mean response times for the eigh t experiments that show lateralized Whorf effects, with indication of significant LVF CP. Only the two studies with RTs well over 1000 ms show LVF CP, suggesting trans-callosal transfer and scanning as possible causes. *In this experiment one color boundary showed LVF CP and one did not

PATIENTS CONTROLS

Figure 13. Results of 15 aphasic patients and 12 controls tested on the no-interference visual search task of Gilbert, et al. (2006). Source: Paluy, Y ., Gilbert, A.L., Baldo, J.V., & Ivry, R.B. (2007, August). Is Whorf Right? (or Left?). Poster presented at the 29th Annual Cog nitive Science Society. Nashville, TN

SUMMARY SO FAR: 1. Lateralized RVF color CP has been found in a variety of tasks on normal adults. 2. Lateralized RVF color CP has been found in callosotomy patients 3. Lateralized RVF color CP is absent in aphasics with left hemisphere lesions; these patients show LVF color CP. 4. Lateralized RVF color CP has been found at lexical boundaries not present in English for speakers of languages containing those boundaries (Korean and Greek) and not for English speakers. 5. Lateralized RVF CP has been found outside the color domain (dog and cat silhouettes). 6. Lateralized RVF CP in normal adults is consistently suppressed by verbal, but not by non-verbal, interference. 7. Color CP has been found in LVF in some experiments. It is always weaker than RVF CP and the longer response times in these experiments suggest trans-callosal transfer and/or scanning. Either or both of the latter factors could render apparent LVF CP illusory. 8. The inference from RVF CP to left hemisphere activity has been confirmed in an EEG study. 9. The inference from RVF CP to left hemisphere activity has been confirmed in an fMRI study So far everything is consistent with left hemisphere language causing color CP in normal adults.



Figure 14: Prelinguistic infants show color CP lateralized to LVF. Source: (A. Franklin, G.V. Drivonikou, L. Bevis, I.R.L. Davies, P. Kay, & T. Regier. PNAS 105, 3221–3225.(2008).)

Assuming:

1. LVF CP in normals is illusory, reflecting trans-callosal transferand/or scanning,2. LVF CP in aphasics with LH lesions reflects transfer of function,

Then the apparent conflict between RVF CP in normal adults and LVF CP in infants can be explained as language acquisition in the LH taking over the categorization function from the RH as the child matures.

Take home lesson:

Normal adults, under normal viewing conditions, get two pictures of the word simultaneously: one filtered through linguistic categories and one not so filtered.



“Me worry?”

Documents

Relativity versus Universals: an obsolete dichotomy