Context Notes Fixed

7/26/2019 Context Notes Fixed

1/34

Context

On pages 8 and 9 of Basic Vision, Snowden et al introduce the concept of visual illusions with these two figures. Generallyspeaking, the appearance of any visual stimulus, e.g. the shapes of the parallelograms on the left or the brightness of theparallelograms on the right (and note that those are parallelograms, not squares!), depends upon what other stimuli appearnearby.

Generally speaking, these illusions can be considered e!ects of visual context. In these particular examples, the contexts

serve to suggest specific ways to interpret each visual stimulus (i.e. as part of a table, or chessboard). But context can a!ectappearance even when it isnt obviously consistent or inconsistent with any high-level interpretations.


2/34

Simultaneous contrast

illusions

This figure comes from one of my favourite papers on the e!ects of visual context, by Jenny Bosten and John Mollon (2010).It illustrates the way in which they measured 10 illusions of simultaneous contrast.Note that the word contrast in this context has a quite general meaning. That is, it doesnt merely denote modulations ofluminance; rather, it denotes modulations of any arbitrary stimulus quantity.At the upper left, you see a grating with relatively high spatial frequency surrounded by a grating of relatively low spatialfrequency. If you took the surrounding grating away, the frequency of the central grating would appear to decrease.To the right of that, Bosten & Mollon have illustrated an upwards-drifting central grating surrounded by a downwards driftinggrating. Remove the latter and the central grating will appear to slow down. As I mentioned, these are both illusions of

simultaneous contrast. In general, all sensory systems (not just the visual system) tend to exaggerate the di!erences betweenadjacent stimuli. Thus, a grey centre will appear even less yellow (that is, it will appear more blue) when surrounded byyellow, a low-contrast red and green pattern will appear to have even less chromatic contrast when surrounded by a highlysaturated red and green pattern, etc.The other illusions described in this slide are illusions of numerosity, luminance contrast, yellow-blue chromatic contrast, tilt,luminance, and chromaticity on the red-green dimension.Ill demonstrate some of these illusions later in this lecture.


3/34

This is a demonstration of simultaneous luminance contrast.The two grey discs are identical, but the one on the left appears darker than the one on the right.This illusion is usually ascribed to gain control.


4/34

Compressive transduction is one kind of gain control. Neurones are incapable of sustaining firing rates of more than 300spikes per second for more than a very short time. In other words, neurones have a built-in maximum response. The dynamicrange of a neurone refers to the inputs over which the neural response is greater than its minimum and smaller than itsmaximum. Anything that increases a neurones dynamic range qualifies as a type of gain control. Expansive transduction,which is what is happening in this blue region, gives a relatively large change in response for a small change in input. That is,it reduces dynamic range. Compressive transduction is its opposite, and that is what is happening in the green region. Youneed a large change in input to produce the same change in output. Thus, this type of non-linear transduction producessome gain control, but not enough, and not the kind responsible for the exaggeration of simultaneous contrasts.


5/34

y

xl

x

The kind of Gain control said to be responsible for the visual systems exaggeration of simultaneous contrast is lateralinhibition.

I know youve already been introduced to the concept of lateral inhibition in the retina. Nonetheless, Im going to go over itagain, in a way that should facilitate how it works throughout the visual system.

The dashed line represents a slice through the stimulus you saw a few slides ago. Below that stimulus, Ive plotted theluminance of each pixel in that slice.


6/34

l

xxgc

x

Now Id like you to consider the responses of 10 cones trained on that horizontal slice. Their positions are indicated by the 10rectangles and the bottom of this slide. Each cone has a very small receptive field, that nonetheless isnt merely a point; itsan area. Light from the centre of that area maximally excites the cone, but nearby light can also excite it. The middle graphshows a slice through their receptive fields.


7/34

l

xe

x

Each cone excites a bipolar cell. There may be some bipolar cells with input from more than one cone, but lets keep thingssimple in our model. Each of these bipolar cells has excitatory input from exactly one cone. Thus the blue curves in themiddle graph now describe the excitatory inputs to 10 bipolar cells.


8/34

l

xe,i

x

Bipolar cells, in turn, excite horizontal cells. And these cells inhibit nearby bipolars. This inhibition is a form of gain control. Ithelps prevent bipolar cells from reaching their maximum response (which isnt in spikes, but the cells still have a maximumelectrical potential). Since the inhibition comes from neighbouring cells (albeit indirectly), the ultimate cause of that inhibitionis light that comes from parts of the visual field outside the excitatory zone. Ive plotted these 10 inhibitory zones in red.


9/34

l

x

x

gbe- i =

one receptive field in the retina

To get a bipolar cells receptive field, you need to subtract the inhibitory part from the excitatory part, and Ive done that herefor this purple bipolar cell. Remember that this curve represents a 1-dimensional slice through a 2-dimensional receptivefield. You can see a 2-D representation .


10/34

l

x

x

l *gb

As previously discussed, we can model the output of a visual neurone as the dot product of its receptive field and the image.This slide shows the dot products for our 10 bipolar cells.

The responses ive plotted here are for 10 widely spaced bipolar cells. Actual bipolar cells are much more densely packed. Ifthere were, say 200 similar cells responding to a horizontal line through the simultaneous contrast stimulus, their responseswould look something like this:


11/34

l

x

x

l *gb

This curve represents the activity in a line of cells all having receptive fields of the same shape. You can see that maximumactivity occurs wherever there is an abrupt change in luminance.

Away from the edges, these cells all have a similar response. That is, they respond similarly to the white bits, the grey bits,and the black bits. Nonetheless we obviously do see the white bits, grey bits and black bits di!erently. To understand why, wemay have to move out of the eye and into the brain.


12/34

l

x

x

one receptive field in the cortex

gc

l *gc

In primary visual cortex, most receptive fields have orientation preferences.Nonetheless, some have a similar spatial scale to the receptive fields in the retina, and therefore the profile of their activitywill resemble that of retinal cells.


13/34

l

x

x

another receptive field in thecortex

l *gc

Other cells in the cortex prefer lower spatial frequencies.Note that these cells also respond to edges, but the details are lost at this scale.


14/34

l

x

x

another receptive field in thecortex

l *gc

Even more detail is lost when the output of particularly large-scale cells is examined.These these cells respond MUCH more in the region of the grey circle on the right than they do in the region of the grey circleon the left. Thats because the grey circle on the right was surrounded by black and the grey circle on the left was surroundedby white.


15/34

Thus, there are at least two components to a complete explanation of the visual systems exaggeration of simultaneousluminance contrast.The first component is that fine-scale lateral inhibition makes the visual system particularly sensitive to abrupt changes inluminance. We experience those abrupt changes as boundaries.Our experience of what lies between those boundaries depends on cells with much larger receptive fields. These too aresubject to lateral inhibition, and that lateral inhibition exaggerates the cross-boundary di!erences.


16/34

Simultaneous

orientation contrast

Here is another example of the visual system exaggerating simultaneous contrast.In this case, its orientation contrast.The truly vertical central elements on either side of fixation will appear to be tilted in opposite directions when you staredirectly between them.Thats because the visual system exaggerates the di!erence between the orientation of each central element and theorientation of its flanking distractors.

This exaggeration is usually pretty large for briefly presented stimuli (i.e. short flashes), but it also depends on the retinal

eccentricity of the central Gabor patterns.


17/34

y

x!(deg)

x

0

-45

45

-90

Here Ill take just one side of this demo, turn it 90 degrees so that the middle Gabor is now horizontal, and take a slice out ofit. This will allow us to examine how lateral inhibition exaggerates simultaneous orientation contrast.

The curve in the lower graph represents orientation -- not luminance -- as a function of space. In mathematics, it isconventional to use the Greek letter Theta to indicate orientations between -90 and 90 degrees with respect to horizontal.

Note there are really only two orientations in this piece of the stimulus.


18/34

!

x!

x

0

-45

45

0

-45

45

Ive moved the slice through orientation space up to the top of the slide, and Ive drawn a cartoon of 75 receptive fieldsbelow. Each of these receptive fields is oriented. The ones in the middle row belong to cells with a preference for horizontal,the ones below that belong to cells with a preference for clockwise tilts, the ones above belong to cells that like anti-clockwise tilts.Each column in this graph represents 5 cells of varying orientation preference but identical spatial preference. That is, theirreceptive fields are all trained on the same part of the visual field. Cells in primary visual cortex really are arranged this way,and Hubel and Weisel, who made this discovery circa 1950 used the word hypercolumn to describe a group of cells withvarying orientation preference, all trained on the same part of the visual field.

Because inhibition spreads laterally in the cortex, it a!ects nearby cells having a preference for di!erent orientations and thesame position as well as cells having a preference for di!erent positions and the same orientation.In this cartoon ive used blue to illustrate the spread of inhibition around the cell whose receptive field is red.


19/34

!

x!

x

one extra-classical receptive field

0

-45

45

0

-45

45

In this slide, Ive gotten rid of all the cells that dont inhibit the one in middle. Remember that each of these circles representsthe receptive field of one cell in primary visual cortex.More specifically, each circle represents a classical receptive field, such as those we discussed in my lecture on orientation.

Like the bipolar and ganglion cells in the retina, the neurones in primary visual cortex have an excitatory centre and aninhibitory surround. The di!erence is that here centre and surround are defined with respect to orientation and space, not

just space. Another di!erence is that, in cortex, inhibition is better modelled as divisive rather than subtractive.

In the cortex, the combination of excitatory centre and divisively inhibitory surround is usually called the extra-classicalreceptive field.


20/34

!

x!

xlocal

population response

before

inhibition

response

after

inhibition

lateral

inhibition =

=

!(deg)

0

-45

45

0

-45

45

Now Id like to cut through the cortex and plot the activity of cells in one hypercolumn. The population of cells that is trainedon the centre of the horizontal grating.The three plots on the bottom of this slide may be kind of hard to read, but theyre important.Their horizontal axes are identical to the vertical axis in the middle graph.That is, they indicate the orientation preference of cells in visual cortex.If there were no lateral inhibition, then the cells preferring horizontal (i.e. 0 deg) would be maximally stimulated by thehorizontal grating whose position is indicated by the dashed line.Note that Ive written local population response before inhibition above the left plot. Thats not an accident. Because more

synapses are involved, inhibition is thought to lag very slightly behind what is known as the forward sweep of excitation.Psychophysical evidence of this lag is equivocal, but electrophysiologists can quantify it.Anyway, a millisecond or two after the forward sweep, inhibition from other hypercolumns arrives at this particularpopulation, and because those hypercolumns were activated by a tilt of -45 deg, the corresponding cells in the centrepopulation are most inhibited. The formula for calculating the post-inhibitory population response is shown here. Its asimple division.Greater inhibition of cells preferring negative tilts increases the relative activity in cells preferring positive tilts, and that is the most popular model of simultaneous orientation contrast.


21/34

!

x

0

-45

45

Lets take a look at this receptive field again. You can see that inhibition spreads over about 45 degrees of orientation. Youmay want to know how far it spreads across the visual field.


22/34

!

x

0

-45

45

My psychophysical experiments suggest that it spreads very far indeed.These data points come from a 2010 study by Mareschal, Morgan, and Solomon. In that study Isabelle Mareschal measuredthe tilt illusion, which is the name most researchers give to the visual exaggeration of simultaneous orientation contrast. Sheused displays similar to the one youve seen, in which a horizontal, or near-horizontal target was flanked by other Gabors.Open symbols summarise trials in which the target was presented at 4 degrees eccentricity and the filled symbols summarisetrials in which the target was presented at 10 degrees eccentricity.First consider the red symbols. They represent trials in which the flankers were tilted either 22 degrees clockwise or 22degrees anti-clockwise of horizontal. When these flankers were present, they caused the horizontal target to appear tilted by

roughly 10 degrees in whichever direction was opposite to that of the flankers. That is, they produced a negative orrepulsive bias in the targets apparent tilt. This was true even when the distance between flanker and target was almost halfthe latters eccentricity.The blue symbols, on the other hand, represent trials in which the flankers were tilted just 5 degrees o!horizontal. Whenthose flankers were close to the target, they seemed to produce a positive bias. You may be interested to learn that theopposite of repulsive in this particular sense is not attractive, its assimilative.


23/34

Small-angle assimilation

And, sure enough, the visual system does indeed make it hard to notice subtle changes in orientation and any other featuredimension you care to name, particularly when you dont look directly at the stimulus.

Again, it depends where youre sitting. You might need to fixate on the right side of this slide rather than the middle, but allof you should be able to find a fixation for which you can still see all three Gabors without being able to see the di!erences intheir orientations.

Those of you still awake might be slightly confused. In a previous lecture, I discussed orientation bandwidth, and I noted that

it didnt really matter if individual neurones had fairly poor selectivity for orientation because the visual system was verye"cient at decoding population responses, and consequently most observers could easily discriminate between Gaborpatterns di!ering in orientation by just a degree or so.

On the other hand, here I am demonstrating that you really cannot see that these three Gabors have di!erent orientationsunless you look almost directly at them. WHAT GIVES? Well, what gives is currently the subject of rather intensive research.


24/34

Crowding

Now, what Im about to say is rather speculative, but I have good reason to believe that whatever is responsible for small-angle assimilation, it too extends over a vast swathe of visual cortex. The only reason small-angle assimilation disappearswhen flankers and target are widely separated is because lateral inhibition is stronger. To get around the problem of lateralinhibition, some researchers have adopted tasks that should be immune to it. That is, they use visual tasks that do notrequire making fine discriminations between similar orientations or luminances or whatever. One task they particularly like isletter identification. This task is particularly popular because its easy to tell observers what to do. Its the same reasonoptometrists use Snellen letters when they want to measure acuity.The idea behind this figure is that the visual system is compelled to combine information from discrete stimuli, and

consequently cannot disentangle the identity of one individual element from the overall statistics of information coming fromthat region of the visual field. In general, the name given to this phenomenon is Crowding, and experiments with letteridentification suggest a compulsory combination of visual information from individual letters separated by anything up to halftheir average eccentricity.


25/34

I know youve seen this slide before, but just consider the basic idea behind crowding. The idea is that we dont experienceindividual stimuli away from fixation. Instead we experience something more like the average stimulus in that region.

The scientists who made this figure took that idea quite a bit further. In each panel, the two images have the same averageluminance, the same average frequency, the same average orientation and same average colour in spatially correspondingregions. Where they di!er is in the size of those regions. At the red spots in A or the intersection of crosshairs in B and C, theregion is very small: just one pixel. Region size increases with distance from this point, and if you stare directly at any ofthese points, it really is hard to see that the rest of the image is garbage.

This is why I think that small-angle assimilation is a lot like crowding. In both cases, the visual system homogenises imagestatistics in regions outside the fovea. When the image contains closely spaced Gabors of similar orientation, observers tendto perceive their average orientation. However, there is also good reason to suspect that perception might depend onstatistics other than the average.


26/34

Spatiotemporal

crowdin

This is possibly my all-time favourite demonstration of perceptions reliance on statistical summaries rather than the specificsof any particular stimulus.It was made by George Alvarez and Jordan Suchow.Fixate on the small white speck in the centre of the dot ring. At first, the ring is motionless and its easy to tell that the dotsare changing colour. When the ring begins to rotate, you no longer see the individual dots changing colour...unless you lookright at them.The reason you dont notice the colour change is because the statistics of the image, in particular the variance of colours inany region at any time, would be the same regardless whether the colours were changing or not.

And its these regional statistics that are all we can perceive, unless we look directly at the region.


27/34

And that brings us full circle to the beginning of the last lecture. Subtle changes in image information go unnoticed, and ourvisual experience better corresponds to the statistics in that region.

Last week I had you look at one corner of the black rectangle. Those of you who were particularly good at inhibiting saccadesand drift will have seen a single sort of uniform muddy yellow instead of distinctly coloured blotches here and there. This wasan example of interpolation of colour statistics, but all sorts of statistics can be interpolated, and I have one last cool demo toshow you.


28/34

Artificial scotoma demoDont watch this if you have photosensitive epilepsy.

If you stare at the red dot, youll again experience a form of Troxler fading. Thats what all these types of phenomena are, bythe way. In this case, the thing that fades is the border between the flashing black and white texture and the uniform grey. Afew seconds of good fixation should be su"cient to e!ectively silence those neurones that are sensitive to this border. In thenext lecture I will discuss why those neurones are silenced, but for now, simply accept the fact that they are. Consequently,with no indication that something is di!erent above fixation, we simply experience that , well , there isnt anything di!erentthere. Whichever neurones are responsible for our experience of that region, they are e!ectively reporting the same thing thatall nearby neurones are reporting. This general phenomenon is called filling-in, and something like it is responsible for theuniform appearance of the grey discs in the simultaneous contrast illusion.


29/34

Artificial scotoma demoDont watch this if you have photosensitive epilepsy.

Remember: if our perception simply reflected ganglion cell responses, then a grey disc surrounded by white would seemdarker near the edge than at its centre. It doesnt because some filling-in process over-rides and assimilates subtledi!erences in neural activity that arent likely to reflect real di!erences in any stimulus. Ramachandran and Gregory calledthis particular demonstration the artificial scotoma because even patients with real scotomata experience homogenousimages as homogenous. Localised retinal damage can prevent the visual system from gathering evidence contrary to regionalstatistics, thus it is those statistics that determine visual experience. I think the artificial scotoma is particularly cool becauseit demonstrates that regional statistics arent computed instantaneously. It takes a bit of time. Consequently, when the imagestatistics change everywhere else, it takes a little while for our experience of homogeneity to catch up. Specifically, you

should continue to see some dynamic texture in the region where there never was any in the first place. Michael Morgan callsthis briefly seen dynamic texture The Phantasm. Note that if you saw some of this phantasmagoric texture somewherebesides the little region above fixation, then you probably have a real scotoma, and I recommend seeing an optometrist aboutit.


30/34

RECAP

Sensory systems (incl. vision) exaggeratethe contrast of all adjacent features.

Well, I never got around to demonstrating induced motion or numerosity contrast or


31/34

Simultaneous contrast

contrast

turning demo

Id be remiss if I didnt show a demo of simultaneous contrast contrast.This was described in my very first scientific paper.You should know that the rotation isnt necessary for this illusion.Even if the texture were static, the apparent contrast of the centre would still fluctuate out of phase with the physical contrastof the surround, even though the centres physical contrast isnt changing.I wanted to rotate the centre and surround so the illusion couldnt be ascribed to any high-level scene interpretations, like apiece of frosted glass or a cloud over the centre of a uniform surface. None of that cognitive stu!is necessary here. Theexaggeration of contrast di!erences is a straightforward consequence of lateral inhibition or gain control between

neighbouring neurones sensitive to texture contrast.


32/34

Context RECAP

Sensory systems (incl. vision) exaggeratethe contrast of all adjacent features.

Lateral inhibition can cause this. Small feature contrasts go unnoticed,

especially in the periphery.

Other feature statistics (besides averages)can also be calculated outside central

vision.

Sensory systems will exaggerate the contrast between adjacent stimuli, whatever their feature di!erences.Lateral inhibition can cause this. Lateral inhibition is an important component of gain control. Gain control is necessary tokeep neural activity low. Neural activity is metabolically very expensive, but I dont know of any evidence suggesting that youcan burn more calories by thinking harder. Im not sure thinking harder even makes sense.When feature contrasts are small, the visual system sometimes assimilates them, and what the observer experiences is betterdescribed as an average of the two feature intensities.Finally, Freeman and Simoncellis soldier in Iraq and Alvarez and Suchows rotating spots clearly demonstrate that otherfeature statistics (such as the colour variance) are also calculated outside central vision. When those statistics match what

were attending at the fovea, our experience is one of stimulus homogeneity, even if it isnt physically homogeneous.


33/34

Further reading

Schwartz, O., Hsu, A., & Dayan, P. (2007).Space and time in visual context. NatureReviews Neuroscience, 8,522-535.

Pelli, D. G., & Tillman, K. A. (2008). Theuncrowded window of object recognition.Nature Neuroscience, 11, 1129-1135.

Although I tend to think of contrast illusions and crowding as two sides of the same contextual coin, most researchers dont.Thus, Im going to recommend two very di!erent papers. Schwartz et al provide a very thought-provoking review of contrastillusions and adaptation, the latter of which will be the topic of my next lecture. Pelli & Tillman have a really fun review ofcrowding, but neither of these papers discusses filling in.


34/34

Further reading

Solomon, J. A. & Mareschal, I. (2013) Theincompatibility of feature contrast and

feature acuity. In C. Chubb, B. Dosher, Z.-L.

Lu & R. Shiffrin (Eds.) Human InformationProcessing: Vision, Memory, Attention.Washington DC: American Psychological

Association.

Perhaps the best place to read about the topics covered in this lecture is an old Chapter by me & Isabelle Mareschal. Despitethe 2013 publication date, this book chapter, which I suppose I can upload to Moodle, was written in 2008 for a festschrift inthe honour of George Sperling.It was Sperling and Chubb who gave my career a push with the simultaneous contrast contrast illusion back in 1989. And,with my co-author Isabelles help, I tried to summarise just how far that push had gotten me back in 2008. As it turns out,there has been more research on contrast contrast in the last five years than in all the time between its discovery and 2008,so if you really want the latest on contrast contrast, I will also upload Chapter 2 in Artie Shapiros forthcoming book on VisualIllusions.

Documents

Context Notes Fixed