A two-stage model of orientation integration for Battenberg-modulated micropatterns

Alex S Baldwin $School of Life and Health Sciences Aston University

Birmingham UK

Jesse S Husk $Department of Ophthalmology McGill University

Royal Victoria Hospital Montreal Quebec Canada

Tim S Meese $School of Life and Health Sciences Aston University

Birmingham UK

Robert F Hess $Department of Ophthalmology McGill University

Montreal Quebec Canada

The visual system pools information from local samplesto calculate textural properties We used a novelstimulus to investigate how signals are combined toimprove estimates of global orientation Stimuli were 29middot 29 element arrays of 4 cdeg log Gabors spaced 18apart A proportion of these elements had a coherentorientation (horizontalvertical) with the remainderassigned random orientations The observerrsquos task wasto identify the global orientation The spatialconfiguration of the signal was modulated by acheckerboard pattern of square checks containingpotential signal elements The other locations containedeither randomly oriented elements (lsquolsquonoise checkrsquorsquo) orwere blank (lsquolsquoblank checkrsquorsquo) The distribution of signalelements was manipulated by varying the size andlocation of the checks within a fixed-diameter stimulusAn ideal detector would only pool responses frompotential signal elements Humans did this for mediumcheck sizes and for large check sizes when a signal waspresented in the fovea For small check sizes howeverthe pooling occurred indiscriminately over relevant andirrelevant locations For these check sizes thresholds forthe noise check and blank check conditions were similarsuggesting that the limiting noise is not induced by theresponse to the noise elements The results aredescribed by a model that filters the stimulus at thepotential target orientations and then combines thesignals over space in two stages The first is a mandatoryintegration of local signals over a fixed area limited byinternal noise at each location The second is a task-dependent combination of the outputs from the firststage

Introduction

Combining orientation signals over space

The perception of coherent textures requires theintegration of orientation signals over space Thedefinition of regions in an image that lsquolsquobelongrsquorsquo to thesame texture is a necessary intermediate step tohigher-level processes such as finding boundariesbetween different textures (Marr 1982 Landy ampGraham 2004) This study focuses on examining thestrategies used for choosing which local samples tocombine over space to calculate a global orientationestimate from a stimulus (which we shall call thelsquolsquopooling strategyrsquorsquo) rather than focusing on address-ing the process by which the individual local signalsare combined (which we shall call the lsquolsquocombinationprocessrsquorsquo as investigated previously by Dakin amp Watt1997 Jones Anderson amp Murphy 2003 WebbLedgeway amp McGraw 2010 Husk Huang amp Hess2012) Our observers are tasked with making globalorientation judgments for displays containing orien-tation micropatterns with various orientations Bymanipulating the spatial layout of these micropat-terns it is possible to distinguish between some of thedifferent spatial-pooling strategies that have beenproposed previously

Citation Baldwin A S Husk J S Meese T S amp Hess R F (2014) A two-stage model of orientation integration forBattenberg-modulated micropatterns Journal of Vision 14(1)30 1ndash21 httpwwwjournalofvisionorgcontent14130doi10116714130

Journal of Vision (2014) 14(1)30 1ndash21 1httpwwwjournalofvisionorgcontent14130

doi 10 1167 14 1 30 ISSN 1534-7362 2014 ARVOReceived June 11 2012 published January 30 2014

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

Signal-combination processes

Effects of spatial configuration aside the combina-tion process by which the visual system calculates aglobal orientation from an array of local orientationshas been found to depend on the task set to theobserver Similar dependencies have also been reportedin studies that investigated the integration of localmotion signals When the observer is required todistinguish between stimuli with weak horizontal orvertical orientation coherence (ie with a largedifference between the two target orientations) ob-servers filter the image at the two potential targetorientations and then choose the orientation of themore strongly activated filter (Husk et al 2012) Awinner-takes-all process similar to this has been foundin analogous motion studies performed in monkeys(Salzman amp Newsome 1994)

Under conditions in which finer judgments of theglobal orientation of a texture need to be made theobserver calculates the vector-average of the localorientations (Dakin amp Watt 1997 Webb et al 2010)Similar changes in the combination process used byobservers based on the difference between thediscriminated orientations have been demonstrated inthe motion domain (Nichols amp Newsome 2002Webb Ledgeway amp McGraw 2007) The largeorientation differences used in the experimentsreported here would be expected to cause the observerto max over filter outputs (the design of this study issimilar to that of Husk et al 2012) For our purposeshowever it is not necessary to assume that theobserver makes use of a particular combinationprocess Models that use a vector-averaging combi-nation process produce very similar predictions tothose made by the filter-maxing model presented inthe body of this paper (see Appendix C)

Pooling strategies and summation effects

Most signal-combination processes would predictan improvement in performance for detecting weaksignals as the number of samples increases Providedthat the noise affecting each sample is at leastpartially independent the limiting effect of the noiseon performance can be reduced by exploiting theinformation from multiple samples Pooling overadditional samples in this manner will improveperformance regardless of whether the observer isfilter-maxing or vector-averaging There are variouspossible strategies for pooling signals over spacewhich make different predictions for how perfor-mance should improve with the availability ofadditional signal samples Previous work in which thenumber of samples available for combination is

varied have reported conflicting results Dakin (2001)found a completely flexible combination with respectto signal location over a proportion of the samples inthe display This was presented as an lsquolsquoinformationlimitrsquorsquo for orientation integration Other studies haveshown either improvements reflecting ideal summa-tion under a flexible pooling strategy up to somemaximum integration area (Jones et al 2003) or nobenefit from increasing the number of sampleswhatsoever (Husk et al 2012)

This study

The summation effects resulting from increasingthe number of samples available for integration areinvestigated here using psychophysics and computermodeling The novel lsquolsquoorientation Battenbergrsquorsquo stimuliused allow for manipulation of the spatial arrange-ment of signal within a stimulus of fixed extent andeccentricity This reduces the confounding effects ofany inhomogeneities in sensitivity for performing theglobal orientation task (similar to the contrastBattenberg stimuli used by Meese 2010) Jones et al(2003) suggested that such an effect might havereduced the level of summation measured in theirstudy Our results show approximately linear sum-mation over short distances (reflecting the summationof signal against a constant noise floor with anincreasing number of samples) followed by a perfor-mance improvement consistent with ideal summationover longer distances (reflecting summation of bothsignal and the variances of per-location noise as thenumber of sampled locations increases) Previousinvestigations of the mechanisms underlying theperception of coherent texture have described modelsfeaturing an initial local integration stage in which theorientation statistics at each location are estimatedfollowed by further operations performed over thoselocal estimates (Vorhees amp Poggio 1988 Sagi 1990Dakin amp Watt 1997) A two-stage model of this kindis supported by this study the results of which suggestthat observers perform mandatory local integration(affected by internal noise at each location) followedby flexible long-range pooling

Methods

Equipment

Stimuli were presented on a gamma-corrected CRTmonitor using Psychtoolbox (Brainard 1997 KleinerBrainard amp Pelli 2007) running under MATLAB Thedata collection for these experiments was split between

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 2

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

two different equipment setups The first was an AppleMacbookProwith anNVIDIAGeForce 9600Mgraphicscard presenting stimuli on a Philips MGD403 monitorthe second was an AppleMacbook Pro with anNVIDIAGeForce 8600M graphics card presenting stimuli on aCompaqmonitorThemonitorshadrefresh ratesof 75Hzand 90 Hz and mean luminances of 772 and 269 cdm2respectively Observers viewed the monitors from adistance of 051 m at this viewing distance giving 6 pixelsper cycle for the 4 cdeg stimuli used here

Stimuli

Stimuli were 29 middot 29 element arrays of 4 cdegcosine-phase log-Gabor patches (spatial frequency andorientation bandwidths of 16 octaves and 6258respectively see Meese 2010) spaced 1 degree apart ina square grid Stimuli were displayed at 80 delta-contrast

cdelta frac14maxethjL LmeanjTHORN

Lmean eth1THORN

where L is the stimulus image and Lmean its meanluminance Each log-Gabor was either a potentialsignal element or a noise element Potential signalelements had probability P(coherence) of assuming thetarget orientation (which was either horizontal orvertical) otherwise they assumed an orientation drawnat random from a uniform distribution Therefore on atrial-by-trial basis a particular coherence level did notguarantee that a certain number of signal elementswould appear in the stimulus but over many trials theaverage (or lsquolsquoexpectedrsquorsquo) proportion of signal elementsin the stimulus would be equal to the coherence levelAll noise elements assumed random orientations Therange of potential element orientations was 08 to 1798(orientations were rounded to the nearest degree beforestimulus generation)

Two stimulus types were tested lsquolsquofullrsquorsquo andlsquolsquocheckedrsquorsquo In the full stimuli all elements were

potential signal elements For the checked stimuli thepotential signal elements were assigned to locations inthe stimulus defined by a checkerboard (a square-waveplaid) This gave a stimulus tiled with square signal andnonsignal regions Two types of checked stimuli weretested For the lsquolsquonoise checkrsquorsquo condition the nonsignalregions contained randomly oriented elements There-fore each of the checked conditions contained the sametotal number of elements but approximately half asmany signal elements as the full condition (see Table 1)For the lsquolsquoblank checkrsquorsquo condition the nonsignal regionswere blank and so the checked stimuli containedapproximately half as many elements as the full stimuli

The spatial arrangement of the signal regions in thestimulus was manipulated by adjusting the frequencyand the phase of the square-wave plaid modulator thatdefined the checkerboard Decreasing or increasing thefrequency made the signal regions larger or smallerrespectively and this was used to create the differentcheck sizes These gave stimuli tiled with 1- 3- 5- 9-and 15-element square signal regions (ie the largesthad 15 middot 15 element lsquolsquochecksrsquorsquo) The phase of themodulation was also manipulated to test stimuli inboth the cosine (frac14 908) and anticosine ( frac14 2708)phases Thus in the frac14908 condition the dispersion ofsignal was such that it was included in the central partof the display whereas in the frac14 2708 condition therewas no signal in the central region Miniature examplestimuli are shown in Figure 1 Full-size examples ofeach of the stimuli used in the experiments are availableas Supplementary material (Figures S1ndashS21)

Procedures

A blocked single-interval identification task wasperformed to find the orientation identificationthreshold for each check size phase (frac14 908 vs 2708)and checked Battenberg type (noise check vs blankcheck) Thresholds were tracked using a pair of three-down one-up staircases (maximum 120 trials or 12reversals) one for horizontal and the other for vertical

Check size

Modulator phase frac14 908 Modulator phase frac14 2708

signal elements Proportion signal elements Proportion

1 middot 1 421 501 420 499

3 middot 3 420 499 421 501

5 middot 5 420 499 421 501

9 middot 9 445 529 396 471

15 middot 15 421 501 420 499

Table 1 Numbers and proportions of potential signal elements in the various checked lsquolsquoBattenbergrsquorsquo stimuli used in this study NotesThe total number of elements in the full Battenberg stimulus was 841 Noise check stimuli always contained 841 elements with thenonsignal elements set to random orientations Blank check stimuli did not contain any elements other than those that werepotential signal elements

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 3

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

two different equipment setups The first was an AppleMacbookProwith anNVIDIAGeForce 9600Mgraphicscard presenting stimuli on a Philips MGD403 monitorthe second was an AppleMacbook Pro with anNVIDIAGeForce 8600M graphics card presenting stimuli on aCompaqmonitorThemonitorshadrefresh ratesof 75Hzand 90 Hz and mean luminances of 772 and 269 cdm2respectively Observers viewed the monitors from adistance of 051 m at this viewing distance giving 6 pixelsper cycle for the 4 cdeg stimuli used here

Stimuli

Stimuli were 29 middot 29 element arrays of 4 cdegcosine-phase log-Gabor patches (spatial frequency andorientation bandwidths of 16 octaves and 6258respectively see Meese 2010) spaced 1 degree apart ina square grid Stimuli were displayed at 80 delta-contrast

cdelta frac14maxethjL LmeanjTHORN

Lmean eth1THORN

where L is the stimulus image and Lmean its meanluminance Each log-Gabor was either a potentialsignal element or a noise element Potential signalelements had probability P(coherence) of assuming thetarget orientation (which was either horizontal orvertical) otherwise they assumed an orientation drawnat random from a uniform distribution Therefore on atrial-by-trial basis a particular coherence level did notguarantee that a certain number of signal elementswould appear in the stimulus but over many trials theaverage (or lsquolsquoexpectedrsquorsquo) proportion of signal elementsin the stimulus would be equal to the coherence levelAll noise elements assumed random orientations Therange of potential element orientations was 08 to 1798(orientations were rounded to the nearest degree beforestimulus generation)

Two stimulus types were tested lsquolsquofullrsquorsquo andlsquolsquocheckedrsquorsquo In the full stimuli all elements were

potential signal elements For the checked stimuli thepotential signal elements were assigned to locations inthe stimulus defined by a checkerboard (a square-waveplaid) This gave a stimulus tiled with square signal andnonsignal regions Two types of checked stimuli weretested For the lsquolsquonoise checkrsquorsquo condition the nonsignalregions contained randomly oriented elements There-fore each of the checked conditions contained the sametotal number of elements but approximately half asmany signal elements as the full condition (see Table 1)For the lsquolsquoblank checkrsquorsquo condition the nonsignal regionswere blank and so the checked stimuli containedapproximately half as many elements as the full stimuli

The spatial arrangement of the signal regions in thestimulus was manipulated by adjusting the frequencyand the phase of the square-wave plaid modulator thatdefined the checkerboard Decreasing or increasing thefrequency made the signal regions larger or smallerrespectively and this was used to create the differentcheck sizes These gave stimuli tiled with 1- 3- 5- 9-and 15-element square signal regions (ie the largesthad 15 middot 15 element lsquolsquochecksrsquorsquo) The phase of themodulation was also manipulated to test stimuli inboth the cosine (frac14 908) and anticosine ( frac14 2708)phases Thus in the frac14908 condition the dispersion ofsignal was such that it was included in the central partof the display whereas in the frac14 2708 condition therewas no signal in the central region Miniature examplestimuli are shown in Figure 1 Full-size examples ofeach of the stimuli used in the experiments are availableas Supplementary material (Figures S1ndashS21)

Procedures

A blocked single-interval identification task wasperformed to find the orientation identificationthreshold for each check size phase (frac14 908 vs 2708)and checked Battenberg type (noise check vs blankcheck) Thresholds were tracked using a pair of three-down one-up staircases (maximum 120 trials or 12reversals) one for horizontal and the other for vertical

Check size

Modulator phase frac14 908 Modulator phase frac14 2708

signal elements Proportion signal elements Proportion

1 middot 1 421 501 420 499

3 middot 3 420 499 421 501

5 middot 5 420 499 421 501

9 middot 9 445 529 396 471

15 middot 15 421 501 420 499

Table 1 Numbers and proportions of potential signal elements in the various checked lsquolsquoBattenbergrsquorsquo stimuli used in this study NotesThe total number of elements in the full Battenberg stimulus was 841 Noise check stimuli always contained 841 elements with thenonsignal elements set to random orientations Blank check stimuli did not contain any elements other than those that werepotential signal elements

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 3

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

signal trials The staircases for the two signal orienta-tions were interleaved randomly Once the staircase forone orientation had terminated dummy trials (in whichno data were recorded) were still presented with thatorientation until the staircase for the other orientationterminated Staircases started at a high signal level toinform the observers of what stimulus to expect in eachblock Stimuli were presented for 250 ms Stimulusonset was accompanied by a beep Observers main-tained continuous central fixation with the aid of ablack fixation dot that was shown between trials Theobservers pressed a key on a keyboard to indicatewhether the stimulus contained either lsquolsquohorizontalrsquorsquo orlsquolsquoverticalrsquorsquo coherence The response was followed by afeedback beep that indicated whether it was correct orincorrect and then a 300-ms pause before the presen-

tation of the next stimulus Each observer performedfour repetitions for each combination of check size(full 1 3 5 9 and 15) Battenberg modulator phase (frac14 908 or 2708) and Battenberg type (noise check orblank check) As the full stimulus was identicalregardless of Battenberg type or modulator phase eachobserver collected four times as much data for thiscondition (16 repetitions) These were averaged to givea single threshold per observer

Observers

Seven observers were used Five were experiencedpsychophysical observers (ASB DHB JSH RJS andSAW) including two of the authors (ASB and JSH)

Figure 1 Example of the stimulus design used in these experiments The stimuli shown are 7 middot 7 element arrays (smaller than the 29

middot 29 arrays used in our experiments) with a 3 middot 3 check size The figure includes frac14 908 and 2708 check versions of the noise check

and blank check stimuli each shown at 100 coherence

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 4

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

Two were naıve undergraduates (LFE and VRP) Allhad either normal or corrected-to-normal visionObservers DHB RJS and SAW were tested on the firstequipment setup described above Observers JSHLFE and VRP were tested on the second equipmentsetup Observer ASB was tested on the first setup forthe noise check and on the second setup for the blankcheck conditions The study was conducted in accor-dance with the Declaration of Helsinki

Analysis

Data from the horizontal and vertical staircases werecombined into a single psychometric function for eachrepetition condition and observer This was then fittedby a cumulative normal function using Palamedes(Prins amp Kingdom 2009) The fitted function gave theprobability of responding lsquolsquohorizontalrsquorsquo to either avertical stimulus (plotted as negative coherence) or ahorizontal stimulus (plotted as positive coherence) Thecoherence level at which the function reached P(lsquolsquoHor-izontalrsquorsquo) frac14 05 gave the bias for the observercategorizing a stimulus as horizontal rather thanvertical (the distributions of the biases found for eachobserver are presented in Appendix A) and theorientation identification threshold could be calculatedas the difference between the coherence level at thispoint and that at which P(lsquolsquoHorizontalrsquorsquo) frac14 075

Results

Noise check

The orientation identification thresholds averagedacross the seven observers are shown in Figure 2 Thethree rows present the data plotted as the expectednumber of signal elements in the stimulus at threshold(Figure 2a and b) the threshold probability of theelements in the signal region assuming the targetorientation (Figure 2c and d) which is equivalent to theexpected proportion of signal elements within the signalregions at threshold (hereafter termed the lsquolsquocoherencethresholdrsquorsquo) and the expected proportion of signalelements across the entire stimulus at threshold (Figure2e and f) In subsequent figures the metric shown inFigure 2c and d is used with this coherence thresholdfor each checked stimulus plotted as a multiple relativeto that for the full condition (which had a coherencethreshold of approximately 10)

Figure 2c presents the coherence thresholds for thenoise check condition For smaller check sizes (1 to 3)the thresholds for the checked stimuli were approxi-mately double that for the full stimulus This means

that around the same total number of signal elementsacross the entire stimulus were required to reach thethreshold performance level in each case (see Figure2a) For medium sizes (5 to 9) the threshold elevationdecreased to a factor of

ffiffiffi2p

This is consistent with astrategy that uses information from potential signalregions but ignores irrelevant (noise-only) regions asthe noise resulting from the combination of multipleequally noisy samples is proportional to the square rootof the number of samples combined (a simple lsquolsquosum ofvariancesrsquorsquo rule) As the stimuli were blocked by checksize and modulator phase the observer could predictin each trial which areas of the display were potentialsignal regions and which would contain only irrelevantnoise

One of the purposes of the Battenberg stimulusdesign is to reduce the effect of visual field inhomo-geneities in sensitivity on the measurement of areasummation (Meese amp Summers 2007 Meese 2010)For stimuli with large check sizes relative to theirextent however these effects will return For thelargest (15) check size in Figure 2c performancediverged dependent on whether the stimulus was in the frac14 908 (foveal signal peripheral noise) or frac14 2708(foveal noise peripheral signal) phase Coherencethresholds were almost as low for the frac14 908 stimulusas they were for the full stimulus This means that theobservers required around half as many signalelements across the entire stimulus when most of thoseelements were presented in the center of the display(Figure 2a) For the frac14 2708 stimulus coherencethresholds were approximately double that of the fullstimulus Therefore observers required the samenumber of signal elements in the largest frac14 2708stimulus as they did in the full stimulus (because thefull stimulus has approximately twice the checkedstimulusrsquos signal area) This behavior would beconsistent either with a relative insensitivity for thistask in the periphery or a failure in segregating thenoise present in the center of the display (this isdiscussed further below)

Blank check

Figure 2d shows the coherence thresholds for theblank check condition For the smaller check sizes inthe blank check condition coherence thresholdsincrease to approximately double that for the fullstimulus in a similar manner to that seen in the noisecheck condition This is unexpected as the predictedcoherence threshold for this condition based on thestimulus properties alone would be a factor of

ffiffiffi2p

above that for the full condition (there are no noise-only elements so the threshold would be proportionalto the square root of the number of elements in the

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 5

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

display the sum of variances rule again) These resultssuggest that the performance-limiting noise for thenoise check task does not arise from the responses tothe interstitial noise elements otherwise the removal ofthese elements in the blank check condition wouldresult in a decrease in threshold On the other handhowever the noise cannot be lsquolsquolatersquorsquo and constant

across conditions as this would predict the same

performance level for all of the checked stimuli

Instead these results suggest that observers are

mandatorily integrating internal noise from blank

display regions (or are limited by a noise source that is

proportional to the monitored area) for the smaller

Figure 2 Thresholds from the identification task (averaged over seven observers) shown in three different ways The results are

plotted here as the expected number of signal elements in the stimulus at threshold (andashb) the expected percentage of signal

elements in the potential signal regions at threshold (cndashd) and the expected percentage of signal elements in the entire stimulus at

threshold (endashf) The metric used in other results figures in this paper is that in c and d with these lsquolsquocoherence thresholdsrsquorsquo expressedas multiples relative to that of the full (F) stimulus Error bars show 61 standard error here and in all other graphs

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 6

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

check sizes but are able to exclude this noise for thelarger check sizes

The similarities in the noise check and blank checkdata shown in Figure 2c and d can be confirmed bylooking ahead to Figure 5 which compares thethresholds from these conditions directly by showingthe amount of threshold elevation caused by thepresence of the noise checks Up to the largest (15)

check size thresholds for the frac14 908 stimuli are similar(threshold elevation factor of approximately one)suggesting that the pooling of samples (and segregationof noise) in these two conditions is similar For the frac142708 stimuli however threshold elevation increaseswith check size Performance for the largest (15) frac142708 stimulus in the blank check condition is a factor offfiffiffi

2p

better than that in the noise check condition

Figure 3 Coherence thresholds for the checked stimuli expressed as multiples of the full stimulus threshold plotted with predictions

from the SA (andashb) SI (cndashd) and TS (endashf) models Each row shows the same averaged data replotted from Figure 2c and d The only

fitted parameter was k the size of the pooling region in the TS model (3 middot 3) This fitting was performed by hand (Appendix B) RMS

errors between the model predictions and the data are shown in dB

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 7

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

meaning that at least part of the deficit for detecting

that stimulus was due to inefficient pooling of noise in

the noise check condition rather than a relative

insensitivity for performing the task in the periphery

(eg it may be more difficult for the observer to keep

track of the signal and nonsignal regions in the frac142708

noise check condition)

Modeling

Monte Carlo simulations

A set of models was developed to investigate thepooling strategy used by the observers The initial aim

Figure 4 Coherence thresholds plotted with predictions from the TN model fitted to the blank check data (andashb) and from the TA

model with the best-fitting parameters for the noise check data (TA1 cndashd) and the best-fitting parameters for the blank check data

(TA2 endashf) Each row shows the same averaged data replotted from Figure 2c and d The fitted parameters were the size of the pooling

region (k) the standard deviation of the internal noise (rint) and the size of the integration aperture (a)

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 8

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

of the modeling was to explain the surprising equiva-lence between the smallest check sizes in the noise checkand blank check conditions (see Figure 2c and d)which would not be expected if the source of thelimiting noise was in the responses to the individualmicropattern elements (as the additional randomlyoriented elements in the noise check condition wouldcause thresholds for those stimuli to be higher) Modelpredictions were obtained using stochastic MonteCarlo methods A set of model observers was developedwith different pooling strategies in MATLAB and runthrough 2000 simulated trials per stimulus level of alsquolsquomethod of constant stimulirsquorsquo version of the experi-ment The coherence thresholds for each modelobserver were calculated using the simulated data andexpressed as multiples of the full stimulus thresholdallowing them to be compared to the human resultswithout the need for fitting Three fitted parameters(the size of the local pooling regions k the standarddeviation of the internal noise rint and the size of theglobal pooling region a) were then added in order todevelop a model that provided a close account of thedata These parameters were each fitted by hand (seeAppendix B)

The filter-maxing combination processes

Several combination processes for the calculation ofglobal orientation from individual local samples havebeen suggested previously Here we implement thestrongest candidate for this task in which the observer

selects the orientation of the most strongly activatedoriented filter (after Jones et al 2003 Husk et al2012) This choice is not crucial to our findingshowever and a set of models developed using a vector-averaging combination process made similar predic-tions (Appendix C) In our filter-maxing model thestimulus is first filtered at the two potential targetorientations The filter elements are a pair of log-Gaborelements with the same tuning properties as those usedin the generation of the stimuli (spatial frequencybandwidth of 16 octaves and orientation bandwidth of6258) These bandwidths are typical of those usedpreviously in the literature to model simple cellresponses and compatible with those found in neuro-physiological investigations (De Valois amp De Valois1990 Meese 2010) As our filter elements wereidentical to the target elements in our stimuli they alsobehaved as the ideal detector for those elements Thefilter outputs are rectified and passed to the poolingstage In the pooling stage the outputs can be weightedaccording to their expected signal-to-noise ratio (de-pendent on the pooling strategy in operation seebelow) The weighted filter responses are then summedover the image for each orientation and these valuesare compared to each other The model observer picksthe orientation with the greater filter response

Pooling strategies

The simplest pooling strategy considered is the lsquolsquosumallrsquorsquo (SA) model in which the observer combines

Figure 5 Threshold elevation introduced by the noise check elements calculated from the data in Figure 2c and d as the thresholds

from the noise check condition divided by the thresholds from the blank check condition The predicted threshold elevations from

each of the models shown in Figures 3 and 4 are also plotted here Only the SA and TS models predict any systematic elevation effect

and neither of those match the pattern seen in the data

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 9

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

information from every element in the stimulus with anequal weighting regardless of whether it is a noise orpotential signal element The model with this strategypredicts that within each condition (noise check andblank check) there should be no effect of check size ormodulator phase The predicted coherence thresholdfor the noise check stimuli is approximately twice thatfor the full stimulus (Figure 3a) which is equivalent torequiring the same threshold proportion of coherentsignal elements across the whole stimulus For theblank check stimuli (Figure 3b) the predicted thresholdelevation is a factor of

ffiffiffi2p

as there are no interstitialnoise elements to limit performance in that conditionThese predictions capture the performance for thesmall check sizes (1 to 3) in the noise check conditionthe medium check sizes (5 to 9) in the blank checkcondition and the largest frac14 2708 size (15) in both thenoise check and blank check conditions (see Figure 3aand b) The predictions fail however to describe thegeneral form of either set of results which both featurea transition from greater to lesser summation as thecheck size increases

The lsquolsquosum ideallyrsquorsquo (SI) strategy involves the combi-nation of orientation information only from potentialsignal elements This is equivalent to weighting theelements according to their expected signal-to-noiseratio as the responses to the elements in the noiseregions provide zero signal Models with this strategypredict threshold elevation of a factor of approximatelyffiffiffi

2p

for both of the checked conditions (Figure 3c andd) which is identical to the prediction for the blankcheck stimuli with the SA model For this reason theresults in the blank check condition which were welldescribed by SA models are fit just as well by the SImodels In addition the SI models predict the

ffiffiffi2p

summation for the medium check sizes in the noisecheck condition However this model also fails tocapture the general form of the human data for eitherstimulus type

Two-stage hybrid models

The fact that the simpler candidate models featuringthe two different pooling strategies (SA and SI) eachpredicted performance for different subsets of theresults suggested that a more complete account couldbe provided by a model that combined their twobehaviors In the lsquolsquotwo-stage (TS) hybrid modelsrsquorsquomandatory local combination over a k middot k region(behaving like the SA model) is followed by flexiblepooling of the outputs from those regions weighted bytheir expected signal-to-noise ratios which wereapplied as a template (as in the SI model) The size ofthe local pooling region (k) was the only fittedparameter in this model The best fit to the noise check

data was provided by a k of 3 although the fit providedby a k of 2 was only marginally worse (see Appendix BFigure A2) In the noise check condition this modelpredicts an initial doubling of the coherence thresholdfor the small check sizes followed by an improvementin performance to approach a factor of

ffiffiffi2p

for themedium and large check sizes (see Figure 3e) Thiscaptures the performance for all but the largest (15)check sizes In the blank check condition (Figure 3f)the predictions are once again the same as for modelswith the SA and SI strategies

Internal noise

The SA SI and TS models all make predictions thatare identical to each other for the blank checkcondition This is because there are no interstitial noiseelements in that condition that can be inappropriatelypooled to elevate the coherence threshold The resultshowever show a doubling of threshold for the smallcheck sizes that is the same in the noise check and blankcheck conditions This is a larger performance deficitthan can be accounted for by any of the three models(which all predict a

ffiffiffi2p

threshold increase for the blankcheck condition) A two-stage hybrid model featuringadditive Gaussian internal noise at each location (TN)does however predict similar performance for theblank check conditions as for the noise check condi-tions (see Figure 4a and b) The internal noise in themodel is added after the rectification stage (meaningthat the noisy local outputs can be negative) Becauseof the mandatory combination rule at the first poolingstage this early noise model is equivalent to a model inwhich performance is limited by noise affecting theoutput of the first pooling stage

The standard deviation of the internal noise (rint) isnow an additional parameter in the model It isexpressed as a percentage of the summed element-wisefilter response to a matched log-Gabor element (ierelative to the maximum local filter output) As rint isincreased from zero it initially has the effect ofincreasing thresholds for the 1 middot 1 blank checkcondition to bring them in line with those from thenoise check condition Once these have becomeequivalent higher values of rint serve mainly to increaseor decrease the sensitivity for all conditions Calculat-ing the relative coherence thresholds from the output ofthe model therefore largely factors out the effects ofthis model parameter once rint is sufficiently high (seeAppendix B) Figure 4a and b shows the TN modelprediction with the best-fitting k and rint values for theblank check data (note that the best-fitting k for theblank check data is 2 whereas kfrac14 3 provided amarginally better fit to the noise check data as shown inFigure A2) For both the noise check and the blank

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 10

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

check conditions the TN model predicts an initialtwofold summation for the 1 middot 1 check size which thendecreases to approximately

ffiffiffi2p

as the check sizeincreases The predictions from this model account forthe average human thresholds in the noise checkcondition for all but the largest check size (Figure 4a)In the blank check condition (Figure 4b) thresholds arelower than those predicted by the model for themedium (9) and largest (15) frac14 908 condition but forall other conditions the human behavior is captured bythe model

Maximum integration aperture

In the TN model there are several reasons whycoherence thresholds could be elevated when signal ispresented only in the periphery (compare frac14 908 and2708 thresholds for the largest check size stimulus inFigure 4) The simplest would be if the observer wereonly able to pool information from elements in thecenter of the display This was tested using a lsquolsquotwo-stagehybrid model with internal noise and maximumintegration aperturersquorsquo (TA) in which the modelobserver only had access to information from elementsthat were within a central a middot a element squareaperture (equivalent to a degrees or 4a carrier cycles)This would also be equivalent to a model that featureda step-edge decline in sensitivity at this eccentricityMore complex accounts of the effect of visual fieldeccentricity are not explored here as our stimuli weredesigned to factor out these effects when possiblemeaning that any explanation we might provide wouldnot be well constrained by our data For examplefurther summation may be achieved by probabilitysummation between multiple integration apertures atdifferent locations across the visual field (resulting in amodel similar to that proposed for the area summationof contrast by Baker amp Meese 2011)

In order to show which features of our results the TAmodel could and could not account for two separatefits were performed to determine the optimal set ofparameters for the noise check (TA1) and the blankcheck (TA2) data The TA1 prediction is shown inFigure 4c and d The best-fitting integration aperturesize was 18 middot 18 degrees (compared to other squareapertures with integer dimensions see Appendix B)The TA1 model prediction is similar to that made bythe TN model (for the same kernel size see AppendixB) for all conditions except for the largest check size(15) For the 15 middot 15 check stimuli the predictedthresholds for the frac14 908 stimulus are reduced (whenexpressed relative to the full condition threshold) andthe thresholds for the frac14 2708 stimulus are elevatedThis model prediction provides a good fit to the noisecheck data (RMSefrac14 054 dB) and for the smaller check

sizes (1ndash3) in the blank check condition For the largercheck sizes in the blank check condition however theTA1 model prediction systematically underestimatesthe sensitivity of the human observers (resulting in arelatively large RMSe of 178 dB)

Fitting the aperture model to the blank checkcondition produces the TA2 prediction shown in Figure4e and f As in the TN model the best-fitting k for theblank check condition was 2 (as opposed to the valuefound from the TA1 fit to the noise check data whichfavored a k of 3) The best-fitting aperture size (a) was19 middot 19 degrees (18 wider than the size found by fittingto the noise check data) As would be expected thisversion of the model provides an inferior fit to the noisecheck data compared to the TA1 prediction (116 dB vs054 dB) underestimating both the threshold for the 3middot 3 check condition and the amount of separationbetween the two modulator phases at the largest checksize In the blank check condition although the qualityof the fit is improved from the TA1 prediction (112 vs178 dB) sensitivity is still underestimated for the larger(9ndash15) check sizes This indicates that there is nocombination of parameters for this model architecturethat can capture the performance for those conditions

Residual effects of check condition andmodulator phase

Figure 5 replots the data from Figure 2c and dshowing directly the threshold elevation effect that theinterstitial noise elements in the noise check conditionhave Also shown in Figure 5 are the thresholdelevation predictions from each of the six modelvariants presented earlier All models except SA and TSpredict no threshold elevation from the noise checks ineither modulator phase For the frac14 908 modulatorphase the data agree with this prediction with the ratiobetween the noise check and blank check thresholdsremaining close to one for all check sizes For the frac142708 modulator phase there is no threshold elevationintroduced by the noise checks at the smallest (1) checksize but as the check size increases the noise checkshave the effect of raising thresholds relative to theblank check condition up to a maximum factor of

ffiffiffi2p

for the largest (15) check size Only the SA and TSmodels have their performance limited by the intersti-tial randomly oriented elements in the noise checkstimuli and therefore predict threshold elevation in thenoise check condition The pattern of elevationpredicted however does not match that seen in the frac142708 data The key difference is that the data show athreshold elevation effect that both increases withcheck size and is dependent on the modulator phasewhereas none of the models investigated here make thisprediction One possible explanation for this effect

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 11

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

would be an increase in the size of the mandatory localintegration region with eccentricity (linking this to theexplanation of crowding provided by Parkes LundAngelucci Solomon amp Morgan 2001)

Discussion

Orientation integration is a noisy two-stageprocess

The results of this study suggest that the combina-tion of orientation information over space is a noisytwo-stage process (Figure 6) The key results reportedhere can be accounted for by a model that performsmandatory local integration affected by internal noiseat each location followed by flexible pooling over theoutputs from those regions (for the vector-averagingversion of the model these would seem equivalent tothe involuntary and voluntary averaging discussed byDakin Bex Cass amp Watt 2009) This account is inagreement with previous studies that have found lowerthresholds for stimuli with a greater signal area (Dakin2001 Jones et al 2003) The effect found for thearrangement of the elements in the display contradictsthe flexibility attributed to the pooling of local samplesby Dakin (2001) although it is possible that thedifferences in terms of the task set to the observer(signal in noise here vs fine discrimination in Dakin)would mean that they did not investigate the samesignal-combination process (as discussed in the Intro-duction to this paper) The close spacing of theelements in our stimuli and their extension into theperiphery would lead us to expect the individualelements to be crowded by each other Under theaccount provided by Dakin et al (2009) it is suggestedthat crowding only limits performance in tasks in whichthere is little orientation variability (eg a finediscrimination task) whereas for a signal-in-noise tasksuch as ours performance should be limited by thenumber of pooled samples If crowding were a limit onattentional resolution (Strasburger 2005) howeverthen we would expect it to affect the ability of ourobservers to segregate potential signal and noiseelements possibly explaining the elevated thresholds inthe frac142708 noise check condition In addition it is notpossible from our study to determine whether theobservers were pooling all of the available samples inthe stimulus or if they were making their decisionsbased on a subset of the local samples in the display (aswas found by Dakin 2001)

It is not entirely clear how the results presented herecan be reconciled with those of Husk et al (2012) whofound no summation with increasing signal area forsimilar stimuli The main difference between the stimuli

used in the two studies is that this study used theBattenberg summation paradigm whereas Husk et alincreased the signal area of their stimuli by increasingdiameter It is possible that a combination of decreas-ing sensitivity for the local orientation-discriminationtask and increasing the mandatory summation regionsize with eccentricity might flatten the threshold versusarea functions This question shall be addressed infuture work which will combine the Battenbergstimulus paradigm used here with a conventional area-summation design comparing stimuli of differentdiameters Performing this experiment at a variety ofspatial frequencies will also allow us to determinewhether the size of the mandatory integration region islinked to the scale of the elements that are beingpooled Previous studies that have investigated theprocessing of visual texture would lead us to predictthis to be the case (Kingdom Keeble amp Moulden1995 Kingdom amp Keeble 1999) however if themandatory pooling were related to crowding we wouldexpect its extent to be independent of the scale of thestimulus (Levi Hariharan amp Klein 2002)

It is noteworthy that the results presented in Figure2c and d bear a resemblance to those found in thecontrast Battenberg study by Meese (2010) with short-range linear summation followed by long-range square-law summation The explanation for the square-lawsummation differs however between the two studiesFor the orientation result presented here this nonlinearsummation is explained by a flexible pooling strategythat segregates out the input from the nonsignal regionsof the display whereas in the contrast study by Meesethe output from those regions was included with thatfrom the signal regions in the global combinationprocess and the nonlinear summation was accountedfor by a square-law transducer

The nature of the limiting internal noise

Figure 6 shows three possible locations for thelimiting noise in the modeling for this study (NearlyNmid or Nlate) The similarity of the results from thenoise check and blank check conditions indicate thatthe limiting noise is not driven by the response toindividual elements Instead they suggest that a level ofinternal noise is pooled that is proportional to thenumber of locations being monitored which includesblank locations that are being integrated at the firststage of the model Jones et al (2003) included lsquolsquolatenoisersquorsquo when modeling data from an orientationcoherence experiment that used filtered noise as stimuliIn the modeling for that study the noise was constantfor different signal areas (this would be Nlate in Figure6) A model based on dominant noise at Nlate wouldmake the same prediction as the SA model shown in

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 12

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

Figure 3a for both the noise check and blank checkconditions Such a model would not explain the resultspresented here as the improvement in performance seenfor the medium check size stimuli requires that theobserver is able to segregate out the limiting noise inirrelevant regions from the second combination stageIf dominant noise is contributed from each monitored

location then this could be performed by weighting thelocal outputs according to a template w as shown inFigure 6

In the internal noise model developed in this study(see Figure 4) the noise was implemented at eachpooled location after the initial filtering stage (Nearly)however due to the mandatory local combination at

Figure 6 Diagram of the TN model This diagram shows how the lsquolsquoverticalrsquorsquo response to an example stimulus (which includes blank

spaces in stimulus region n) is determined by filtering with a vertical filter element mandatory local summation and then global

summation of the local outputs (from stimulus regions 1 to n) weighted by the expected signal-to-noise ratio at each location (w) For

ease of presentation the local summation region shown is 1 middot 4 elements (our results suggest 3 middot 3) and only the responses to the

first and last stimulus regions are presented The lsquolsquohorizontalrsquorsquo response would be calculated in an identical manner except with a

horizontal filter element at the convolution stage Nearly Nmid and Nlate show three possible locations for the limiting internal noise

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 13

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

the first combination stage this is equivalent to addingnoise to the combined local outputs (Nmid) From theresults of this study it is not possible to determinewhether the limiting noise should be Nearly or Nmid inFigure 6 however Nmid seems more plausible as thelevel of early noise needed to exceed the external noiseintroduced by the randomly oriented elements in thenoise check stimuli would be very high In the best-fitting prediction to the noise check data (TA1) thestandard deviation of the early noise would be 85 ofthe mean response of the local detector to its idealstimulus in the best-fitting prediction to the blankcheck data (TA2) prediction it would be even higher(147) There is also the possibility that the limitinginternal noise could be multiplicative rather than theadditive noise implemented in the modeling here Weshall address this question through the use of anequivalent noise paradigm in future work

Keywords orientation summation integration tex-ture perception computational modeling

Acknowledgments

This work was supported by a grant from theNatural Sciences and Engineering Research Council(Canada) awarded to Robert Hess (46528-11) and agrant from the Engineering and Physical SciencesResearch Council (UK) awarded to Tim Meese andMark Georgeson (EPH0000381) The authors wouldlike to thank three anonymous reviewers for theirhelpful criticisms and suggestions

Commercial relationships noneCorresponding author Alex S BaldwinEmail alexsbaldwingooglemailcomAddress School of Life and Health Sciences AstonUniversity Birmingham UK

References

Baker D H amp Meese T S 2011 Contrastintegration over area is extensive A three-stagemodel of spatial summation Journal of Vision11(14)14 1ndash16 httpwwwjournalofvisionorgcontent111414 doi101167111414 [PubMed][Article]

Brainard D H 1997 The Psychophysics ToolboxSpatial Vision 10(4) 433ndash436

Dakin S C 2001 Information limit on the spatialintegration of local orientation signals Journal ofthe Optical Society of America A 18(5) 1016ndash1026

Dakin S C Bex P J Cass J R amp Watt R J 2009Dissociable effects of attention and crowding onorientation averaging Journal of Vision 9(11)281ndash16 httpwwwjournalofvisionorgcontent91128 doi10116791128 [PubMed] [Article]

Dakin S C amp Watt R J 1997 The computation oforientation statistics from visual texture VisionResearch 37(22) 3181ndash3192

De Valois R L amp De Valois K K 1990 Striatecortex In Spatial Vision (pp 94ndash146) Oxford UKOxford University Press

Husk J S Huang P-C amp Hess R F 2012Orientation coherence sensitivity Journal of Vision126(18) 1ndash15 httpwwwjournalofvisionorgcontent12618 doi10116712618 [PubMed][Article]

Jones D G Anderson N D amp Murphy K M 2003Orientation discrimination in visual noise usingglobal and local stimuli Vision Research 43 1223ndash1233

Kingdom F A Keeble D amp Moulden B 1995Sensitivity to orientation modulation in micro-pattern-based textures Vision Research 35(1) 79ndash91

Kingdom F A amp Keeble D R 1999 On themechanism for scale invariance in orientation-defined textures Vision Research 39(8) 1477ndash1489

Kleiner M Brainard D H amp Pelli D G 2007Whatrsquos new in Psychtoolbox-3 Perception 36(ECVP Abstract Supplement)

Landy M S amp Graham N 2004 Visual perceptionof texture In L M Chalupa amp J S Werner (Eds)The visual neurosciences (pp 1106ndash1118) Cam-bridge MA MIT Press

Levi D M Hariharan S amp Klein S A 2002Suppressive and facilitatory spatial interactions inperipheral vision Peripheral crowding is neithersize invariant nor simple contrast masking Journalof Vision 2 167ndash177 httpwwwjournalofvisionorgcontent223 doi101167223 [PubMed][Article]

Marr D 1982 Vision San Francisco W H Freemanand Company

Meese T S 2010 Spatially extensive summation ofcontrast energy is revealed by contrast detection ofmicro-pattern textures Journal of Vision 10(8)141ndash21 httpwwwjournalofvisionorgcontent10814 doi10116710814 [PubMed] [Article]

Meese T S amp Summers R J 2007 Area summationin human vision at and above detection thresholdProceedings of the Royal Society B 274(1627)2891ndash2900

Nichols M J amp Newsome W T 2002 Middle

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 14

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

Two were naıve undergraduates (LFE and VRP) Allhad either normal or corrected-to-normal visionObservers DHB RJS and SAW were tested on the firstequipment setup described above Observers JSHLFE and VRP were tested on the second equipmentsetup Observer ASB was tested on the first setup forthe noise check and on the second setup for the blankcheck conditions The study was conducted in accor-dance with the Declaration of Helsinki

Analysis

Data from the horizontal and vertical staircases werecombined into a single psychometric function for eachrepetition condition and observer This was then fittedby a cumulative normal function using Palamedes(Prins amp Kingdom 2009) The fitted function gave theprobability of responding lsquolsquohorizontalrsquorsquo to either avertical stimulus (plotted as negative coherence) or ahorizontal stimulus (plotted as positive coherence) Thecoherence level at which the function reached P(lsquolsquoHor-izontalrsquorsquo) frac14 05 gave the bias for the observercategorizing a stimulus as horizontal rather thanvertical (the distributions of the biases found for eachobserver are presented in Appendix A) and theorientation identification threshold could be calculatedas the difference between the coherence level at thispoint and that at which P(lsquolsquoHorizontalrsquorsquo) frac14 075

Results

Noise check

The orientation identification thresholds averagedacross the seven observers are shown in Figure 2 Thethree rows present the data plotted as the expectednumber of signal elements in the stimulus at threshold(Figure 2a and b) the threshold probability of theelements in the signal region assuming the targetorientation (Figure 2c and d) which is equivalent to theexpected proportion of signal elements within the signalregions at threshold (hereafter termed the lsquolsquocoherencethresholdrsquorsquo) and the expected proportion of signalelements across the entire stimulus at threshold (Figure2e and f) In subsequent figures the metric shown inFigure 2c and d is used with this coherence thresholdfor each checked stimulus plotted as a multiple relativeto that for the full condition (which had a coherencethreshold of approximately 10)

Figure 2c presents the coherence thresholds for thenoise check condition For smaller check sizes (1 to 3)the thresholds for the checked stimuli were approxi-mately double that for the full stimulus This means

that around the same total number of signal elementsacross the entire stimulus were required to reach thethreshold performance level in each case (see Figure2a) For medium sizes (5 to 9) the threshold elevationdecreased to a factor of

ffiffiffi2p

This is consistent with astrategy that uses information from potential signalregions but ignores irrelevant (noise-only) regions asthe noise resulting from the combination of multipleequally noisy samples is proportional to the square rootof the number of samples combined (a simple lsquolsquosum ofvariancesrsquorsquo rule) As the stimuli were blocked by checksize and modulator phase the observer could predictin each trial which areas of the display were potentialsignal regions and which would contain only irrelevantnoise

One of the purposes of the Battenberg stimulusdesign is to reduce the effect of visual field inhomo-geneities in sensitivity on the measurement of areasummation (Meese amp Summers 2007 Meese 2010)For stimuli with large check sizes relative to theirextent however these effects will return For thelargest (15) check size in Figure 2c performancediverged dependent on whether the stimulus was in the frac14 908 (foveal signal peripheral noise) or frac14 2708(foveal noise peripheral signal) phase Coherencethresholds were almost as low for the frac14 908 stimulusas they were for the full stimulus This means that theobservers required around half as many signalelements across the entire stimulus when most of thoseelements were presented in the center of the display(Figure 2a) For the frac14 2708 stimulus coherencethresholds were approximately double that of the fullstimulus Therefore observers required the samenumber of signal elements in the largest frac14 2708stimulus as they did in the full stimulus (because thefull stimulus has approximately twice the checkedstimulusrsquos signal area) This behavior would beconsistent either with a relative insensitivity for thistask in the periphery or a failure in segregating thenoise present in the center of the display (this isdiscussed further below)

Blank check

Figure 2d shows the coherence thresholds for theblank check condition For the smaller check sizes inthe blank check condition coherence thresholdsincrease to approximately double that for the fullstimulus in a similar manner to that seen in the noisecheck condition This is unexpected as the predictedcoherence threshold for this condition based on thestimulus properties alone would be a factor of

ffiffiffi2p

above that for the full condition (there are no noise-only elements so the threshold would be proportionalto the square root of the number of elements in the

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 5

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

display the sum of variances rule again) These resultssuggest that the performance-limiting noise for thenoise check task does not arise from the responses tothe interstitial noise elements otherwise the removal ofthese elements in the blank check condition wouldresult in a decrease in threshold On the other handhowever the noise cannot be lsquolsquolatersquorsquo and constant

across conditions as this would predict the same

performance level for all of the checked stimuli

Instead these results suggest that observers are

mandatorily integrating internal noise from blank

display regions (or are limited by a noise source that is

proportional to the monitored area) for the smaller

Figure 2 Thresholds from the identification task (averaged over seven observers) shown in three different ways The results are

plotted here as the expected number of signal elements in the stimulus at threshold (andashb) the expected percentage of signal

elements in the potential signal regions at threshold (cndashd) and the expected percentage of signal elements in the entire stimulus at

threshold (endashf) The metric used in other results figures in this paper is that in c and d with these lsquolsquocoherence thresholdsrsquorsquo expressedas multiples relative to that of the full (F) stimulus Error bars show 61 standard error here and in all other graphs

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 6

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

check sizes but are able to exclude this noise for thelarger check sizes

The similarities in the noise check and blank checkdata shown in Figure 2c and d can be confirmed bylooking ahead to Figure 5 which compares thethresholds from these conditions directly by showingthe amount of threshold elevation caused by thepresence of the noise checks Up to the largest (15)

check size thresholds for the frac14 908 stimuli are similar(threshold elevation factor of approximately one)suggesting that the pooling of samples (and segregationof noise) in these two conditions is similar For the frac142708 stimuli however threshold elevation increaseswith check size Performance for the largest (15) frac142708 stimulus in the blank check condition is a factor offfiffiffi

2p

better than that in the noise check condition

Figure 3 Coherence thresholds for the checked stimuli expressed as multiples of the full stimulus threshold plotted with predictions

from the SA (andashb) SI (cndashd) and TS (endashf) models Each row shows the same averaged data replotted from Figure 2c and d The only

fitted parameter was k the size of the pooling region in the TS model (3 middot 3) This fitting was performed by hand (Appendix B) RMS

errors between the model predictions and the data are shown in dB

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 7

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

meaning that at least part of the deficit for detecting

that stimulus was due to inefficient pooling of noise in

the noise check condition rather than a relative

insensitivity for performing the task in the periphery

(eg it may be more difficult for the observer to keep

track of the signal and nonsignal regions in the frac142708

noise check condition)

Modeling

Monte Carlo simulations

A set of models was developed to investigate thepooling strategy used by the observers The initial aim

Figure 4 Coherence thresholds plotted with predictions from the TN model fitted to the blank check data (andashb) and from the TA

model with the best-fitting parameters for the noise check data (TA1 cndashd) and the best-fitting parameters for the blank check data

(TA2 endashf) Each row shows the same averaged data replotted from Figure 2c and d The fitted parameters were the size of the pooling

region (k) the standard deviation of the internal noise (rint) and the size of the integration aperture (a)

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 8

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

of the modeling was to explain the surprising equiva-lence between the smallest check sizes in the noise checkand blank check conditions (see Figure 2c and d)which would not be expected if the source of thelimiting noise was in the responses to the individualmicropattern elements (as the additional randomlyoriented elements in the noise check condition wouldcause thresholds for those stimuli to be higher) Modelpredictions were obtained using stochastic MonteCarlo methods A set of model observers was developedwith different pooling strategies in MATLAB and runthrough 2000 simulated trials per stimulus level of alsquolsquomethod of constant stimulirsquorsquo version of the experi-ment The coherence thresholds for each modelobserver were calculated using the simulated data andexpressed as multiples of the full stimulus thresholdallowing them to be compared to the human resultswithout the need for fitting Three fitted parameters(the size of the local pooling regions k the standarddeviation of the internal noise rint and the size of theglobal pooling region a) were then added in order todevelop a model that provided a close account of thedata These parameters were each fitted by hand (seeAppendix B)

The filter-maxing combination processes

Several combination processes for the calculation ofglobal orientation from individual local samples havebeen suggested previously Here we implement thestrongest candidate for this task in which the observer

selects the orientation of the most strongly activatedoriented filter (after Jones et al 2003 Husk et al2012) This choice is not crucial to our findingshowever and a set of models developed using a vector-averaging combination process made similar predic-tions (Appendix C) In our filter-maxing model thestimulus is first filtered at the two potential targetorientations The filter elements are a pair of log-Gaborelements with the same tuning properties as those usedin the generation of the stimuli (spatial frequencybandwidth of 16 octaves and orientation bandwidth of6258) These bandwidths are typical of those usedpreviously in the literature to model simple cellresponses and compatible with those found in neuro-physiological investigations (De Valois amp De Valois1990 Meese 2010) As our filter elements wereidentical to the target elements in our stimuli they alsobehaved as the ideal detector for those elements Thefilter outputs are rectified and passed to the poolingstage In the pooling stage the outputs can be weightedaccording to their expected signal-to-noise ratio (de-pendent on the pooling strategy in operation seebelow) The weighted filter responses are then summedover the image for each orientation and these valuesare compared to each other The model observer picksthe orientation with the greater filter response

Pooling strategies

The simplest pooling strategy considered is the lsquolsquosumallrsquorsquo (SA) model in which the observer combines

Figure 5 Threshold elevation introduced by the noise check elements calculated from the data in Figure 2c and d as the thresholds

from the noise check condition divided by the thresholds from the blank check condition The predicted threshold elevations from

each of the models shown in Figures 3 and 4 are also plotted here Only the SA and TS models predict any systematic elevation effect

and neither of those match the pattern seen in the data

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 9

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

information from every element in the stimulus with anequal weighting regardless of whether it is a noise orpotential signal element The model with this strategypredicts that within each condition (noise check andblank check) there should be no effect of check size ormodulator phase The predicted coherence thresholdfor the noise check stimuli is approximately twice thatfor the full stimulus (Figure 3a) which is equivalent torequiring the same threshold proportion of coherentsignal elements across the whole stimulus For theblank check stimuli (Figure 3b) the predicted thresholdelevation is a factor of

ffiffiffi2p

as there are no interstitialnoise elements to limit performance in that conditionThese predictions capture the performance for thesmall check sizes (1 to 3) in the noise check conditionthe medium check sizes (5 to 9) in the blank checkcondition and the largest frac14 2708 size (15) in both thenoise check and blank check conditions (see Figure 3aand b) The predictions fail however to describe thegeneral form of either set of results which both featurea transition from greater to lesser summation as thecheck size increases

The lsquolsquosum ideallyrsquorsquo (SI) strategy involves the combi-nation of orientation information only from potentialsignal elements This is equivalent to weighting theelements according to their expected signal-to-noiseratio as the responses to the elements in the noiseregions provide zero signal Models with this strategypredict threshold elevation of a factor of approximatelyffiffiffi

2p

for both of the checked conditions (Figure 3c andd) which is identical to the prediction for the blankcheck stimuli with the SA model For this reason theresults in the blank check condition which were welldescribed by SA models are fit just as well by the SImodels In addition the SI models predict the

ffiffiffi2p

summation for the medium check sizes in the noisecheck condition However this model also fails tocapture the general form of the human data for eitherstimulus type

Two-stage hybrid models

The fact that the simpler candidate models featuringthe two different pooling strategies (SA and SI) eachpredicted performance for different subsets of theresults suggested that a more complete account couldbe provided by a model that combined their twobehaviors In the lsquolsquotwo-stage (TS) hybrid modelsrsquorsquomandatory local combination over a k middot k region(behaving like the SA model) is followed by flexiblepooling of the outputs from those regions weighted bytheir expected signal-to-noise ratios which wereapplied as a template (as in the SI model) The size ofthe local pooling region (k) was the only fittedparameter in this model The best fit to the noise check

data was provided by a k of 3 although the fit providedby a k of 2 was only marginally worse (see Appendix BFigure A2) In the noise check condition this modelpredicts an initial doubling of the coherence thresholdfor the small check sizes followed by an improvementin performance to approach a factor of

ffiffiffi2p

for themedium and large check sizes (see Figure 3e) Thiscaptures the performance for all but the largest (15)check sizes In the blank check condition (Figure 3f)the predictions are once again the same as for modelswith the SA and SI strategies

Internal noise

The SA SI and TS models all make predictions thatare identical to each other for the blank checkcondition This is because there are no interstitial noiseelements in that condition that can be inappropriatelypooled to elevate the coherence threshold The resultshowever show a doubling of threshold for the smallcheck sizes that is the same in the noise check and blankcheck conditions This is a larger performance deficitthan can be accounted for by any of the three models(which all predict a

ffiffiffi2p

threshold increase for the blankcheck condition) A two-stage hybrid model featuringadditive Gaussian internal noise at each location (TN)does however predict similar performance for theblank check conditions as for the noise check condi-tions (see Figure 4a and b) The internal noise in themodel is added after the rectification stage (meaningthat the noisy local outputs can be negative) Becauseof the mandatory combination rule at the first poolingstage this early noise model is equivalent to a model inwhich performance is limited by noise affecting theoutput of the first pooling stage

The standard deviation of the internal noise (rint) isnow an additional parameter in the model It isexpressed as a percentage of the summed element-wisefilter response to a matched log-Gabor element (ierelative to the maximum local filter output) As rint isincreased from zero it initially has the effect ofincreasing thresholds for the 1 middot 1 blank checkcondition to bring them in line with those from thenoise check condition Once these have becomeequivalent higher values of rint serve mainly to increaseor decrease the sensitivity for all conditions Calculat-ing the relative coherence thresholds from the output ofthe model therefore largely factors out the effects ofthis model parameter once rint is sufficiently high (seeAppendix B) Figure 4a and b shows the TN modelprediction with the best-fitting k and rint values for theblank check data (note that the best-fitting k for theblank check data is 2 whereas kfrac14 3 provided amarginally better fit to the noise check data as shown inFigure A2) For both the noise check and the blank

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 10

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

check conditions the TN model predicts an initialtwofold summation for the 1 middot 1 check size which thendecreases to approximately

ffiffiffi2p

as the check sizeincreases The predictions from this model account forthe average human thresholds in the noise checkcondition for all but the largest check size (Figure 4a)In the blank check condition (Figure 4b) thresholds arelower than those predicted by the model for themedium (9) and largest (15) frac14 908 condition but forall other conditions the human behavior is captured bythe model

Maximum integration aperture

In the TN model there are several reasons whycoherence thresholds could be elevated when signal ispresented only in the periphery (compare frac14 908 and2708 thresholds for the largest check size stimulus inFigure 4) The simplest would be if the observer wereonly able to pool information from elements in thecenter of the display This was tested using a lsquolsquotwo-stagehybrid model with internal noise and maximumintegration aperturersquorsquo (TA) in which the modelobserver only had access to information from elementsthat were within a central a middot a element squareaperture (equivalent to a degrees or 4a carrier cycles)This would also be equivalent to a model that featureda step-edge decline in sensitivity at this eccentricityMore complex accounts of the effect of visual fieldeccentricity are not explored here as our stimuli weredesigned to factor out these effects when possiblemeaning that any explanation we might provide wouldnot be well constrained by our data For examplefurther summation may be achieved by probabilitysummation between multiple integration apertures atdifferent locations across the visual field (resulting in amodel similar to that proposed for the area summationof contrast by Baker amp Meese 2011)

In order to show which features of our results the TAmodel could and could not account for two separatefits were performed to determine the optimal set ofparameters for the noise check (TA1) and the blankcheck (TA2) data The TA1 prediction is shown inFigure 4c and d The best-fitting integration aperturesize was 18 middot 18 degrees (compared to other squareapertures with integer dimensions see Appendix B)The TA1 model prediction is similar to that made bythe TN model (for the same kernel size see AppendixB) for all conditions except for the largest check size(15) For the 15 middot 15 check stimuli the predictedthresholds for the frac14 908 stimulus are reduced (whenexpressed relative to the full condition threshold) andthe thresholds for the frac14 2708 stimulus are elevatedThis model prediction provides a good fit to the noisecheck data (RMSefrac14 054 dB) and for the smaller check

sizes (1ndash3) in the blank check condition For the largercheck sizes in the blank check condition however theTA1 model prediction systematically underestimatesthe sensitivity of the human observers (resulting in arelatively large RMSe of 178 dB)

Fitting the aperture model to the blank checkcondition produces the TA2 prediction shown in Figure4e and f As in the TN model the best-fitting k for theblank check condition was 2 (as opposed to the valuefound from the TA1 fit to the noise check data whichfavored a k of 3) The best-fitting aperture size (a) was19 middot 19 degrees (18 wider than the size found by fittingto the noise check data) As would be expected thisversion of the model provides an inferior fit to the noisecheck data compared to the TA1 prediction (116 dB vs054 dB) underestimating both the threshold for the 3middot 3 check condition and the amount of separationbetween the two modulator phases at the largest checksize In the blank check condition although the qualityof the fit is improved from the TA1 prediction (112 vs178 dB) sensitivity is still underestimated for the larger(9ndash15) check sizes This indicates that there is nocombination of parameters for this model architecturethat can capture the performance for those conditions

Residual effects of check condition andmodulator phase

Figure 5 replots the data from Figure 2c and dshowing directly the threshold elevation effect that theinterstitial noise elements in the noise check conditionhave Also shown in Figure 5 are the thresholdelevation predictions from each of the six modelvariants presented earlier All models except SA and TSpredict no threshold elevation from the noise checks ineither modulator phase For the frac14 908 modulatorphase the data agree with this prediction with the ratiobetween the noise check and blank check thresholdsremaining close to one for all check sizes For the frac142708 modulator phase there is no threshold elevationintroduced by the noise checks at the smallest (1) checksize but as the check size increases the noise checkshave the effect of raising thresholds relative to theblank check condition up to a maximum factor of

ffiffiffi2p

for the largest (15) check size Only the SA and TSmodels have their performance limited by the intersti-tial randomly oriented elements in the noise checkstimuli and therefore predict threshold elevation in thenoise check condition The pattern of elevationpredicted however does not match that seen in the frac142708 data The key difference is that the data show athreshold elevation effect that both increases withcheck size and is dependent on the modulator phasewhereas none of the models investigated here make thisprediction One possible explanation for this effect

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 11

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

would be an increase in the size of the mandatory localintegration region with eccentricity (linking this to theexplanation of crowding provided by Parkes LundAngelucci Solomon amp Morgan 2001)

Discussion

Orientation integration is a noisy two-stageprocess

The results of this study suggest that the combina-tion of orientation information over space is a noisytwo-stage process (Figure 6) The key results reportedhere can be accounted for by a model that performsmandatory local integration affected by internal noiseat each location followed by flexible pooling over theoutputs from those regions (for the vector-averagingversion of the model these would seem equivalent tothe involuntary and voluntary averaging discussed byDakin Bex Cass amp Watt 2009) This account is inagreement with previous studies that have found lowerthresholds for stimuli with a greater signal area (Dakin2001 Jones et al 2003) The effect found for thearrangement of the elements in the display contradictsthe flexibility attributed to the pooling of local samplesby Dakin (2001) although it is possible that thedifferences in terms of the task set to the observer(signal in noise here vs fine discrimination in Dakin)would mean that they did not investigate the samesignal-combination process (as discussed in the Intro-duction to this paper) The close spacing of theelements in our stimuli and their extension into theperiphery would lead us to expect the individualelements to be crowded by each other Under theaccount provided by Dakin et al (2009) it is suggestedthat crowding only limits performance in tasks in whichthere is little orientation variability (eg a finediscrimination task) whereas for a signal-in-noise tasksuch as ours performance should be limited by thenumber of pooled samples If crowding were a limit onattentional resolution (Strasburger 2005) howeverthen we would expect it to affect the ability of ourobservers to segregate potential signal and noiseelements possibly explaining the elevated thresholds inthe frac142708 noise check condition In addition it is notpossible from our study to determine whether theobservers were pooling all of the available samples inthe stimulus or if they were making their decisionsbased on a subset of the local samples in the display (aswas found by Dakin 2001)

It is not entirely clear how the results presented herecan be reconciled with those of Husk et al (2012) whofound no summation with increasing signal area forsimilar stimuli The main difference between the stimuli

used in the two studies is that this study used theBattenberg summation paradigm whereas Husk et alincreased the signal area of their stimuli by increasingdiameter It is possible that a combination of decreas-ing sensitivity for the local orientation-discriminationtask and increasing the mandatory summation regionsize with eccentricity might flatten the threshold versusarea functions This question shall be addressed infuture work which will combine the Battenbergstimulus paradigm used here with a conventional area-summation design comparing stimuli of differentdiameters Performing this experiment at a variety ofspatial frequencies will also allow us to determinewhether the size of the mandatory integration region islinked to the scale of the elements that are beingpooled Previous studies that have investigated theprocessing of visual texture would lead us to predictthis to be the case (Kingdom Keeble amp Moulden1995 Kingdom amp Keeble 1999) however if themandatory pooling were related to crowding we wouldexpect its extent to be independent of the scale of thestimulus (Levi Hariharan amp Klein 2002)

It is noteworthy that the results presented in Figure2c and d bear a resemblance to those found in thecontrast Battenberg study by Meese (2010) with short-range linear summation followed by long-range square-law summation The explanation for the square-lawsummation differs however between the two studiesFor the orientation result presented here this nonlinearsummation is explained by a flexible pooling strategythat segregates out the input from the nonsignal regionsof the display whereas in the contrast study by Meesethe output from those regions was included with thatfrom the signal regions in the global combinationprocess and the nonlinear summation was accountedfor by a square-law transducer

The nature of the limiting internal noise

Figure 6 shows three possible locations for thelimiting noise in the modeling for this study (NearlyNmid or Nlate) The similarity of the results from thenoise check and blank check conditions indicate thatthe limiting noise is not driven by the response toindividual elements Instead they suggest that a level ofinternal noise is pooled that is proportional to thenumber of locations being monitored which includesblank locations that are being integrated at the firststage of the model Jones et al (2003) included lsquolsquolatenoisersquorsquo when modeling data from an orientationcoherence experiment that used filtered noise as stimuliIn the modeling for that study the noise was constantfor different signal areas (this would be Nlate in Figure6) A model based on dominant noise at Nlate wouldmake the same prediction as the SA model shown in

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 12

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

Figure 3a for both the noise check and blank checkconditions Such a model would not explain the resultspresented here as the improvement in performance seenfor the medium check size stimuli requires that theobserver is able to segregate out the limiting noise inirrelevant regions from the second combination stageIf dominant noise is contributed from each monitored

location then this could be performed by weighting thelocal outputs according to a template w as shown inFigure 6

In the internal noise model developed in this study(see Figure 4) the noise was implemented at eachpooled location after the initial filtering stage (Nearly)however due to the mandatory local combination at

Figure 6 Diagram of the TN model This diagram shows how the lsquolsquoverticalrsquorsquo response to an example stimulus (which includes blank

spaces in stimulus region n) is determined by filtering with a vertical filter element mandatory local summation and then global

summation of the local outputs (from stimulus regions 1 to n) weighted by the expected signal-to-noise ratio at each location (w) For

ease of presentation the local summation region shown is 1 middot 4 elements (our results suggest 3 middot 3) and only the responses to the

first and last stimulus regions are presented The lsquolsquohorizontalrsquorsquo response would be calculated in an identical manner except with a

horizontal filter element at the convolution stage Nearly Nmid and Nlate show three possible locations for the limiting internal noise

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 13

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

the first combination stage this is equivalent to addingnoise to the combined local outputs (Nmid) From theresults of this study it is not possible to determinewhether the limiting noise should be Nearly or Nmid inFigure 6 however Nmid seems more plausible as thelevel of early noise needed to exceed the external noiseintroduced by the randomly oriented elements in thenoise check stimuli would be very high In the best-fitting prediction to the noise check data (TA1) thestandard deviation of the early noise would be 85 ofthe mean response of the local detector to its idealstimulus in the best-fitting prediction to the blankcheck data (TA2) prediction it would be even higher(147) There is also the possibility that the limitinginternal noise could be multiplicative rather than theadditive noise implemented in the modeling here Weshall address this question through the use of anequivalent noise paradigm in future work

Keywords orientation summation integration tex-ture perception computational modeling

Acknowledgments

This work was supported by a grant from theNatural Sciences and Engineering Research Council(Canada) awarded to Robert Hess (46528-11) and agrant from the Engineering and Physical SciencesResearch Council (UK) awarded to Tim Meese andMark Georgeson (EPH0000381) The authors wouldlike to thank three anonymous reviewers for theirhelpful criticisms and suggestions

Commercial relationships noneCorresponding author Alex S BaldwinEmail alexsbaldwingooglemailcomAddress School of Life and Health Sciences AstonUniversity Birmingham UK

References

Baker D H amp Meese T S 2011 Contrastintegration over area is extensive A three-stagemodel of spatial summation Journal of Vision11(14)14 1ndash16 httpwwwjournalofvisionorgcontent111414 doi101167111414 [PubMed][Article]

Brainard D H 1997 The Psychophysics ToolboxSpatial Vision 10(4) 433ndash436

Dakin S C 2001 Information limit on the spatialintegration of local orientation signals Journal ofthe Optical Society of America A 18(5) 1016ndash1026

Dakin S C Bex P J Cass J R amp Watt R J 2009Dissociable effects of attention and crowding onorientation averaging Journal of Vision 9(11)281ndash16 httpwwwjournalofvisionorgcontent91128 doi10116791128 [PubMed] [Article]

Dakin S C amp Watt R J 1997 The computation oforientation statistics from visual texture VisionResearch 37(22) 3181ndash3192

De Valois R L amp De Valois K K 1990 Striatecortex In Spatial Vision (pp 94ndash146) Oxford UKOxford University Press

Husk J S Huang P-C amp Hess R F 2012Orientation coherence sensitivity Journal of Vision126(18) 1ndash15 httpwwwjournalofvisionorgcontent12618 doi10116712618 [PubMed][Article]

Jones D G Anderson N D amp Murphy K M 2003Orientation discrimination in visual noise usingglobal and local stimuli Vision Research 43 1223ndash1233

Kingdom F A Keeble D amp Moulden B 1995Sensitivity to orientation modulation in micro-pattern-based textures Vision Research 35(1) 79ndash91

Kingdom F A amp Keeble D R 1999 On themechanism for scale invariance in orientation-defined textures Vision Research 39(8) 1477ndash1489

Kleiner M Brainard D H amp Pelli D G 2007Whatrsquos new in Psychtoolbox-3 Perception 36(ECVP Abstract Supplement)

Landy M S amp Graham N 2004 Visual perceptionof texture In L M Chalupa amp J S Werner (Eds)The visual neurosciences (pp 1106ndash1118) Cam-bridge MA MIT Press

Levi D M Hariharan S amp Klein S A 2002Suppressive and facilitatory spatial interactions inperipheral vision Peripheral crowding is neithersize invariant nor simple contrast masking Journalof Vision 2 167ndash177 httpwwwjournalofvisionorgcontent223 doi101167223 [PubMed][Article]

Marr D 1982 Vision San Francisco W H Freemanand Company

Meese T S 2010 Spatially extensive summation ofcontrast energy is revealed by contrast detection ofmicro-pattern textures Journal of Vision 10(8)141ndash21 httpwwwjournalofvisionorgcontent10814 doi10116710814 [PubMed] [Article]

Meese T S amp Summers R J 2007 Area summationin human vision at and above detection thresholdProceedings of the Royal Society B 274(1627)2891ndash2900

Nichols M J amp Newsome W T 2002 Middle

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 14

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

display the sum of variances rule again) These resultssuggest that the performance-limiting noise for thenoise check task does not arise from the responses tothe interstitial noise elements otherwise the removal ofthese elements in the blank check condition wouldresult in a decrease in threshold On the other handhowever the noise cannot be lsquolsquolatersquorsquo and constant

across conditions as this would predict the same

performance level for all of the checked stimuli

Instead these results suggest that observers are

mandatorily integrating internal noise from blank

display regions (or are limited by a noise source that is

proportional to the monitored area) for the smaller

Figure 2 Thresholds from the identification task (averaged over seven observers) shown in three different ways The results are

plotted here as the expected number of signal elements in the stimulus at threshold (andashb) the expected percentage of signal

elements in the potential signal regions at threshold (cndashd) and the expected percentage of signal elements in the entire stimulus at

threshold (endashf) The metric used in other results figures in this paper is that in c and d with these lsquolsquocoherence thresholdsrsquorsquo expressedas multiples relative to that of the full (F) stimulus Error bars show 61 standard error here and in all other graphs

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 6

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

check sizes but are able to exclude this noise for thelarger check sizes

The similarities in the noise check and blank checkdata shown in Figure 2c and d can be confirmed bylooking ahead to Figure 5 which compares thethresholds from these conditions directly by showingthe amount of threshold elevation caused by thepresence of the noise checks Up to the largest (15)

check size thresholds for the frac14 908 stimuli are similar(threshold elevation factor of approximately one)suggesting that the pooling of samples (and segregationof noise) in these two conditions is similar For the frac142708 stimuli however threshold elevation increaseswith check size Performance for the largest (15) frac142708 stimulus in the blank check condition is a factor offfiffiffi

2p

better than that in the noise check condition

Figure 3 Coherence thresholds for the checked stimuli expressed as multiples of the full stimulus threshold plotted with predictions

from the SA (andashb) SI (cndashd) and TS (endashf) models Each row shows the same averaged data replotted from Figure 2c and d The only

fitted parameter was k the size of the pooling region in the TS model (3 middot 3) This fitting was performed by hand (Appendix B) RMS

errors between the model predictions and the data are shown in dB

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 7

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

meaning that at least part of the deficit for detecting

that stimulus was due to inefficient pooling of noise in

the noise check condition rather than a relative

insensitivity for performing the task in the periphery

(eg it may be more difficult for the observer to keep

track of the signal and nonsignal regions in the frac142708

noise check condition)

Modeling

Monte Carlo simulations

A set of models was developed to investigate thepooling strategy used by the observers The initial aim

Figure 4 Coherence thresholds plotted with predictions from the TN model fitted to the blank check data (andashb) and from the TA

model with the best-fitting parameters for the noise check data (TA1 cndashd) and the best-fitting parameters for the blank check data

(TA2 endashf) Each row shows the same averaged data replotted from Figure 2c and d The fitted parameters were the size of the pooling

region (k) the standard deviation of the internal noise (rint) and the size of the integration aperture (a)

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 8

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

of the modeling was to explain the surprising equiva-lence between the smallest check sizes in the noise checkand blank check conditions (see Figure 2c and d)which would not be expected if the source of thelimiting noise was in the responses to the individualmicropattern elements (as the additional randomlyoriented elements in the noise check condition wouldcause thresholds for those stimuli to be higher) Modelpredictions were obtained using stochastic MonteCarlo methods A set of model observers was developedwith different pooling strategies in MATLAB and runthrough 2000 simulated trials per stimulus level of alsquolsquomethod of constant stimulirsquorsquo version of the experi-ment The coherence thresholds for each modelobserver were calculated using the simulated data andexpressed as multiples of the full stimulus thresholdallowing them to be compared to the human resultswithout the need for fitting Three fitted parameters(the size of the local pooling regions k the standarddeviation of the internal noise rint and the size of theglobal pooling region a) were then added in order todevelop a model that provided a close account of thedata These parameters were each fitted by hand (seeAppendix B)

The filter-maxing combination processes

Several combination processes for the calculation ofglobal orientation from individual local samples havebeen suggested previously Here we implement thestrongest candidate for this task in which the observer

selects the orientation of the most strongly activatedoriented filter (after Jones et al 2003 Husk et al2012) This choice is not crucial to our findingshowever and a set of models developed using a vector-averaging combination process made similar predic-tions (Appendix C) In our filter-maxing model thestimulus is first filtered at the two potential targetorientations The filter elements are a pair of log-Gaborelements with the same tuning properties as those usedin the generation of the stimuli (spatial frequencybandwidth of 16 octaves and orientation bandwidth of6258) These bandwidths are typical of those usedpreviously in the literature to model simple cellresponses and compatible with those found in neuro-physiological investigations (De Valois amp De Valois1990 Meese 2010) As our filter elements wereidentical to the target elements in our stimuli they alsobehaved as the ideal detector for those elements Thefilter outputs are rectified and passed to the poolingstage In the pooling stage the outputs can be weightedaccording to their expected signal-to-noise ratio (de-pendent on the pooling strategy in operation seebelow) The weighted filter responses are then summedover the image for each orientation and these valuesare compared to each other The model observer picksthe orientation with the greater filter response

Pooling strategies

The simplest pooling strategy considered is the lsquolsquosumallrsquorsquo (SA) model in which the observer combines

Figure 5 Threshold elevation introduced by the noise check elements calculated from the data in Figure 2c and d as the thresholds

from the noise check condition divided by the thresholds from the blank check condition The predicted threshold elevations from

each of the models shown in Figures 3 and 4 are also plotted here Only the SA and TS models predict any systematic elevation effect

and neither of those match the pattern seen in the data

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 9

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

information from every element in the stimulus with anequal weighting regardless of whether it is a noise orpotential signal element The model with this strategypredicts that within each condition (noise check andblank check) there should be no effect of check size ormodulator phase The predicted coherence thresholdfor the noise check stimuli is approximately twice thatfor the full stimulus (Figure 3a) which is equivalent torequiring the same threshold proportion of coherentsignal elements across the whole stimulus For theblank check stimuli (Figure 3b) the predicted thresholdelevation is a factor of

ffiffiffi2p

as there are no interstitialnoise elements to limit performance in that conditionThese predictions capture the performance for thesmall check sizes (1 to 3) in the noise check conditionthe medium check sizes (5 to 9) in the blank checkcondition and the largest frac14 2708 size (15) in both thenoise check and blank check conditions (see Figure 3aand b) The predictions fail however to describe thegeneral form of either set of results which both featurea transition from greater to lesser summation as thecheck size increases

The lsquolsquosum ideallyrsquorsquo (SI) strategy involves the combi-nation of orientation information only from potentialsignal elements This is equivalent to weighting theelements according to their expected signal-to-noiseratio as the responses to the elements in the noiseregions provide zero signal Models with this strategypredict threshold elevation of a factor of approximatelyffiffiffi

2p

for both of the checked conditions (Figure 3c andd) which is identical to the prediction for the blankcheck stimuli with the SA model For this reason theresults in the blank check condition which were welldescribed by SA models are fit just as well by the SImodels In addition the SI models predict the

ffiffiffi2p

summation for the medium check sizes in the noisecheck condition However this model also fails tocapture the general form of the human data for eitherstimulus type

Two-stage hybrid models

The fact that the simpler candidate models featuringthe two different pooling strategies (SA and SI) eachpredicted performance for different subsets of theresults suggested that a more complete account couldbe provided by a model that combined their twobehaviors In the lsquolsquotwo-stage (TS) hybrid modelsrsquorsquomandatory local combination over a k middot k region(behaving like the SA model) is followed by flexiblepooling of the outputs from those regions weighted bytheir expected signal-to-noise ratios which wereapplied as a template (as in the SI model) The size ofthe local pooling region (k) was the only fittedparameter in this model The best fit to the noise check

data was provided by a k of 3 although the fit providedby a k of 2 was only marginally worse (see Appendix BFigure A2) In the noise check condition this modelpredicts an initial doubling of the coherence thresholdfor the small check sizes followed by an improvementin performance to approach a factor of

ffiffiffi2p

for themedium and large check sizes (see Figure 3e) Thiscaptures the performance for all but the largest (15)check sizes In the blank check condition (Figure 3f)the predictions are once again the same as for modelswith the SA and SI strategies

Internal noise

The SA SI and TS models all make predictions thatare identical to each other for the blank checkcondition This is because there are no interstitial noiseelements in that condition that can be inappropriatelypooled to elevate the coherence threshold The resultshowever show a doubling of threshold for the smallcheck sizes that is the same in the noise check and blankcheck conditions This is a larger performance deficitthan can be accounted for by any of the three models(which all predict a

ffiffiffi2p

threshold increase for the blankcheck condition) A two-stage hybrid model featuringadditive Gaussian internal noise at each location (TN)does however predict similar performance for theblank check conditions as for the noise check condi-tions (see Figure 4a and b) The internal noise in themodel is added after the rectification stage (meaningthat the noisy local outputs can be negative) Becauseof the mandatory combination rule at the first poolingstage this early noise model is equivalent to a model inwhich performance is limited by noise affecting theoutput of the first pooling stage

The standard deviation of the internal noise (rint) isnow an additional parameter in the model It isexpressed as a percentage of the summed element-wisefilter response to a matched log-Gabor element (ierelative to the maximum local filter output) As rint isincreased from zero it initially has the effect ofincreasing thresholds for the 1 middot 1 blank checkcondition to bring them in line with those from thenoise check condition Once these have becomeequivalent higher values of rint serve mainly to increaseor decrease the sensitivity for all conditions Calculat-ing the relative coherence thresholds from the output ofthe model therefore largely factors out the effects ofthis model parameter once rint is sufficiently high (seeAppendix B) Figure 4a and b shows the TN modelprediction with the best-fitting k and rint values for theblank check data (note that the best-fitting k for theblank check data is 2 whereas kfrac14 3 provided amarginally better fit to the noise check data as shown inFigure A2) For both the noise check and the blank

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 10

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

check conditions the TN model predicts an initialtwofold summation for the 1 middot 1 check size which thendecreases to approximately

ffiffiffi2p

as the check sizeincreases The predictions from this model account forthe average human thresholds in the noise checkcondition for all but the largest check size (Figure 4a)In the blank check condition (Figure 4b) thresholds arelower than those predicted by the model for themedium (9) and largest (15) frac14 908 condition but forall other conditions the human behavior is captured bythe model

Maximum integration aperture

In the TN model there are several reasons whycoherence thresholds could be elevated when signal ispresented only in the periphery (compare frac14 908 and2708 thresholds for the largest check size stimulus inFigure 4) The simplest would be if the observer wereonly able to pool information from elements in thecenter of the display This was tested using a lsquolsquotwo-stagehybrid model with internal noise and maximumintegration aperturersquorsquo (TA) in which the modelobserver only had access to information from elementsthat were within a central a middot a element squareaperture (equivalent to a degrees or 4a carrier cycles)This would also be equivalent to a model that featureda step-edge decline in sensitivity at this eccentricityMore complex accounts of the effect of visual fieldeccentricity are not explored here as our stimuli weredesigned to factor out these effects when possiblemeaning that any explanation we might provide wouldnot be well constrained by our data For examplefurther summation may be achieved by probabilitysummation between multiple integration apertures atdifferent locations across the visual field (resulting in amodel similar to that proposed for the area summationof contrast by Baker amp Meese 2011)

In order to show which features of our results the TAmodel could and could not account for two separatefits were performed to determine the optimal set ofparameters for the noise check (TA1) and the blankcheck (TA2) data The TA1 prediction is shown inFigure 4c and d The best-fitting integration aperturesize was 18 middot 18 degrees (compared to other squareapertures with integer dimensions see Appendix B)The TA1 model prediction is similar to that made bythe TN model (for the same kernel size see AppendixB) for all conditions except for the largest check size(15) For the 15 middot 15 check stimuli the predictedthresholds for the frac14 908 stimulus are reduced (whenexpressed relative to the full condition threshold) andthe thresholds for the frac14 2708 stimulus are elevatedThis model prediction provides a good fit to the noisecheck data (RMSefrac14 054 dB) and for the smaller check

sizes (1ndash3) in the blank check condition For the largercheck sizes in the blank check condition however theTA1 model prediction systematically underestimatesthe sensitivity of the human observers (resulting in arelatively large RMSe of 178 dB)

Fitting the aperture model to the blank checkcondition produces the TA2 prediction shown in Figure4e and f As in the TN model the best-fitting k for theblank check condition was 2 (as opposed to the valuefound from the TA1 fit to the noise check data whichfavored a k of 3) The best-fitting aperture size (a) was19 middot 19 degrees (18 wider than the size found by fittingto the noise check data) As would be expected thisversion of the model provides an inferior fit to the noisecheck data compared to the TA1 prediction (116 dB vs054 dB) underestimating both the threshold for the 3middot 3 check condition and the amount of separationbetween the two modulator phases at the largest checksize In the blank check condition although the qualityof the fit is improved from the TA1 prediction (112 vs178 dB) sensitivity is still underestimated for the larger(9ndash15) check sizes This indicates that there is nocombination of parameters for this model architecturethat can capture the performance for those conditions

Residual effects of check condition andmodulator phase

Figure 5 replots the data from Figure 2c and dshowing directly the threshold elevation effect that theinterstitial noise elements in the noise check conditionhave Also shown in Figure 5 are the thresholdelevation predictions from each of the six modelvariants presented earlier All models except SA and TSpredict no threshold elevation from the noise checks ineither modulator phase For the frac14 908 modulatorphase the data agree with this prediction with the ratiobetween the noise check and blank check thresholdsremaining close to one for all check sizes For the frac142708 modulator phase there is no threshold elevationintroduced by the noise checks at the smallest (1) checksize but as the check size increases the noise checkshave the effect of raising thresholds relative to theblank check condition up to a maximum factor of

ffiffiffi2p

for the largest (15) check size Only the SA and TSmodels have their performance limited by the intersti-tial randomly oriented elements in the noise checkstimuli and therefore predict threshold elevation in thenoise check condition The pattern of elevationpredicted however does not match that seen in the frac142708 data The key difference is that the data show athreshold elevation effect that both increases withcheck size and is dependent on the modulator phasewhereas none of the models investigated here make thisprediction One possible explanation for this effect

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 11

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

would be an increase in the size of the mandatory localintegration region with eccentricity (linking this to theexplanation of crowding provided by Parkes LundAngelucci Solomon amp Morgan 2001)

Discussion

Orientation integration is a noisy two-stageprocess

The results of this study suggest that the combina-tion of orientation information over space is a noisytwo-stage process (Figure 6) The key results reportedhere can be accounted for by a model that performsmandatory local integration affected by internal noiseat each location followed by flexible pooling over theoutputs from those regions (for the vector-averagingversion of the model these would seem equivalent tothe involuntary and voluntary averaging discussed byDakin Bex Cass amp Watt 2009) This account is inagreement with previous studies that have found lowerthresholds for stimuli with a greater signal area (Dakin2001 Jones et al 2003) The effect found for thearrangement of the elements in the display contradictsthe flexibility attributed to the pooling of local samplesby Dakin (2001) although it is possible that thedifferences in terms of the task set to the observer(signal in noise here vs fine discrimination in Dakin)would mean that they did not investigate the samesignal-combination process (as discussed in the Intro-duction to this paper) The close spacing of theelements in our stimuli and their extension into theperiphery would lead us to expect the individualelements to be crowded by each other Under theaccount provided by Dakin et al (2009) it is suggestedthat crowding only limits performance in tasks in whichthere is little orientation variability (eg a finediscrimination task) whereas for a signal-in-noise tasksuch as ours performance should be limited by thenumber of pooled samples If crowding were a limit onattentional resolution (Strasburger 2005) howeverthen we would expect it to affect the ability of ourobservers to segregate potential signal and noiseelements possibly explaining the elevated thresholds inthe frac142708 noise check condition In addition it is notpossible from our study to determine whether theobservers were pooling all of the available samples inthe stimulus or if they were making their decisionsbased on a subset of the local samples in the display (aswas found by Dakin 2001)

It is not entirely clear how the results presented herecan be reconciled with those of Husk et al (2012) whofound no summation with increasing signal area forsimilar stimuli The main difference between the stimuli

used in the two studies is that this study used theBattenberg summation paradigm whereas Husk et alincreased the signal area of their stimuli by increasingdiameter It is possible that a combination of decreas-ing sensitivity for the local orientation-discriminationtask and increasing the mandatory summation regionsize with eccentricity might flatten the threshold versusarea functions This question shall be addressed infuture work which will combine the Battenbergstimulus paradigm used here with a conventional area-summation design comparing stimuli of differentdiameters Performing this experiment at a variety ofspatial frequencies will also allow us to determinewhether the size of the mandatory integration region islinked to the scale of the elements that are beingpooled Previous studies that have investigated theprocessing of visual texture would lead us to predictthis to be the case (Kingdom Keeble amp Moulden1995 Kingdom amp Keeble 1999) however if themandatory pooling were related to crowding we wouldexpect its extent to be independent of the scale of thestimulus (Levi Hariharan amp Klein 2002)

It is noteworthy that the results presented in Figure2c and d bear a resemblance to those found in thecontrast Battenberg study by Meese (2010) with short-range linear summation followed by long-range square-law summation The explanation for the square-lawsummation differs however between the two studiesFor the orientation result presented here this nonlinearsummation is explained by a flexible pooling strategythat segregates out the input from the nonsignal regionsof the display whereas in the contrast study by Meesethe output from those regions was included with thatfrom the signal regions in the global combinationprocess and the nonlinear summation was accountedfor by a square-law transducer

The nature of the limiting internal noise

Figure 6 shows three possible locations for thelimiting noise in the modeling for this study (NearlyNmid or Nlate) The similarity of the results from thenoise check and blank check conditions indicate thatthe limiting noise is not driven by the response toindividual elements Instead they suggest that a level ofinternal noise is pooled that is proportional to thenumber of locations being monitored which includesblank locations that are being integrated at the firststage of the model Jones et al (2003) included lsquolsquolatenoisersquorsquo when modeling data from an orientationcoherence experiment that used filtered noise as stimuliIn the modeling for that study the noise was constantfor different signal areas (this would be Nlate in Figure6) A model based on dominant noise at Nlate wouldmake the same prediction as the SA model shown in

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 12

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

Figure 3a for both the noise check and blank checkconditions Such a model would not explain the resultspresented here as the improvement in performance seenfor the medium check size stimuli requires that theobserver is able to segregate out the limiting noise inirrelevant regions from the second combination stageIf dominant noise is contributed from each monitored

location then this could be performed by weighting thelocal outputs according to a template w as shown inFigure 6

In the internal noise model developed in this study(see Figure 4) the noise was implemented at eachpooled location after the initial filtering stage (Nearly)however due to the mandatory local combination at

Figure 6 Diagram of the TN model This diagram shows how the lsquolsquoverticalrsquorsquo response to an example stimulus (which includes blank

spaces in stimulus region n) is determined by filtering with a vertical filter element mandatory local summation and then global

summation of the local outputs (from stimulus regions 1 to n) weighted by the expected signal-to-noise ratio at each location (w) For

ease of presentation the local summation region shown is 1 middot 4 elements (our results suggest 3 middot 3) and only the responses to the

first and last stimulus regions are presented The lsquolsquohorizontalrsquorsquo response would be calculated in an identical manner except with a

horizontal filter element at the convolution stage Nearly Nmid and Nlate show three possible locations for the limiting internal noise

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 13

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

the first combination stage this is equivalent to addingnoise to the combined local outputs (Nmid) From theresults of this study it is not possible to determinewhether the limiting noise should be Nearly or Nmid inFigure 6 however Nmid seems more plausible as thelevel of early noise needed to exceed the external noiseintroduced by the randomly oriented elements in thenoise check stimuli would be very high In the best-fitting prediction to the noise check data (TA1) thestandard deviation of the early noise would be 85 ofthe mean response of the local detector to its idealstimulus in the best-fitting prediction to the blankcheck data (TA2) prediction it would be even higher(147) There is also the possibility that the limitinginternal noise could be multiplicative rather than theadditive noise implemented in the modeling here Weshall address this question through the use of anequivalent noise paradigm in future work

Keywords orientation summation integration tex-ture perception computational modeling

Acknowledgments

This work was supported by a grant from theNatural Sciences and Engineering Research Council(Canada) awarded to Robert Hess (46528-11) and agrant from the Engineering and Physical SciencesResearch Council (UK) awarded to Tim Meese andMark Georgeson (EPH0000381) The authors wouldlike to thank three anonymous reviewers for theirhelpful criticisms and suggestions

Commercial relationships noneCorresponding author Alex S BaldwinEmail alexsbaldwingooglemailcomAddress School of Life and Health Sciences AstonUniversity Birmingham UK

References

Baker D H amp Meese T S 2011 Contrastintegration over area is extensive A three-stagemodel of spatial summation Journal of Vision11(14)14 1ndash16 httpwwwjournalofvisionorgcontent111414 doi101167111414 [PubMed][Article]

Brainard D H 1997 The Psychophysics ToolboxSpatial Vision 10(4) 433ndash436

Dakin S C 2001 Information limit on the spatialintegration of local orientation signals Journal ofthe Optical Society of America A 18(5) 1016ndash1026

Dakin S C Bex P J Cass J R amp Watt R J 2009Dissociable effects of attention and crowding onorientation averaging Journal of Vision 9(11)281ndash16 httpwwwjournalofvisionorgcontent91128 doi10116791128 [PubMed] [Article]

Dakin S C amp Watt R J 1997 The computation oforientation statistics from visual texture VisionResearch 37(22) 3181ndash3192

De Valois R L amp De Valois K K 1990 Striatecortex In Spatial Vision (pp 94ndash146) Oxford UKOxford University Press

Husk J S Huang P-C amp Hess R F 2012Orientation coherence sensitivity Journal of Vision126(18) 1ndash15 httpwwwjournalofvisionorgcontent12618 doi10116712618 [PubMed][Article]

Jones D G Anderson N D amp Murphy K M 2003Orientation discrimination in visual noise usingglobal and local stimuli Vision Research 43 1223ndash1233

Kingdom F A Keeble D amp Moulden B 1995Sensitivity to orientation modulation in micro-pattern-based textures Vision Research 35(1) 79ndash91

Kingdom F A amp Keeble D R 1999 On themechanism for scale invariance in orientation-defined textures Vision Research 39(8) 1477ndash1489

Kleiner M Brainard D H amp Pelli D G 2007Whatrsquos new in Psychtoolbox-3 Perception 36(ECVP Abstract Supplement)

Landy M S amp Graham N 2004 Visual perceptionof texture In L M Chalupa amp J S Werner (Eds)The visual neurosciences (pp 1106ndash1118) Cam-bridge MA MIT Press

Levi D M Hariharan S amp Klein S A 2002Suppressive and facilitatory spatial interactions inperipheral vision Peripheral crowding is neithersize invariant nor simple contrast masking Journalof Vision 2 167ndash177 httpwwwjournalofvisionorgcontent223 doi101167223 [PubMed][Article]

Marr D 1982 Vision San Francisco W H Freemanand Company

Meese T S 2010 Spatially extensive summation ofcontrast energy is revealed by contrast detection ofmicro-pattern textures Journal of Vision 10(8)141ndash21 httpwwwjournalofvisionorgcontent10814 doi10116710814 [PubMed] [Article]

Meese T S amp Summers R J 2007 Area summationin human vision at and above detection thresholdProceedings of the Royal Society B 274(1627)2891ndash2900

Nichols M J amp Newsome W T 2002 Middle

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 14

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

check sizes but are able to exclude this noise for thelarger check sizes

The similarities in the noise check and blank checkdata shown in Figure 2c and d can be confirmed bylooking ahead to Figure 5 which compares thethresholds from these conditions directly by showingthe amount of threshold elevation caused by thepresence of the noise checks Up to the largest (15)

check size thresholds for the frac14 908 stimuli are similar(threshold elevation factor of approximately one)suggesting that the pooling of samples (and segregationof noise) in these two conditions is similar For the frac142708 stimuli however threshold elevation increaseswith check size Performance for the largest (15) frac142708 stimulus in the blank check condition is a factor offfiffiffi

2p

better than that in the noise check condition

Figure 3 Coherence thresholds for the checked stimuli expressed as multiples of the full stimulus threshold plotted with predictions

from the SA (andashb) SI (cndashd) and TS (endashf) models Each row shows the same averaged data replotted from Figure 2c and d The only

fitted parameter was k the size of the pooling region in the TS model (3 middot 3) This fitting was performed by hand (Appendix B) RMS

errors between the model predictions and the data are shown in dB

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 7

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

meaning that at least part of the deficit for detecting

that stimulus was due to inefficient pooling of noise in

the noise check condition rather than a relative

insensitivity for performing the task in the periphery

(eg it may be more difficult for the observer to keep

track of the signal and nonsignal regions in the frac142708

noise check condition)

Modeling

Monte Carlo simulations

A set of models was developed to investigate thepooling strategy used by the observers The initial aim

Figure 4 Coherence thresholds plotted with predictions from the TN model fitted to the blank check data (andashb) and from the TA

model with the best-fitting parameters for the noise check data (TA1 cndashd) and the best-fitting parameters for the blank check data

(TA2 endashf) Each row shows the same averaged data replotted from Figure 2c and d The fitted parameters were the size of the pooling

region (k) the standard deviation of the internal noise (rint) and the size of the integration aperture (a)

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 8

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

of the modeling was to explain the surprising equiva-lence between the smallest check sizes in the noise checkand blank check conditions (see Figure 2c and d)which would not be expected if the source of thelimiting noise was in the responses to the individualmicropattern elements (as the additional randomlyoriented elements in the noise check condition wouldcause thresholds for those stimuli to be higher) Modelpredictions were obtained using stochastic MonteCarlo methods A set of model observers was developedwith different pooling strategies in MATLAB and runthrough 2000 simulated trials per stimulus level of alsquolsquomethod of constant stimulirsquorsquo version of the experi-ment The coherence thresholds for each modelobserver were calculated using the simulated data andexpressed as multiples of the full stimulus thresholdallowing them to be compared to the human resultswithout the need for fitting Three fitted parameters(the size of the local pooling regions k the standarddeviation of the internal noise rint and the size of theglobal pooling region a) were then added in order todevelop a model that provided a close account of thedata These parameters were each fitted by hand (seeAppendix B)

The filter-maxing combination processes

Several combination processes for the calculation ofglobal orientation from individual local samples havebeen suggested previously Here we implement thestrongest candidate for this task in which the observer

selects the orientation of the most strongly activatedoriented filter (after Jones et al 2003 Husk et al2012) This choice is not crucial to our findingshowever and a set of models developed using a vector-averaging combination process made similar predic-tions (Appendix C) In our filter-maxing model thestimulus is first filtered at the two potential targetorientations The filter elements are a pair of log-Gaborelements with the same tuning properties as those usedin the generation of the stimuli (spatial frequencybandwidth of 16 octaves and orientation bandwidth of6258) These bandwidths are typical of those usedpreviously in the literature to model simple cellresponses and compatible with those found in neuro-physiological investigations (De Valois amp De Valois1990 Meese 2010) As our filter elements wereidentical to the target elements in our stimuli they alsobehaved as the ideal detector for those elements Thefilter outputs are rectified and passed to the poolingstage In the pooling stage the outputs can be weightedaccording to their expected signal-to-noise ratio (de-pendent on the pooling strategy in operation seebelow) The weighted filter responses are then summedover the image for each orientation and these valuesare compared to each other The model observer picksthe orientation with the greater filter response

Pooling strategies

The simplest pooling strategy considered is the lsquolsquosumallrsquorsquo (SA) model in which the observer combines

Figure 5 Threshold elevation introduced by the noise check elements calculated from the data in Figure 2c and d as the thresholds

from the noise check condition divided by the thresholds from the blank check condition The predicted threshold elevations from

each of the models shown in Figures 3 and 4 are also plotted here Only the SA and TS models predict any systematic elevation effect

and neither of those match the pattern seen in the data

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 9

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

information from every element in the stimulus with anequal weighting regardless of whether it is a noise orpotential signal element The model with this strategypredicts that within each condition (noise check andblank check) there should be no effect of check size ormodulator phase The predicted coherence thresholdfor the noise check stimuli is approximately twice thatfor the full stimulus (Figure 3a) which is equivalent torequiring the same threshold proportion of coherentsignal elements across the whole stimulus For theblank check stimuli (Figure 3b) the predicted thresholdelevation is a factor of

ffiffiffi2p

as there are no interstitialnoise elements to limit performance in that conditionThese predictions capture the performance for thesmall check sizes (1 to 3) in the noise check conditionthe medium check sizes (5 to 9) in the blank checkcondition and the largest frac14 2708 size (15) in both thenoise check and blank check conditions (see Figure 3aand b) The predictions fail however to describe thegeneral form of either set of results which both featurea transition from greater to lesser summation as thecheck size increases

The lsquolsquosum ideallyrsquorsquo (SI) strategy involves the combi-nation of orientation information only from potentialsignal elements This is equivalent to weighting theelements according to their expected signal-to-noiseratio as the responses to the elements in the noiseregions provide zero signal Models with this strategypredict threshold elevation of a factor of approximatelyffiffiffi

2p

for both of the checked conditions (Figure 3c andd) which is identical to the prediction for the blankcheck stimuli with the SA model For this reason theresults in the blank check condition which were welldescribed by SA models are fit just as well by the SImodels In addition the SI models predict the

ffiffiffi2p

summation for the medium check sizes in the noisecheck condition However this model also fails tocapture the general form of the human data for eitherstimulus type

Two-stage hybrid models

The fact that the simpler candidate models featuringthe two different pooling strategies (SA and SI) eachpredicted performance for different subsets of theresults suggested that a more complete account couldbe provided by a model that combined their twobehaviors In the lsquolsquotwo-stage (TS) hybrid modelsrsquorsquomandatory local combination over a k middot k region(behaving like the SA model) is followed by flexiblepooling of the outputs from those regions weighted bytheir expected signal-to-noise ratios which wereapplied as a template (as in the SI model) The size ofthe local pooling region (k) was the only fittedparameter in this model The best fit to the noise check

data was provided by a k of 3 although the fit providedby a k of 2 was only marginally worse (see Appendix BFigure A2) In the noise check condition this modelpredicts an initial doubling of the coherence thresholdfor the small check sizes followed by an improvementin performance to approach a factor of

ffiffiffi2p

for themedium and large check sizes (see Figure 3e) Thiscaptures the performance for all but the largest (15)check sizes In the blank check condition (Figure 3f)the predictions are once again the same as for modelswith the SA and SI strategies

Internal noise

The SA SI and TS models all make predictions thatare identical to each other for the blank checkcondition This is because there are no interstitial noiseelements in that condition that can be inappropriatelypooled to elevate the coherence threshold The resultshowever show a doubling of threshold for the smallcheck sizes that is the same in the noise check and blankcheck conditions This is a larger performance deficitthan can be accounted for by any of the three models(which all predict a

ffiffiffi2p

threshold increase for the blankcheck condition) A two-stage hybrid model featuringadditive Gaussian internal noise at each location (TN)does however predict similar performance for theblank check conditions as for the noise check condi-tions (see Figure 4a and b) The internal noise in themodel is added after the rectification stage (meaningthat the noisy local outputs can be negative) Becauseof the mandatory combination rule at the first poolingstage this early noise model is equivalent to a model inwhich performance is limited by noise affecting theoutput of the first pooling stage

The standard deviation of the internal noise (rint) isnow an additional parameter in the model It isexpressed as a percentage of the summed element-wisefilter response to a matched log-Gabor element (ierelative to the maximum local filter output) As rint isincreased from zero it initially has the effect ofincreasing thresholds for the 1 middot 1 blank checkcondition to bring them in line with those from thenoise check condition Once these have becomeequivalent higher values of rint serve mainly to increaseor decrease the sensitivity for all conditions Calculat-ing the relative coherence thresholds from the output ofthe model therefore largely factors out the effects ofthis model parameter once rint is sufficiently high (seeAppendix B) Figure 4a and b shows the TN modelprediction with the best-fitting k and rint values for theblank check data (note that the best-fitting k for theblank check data is 2 whereas kfrac14 3 provided amarginally better fit to the noise check data as shown inFigure A2) For both the noise check and the blank

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 10

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

check conditions the TN model predicts an initialtwofold summation for the 1 middot 1 check size which thendecreases to approximately

ffiffiffi2p

as the check sizeincreases The predictions from this model account forthe average human thresholds in the noise checkcondition for all but the largest check size (Figure 4a)In the blank check condition (Figure 4b) thresholds arelower than those predicted by the model for themedium (9) and largest (15) frac14 908 condition but forall other conditions the human behavior is captured bythe model

Maximum integration aperture

In the TN model there are several reasons whycoherence thresholds could be elevated when signal ispresented only in the periphery (compare frac14 908 and2708 thresholds for the largest check size stimulus inFigure 4) The simplest would be if the observer wereonly able to pool information from elements in thecenter of the display This was tested using a lsquolsquotwo-stagehybrid model with internal noise and maximumintegration aperturersquorsquo (TA) in which the modelobserver only had access to information from elementsthat were within a central a middot a element squareaperture (equivalent to a degrees or 4a carrier cycles)This would also be equivalent to a model that featureda step-edge decline in sensitivity at this eccentricityMore complex accounts of the effect of visual fieldeccentricity are not explored here as our stimuli weredesigned to factor out these effects when possiblemeaning that any explanation we might provide wouldnot be well constrained by our data For examplefurther summation may be achieved by probabilitysummation between multiple integration apertures atdifferent locations across the visual field (resulting in amodel similar to that proposed for the area summationof contrast by Baker amp Meese 2011)

In order to show which features of our results the TAmodel could and could not account for two separatefits were performed to determine the optimal set ofparameters for the noise check (TA1) and the blankcheck (TA2) data The TA1 prediction is shown inFigure 4c and d The best-fitting integration aperturesize was 18 middot 18 degrees (compared to other squareapertures with integer dimensions see Appendix B)The TA1 model prediction is similar to that made bythe TN model (for the same kernel size see AppendixB) for all conditions except for the largest check size(15) For the 15 middot 15 check stimuli the predictedthresholds for the frac14 908 stimulus are reduced (whenexpressed relative to the full condition threshold) andthe thresholds for the frac14 2708 stimulus are elevatedThis model prediction provides a good fit to the noisecheck data (RMSefrac14 054 dB) and for the smaller check

sizes (1ndash3) in the blank check condition For the largercheck sizes in the blank check condition however theTA1 model prediction systematically underestimatesthe sensitivity of the human observers (resulting in arelatively large RMSe of 178 dB)

Fitting the aperture model to the blank checkcondition produces the TA2 prediction shown in Figure4e and f As in the TN model the best-fitting k for theblank check condition was 2 (as opposed to the valuefound from the TA1 fit to the noise check data whichfavored a k of 3) The best-fitting aperture size (a) was19 middot 19 degrees (18 wider than the size found by fittingto the noise check data) As would be expected thisversion of the model provides an inferior fit to the noisecheck data compared to the TA1 prediction (116 dB vs054 dB) underestimating both the threshold for the 3middot 3 check condition and the amount of separationbetween the two modulator phases at the largest checksize In the blank check condition although the qualityof the fit is improved from the TA1 prediction (112 vs178 dB) sensitivity is still underestimated for the larger(9ndash15) check sizes This indicates that there is nocombination of parameters for this model architecturethat can capture the performance for those conditions

Residual effects of check condition andmodulator phase

Figure 5 replots the data from Figure 2c and dshowing directly the threshold elevation effect that theinterstitial noise elements in the noise check conditionhave Also shown in Figure 5 are the thresholdelevation predictions from each of the six modelvariants presented earlier All models except SA and TSpredict no threshold elevation from the noise checks ineither modulator phase For the frac14 908 modulatorphase the data agree with this prediction with the ratiobetween the noise check and blank check thresholdsremaining close to one for all check sizes For the frac142708 modulator phase there is no threshold elevationintroduced by the noise checks at the smallest (1) checksize but as the check size increases the noise checkshave the effect of raising thresholds relative to theblank check condition up to a maximum factor of

ffiffiffi2p

for the largest (15) check size Only the SA and TSmodels have their performance limited by the intersti-tial randomly oriented elements in the noise checkstimuli and therefore predict threshold elevation in thenoise check condition The pattern of elevationpredicted however does not match that seen in the frac142708 data The key difference is that the data show athreshold elevation effect that both increases withcheck size and is dependent on the modulator phasewhereas none of the models investigated here make thisprediction One possible explanation for this effect

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 11

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

would be an increase in the size of the mandatory localintegration region with eccentricity (linking this to theexplanation of crowding provided by Parkes LundAngelucci Solomon amp Morgan 2001)

Discussion

Orientation integration is a noisy two-stageprocess

The results of this study suggest that the combina-tion of orientation information over space is a noisytwo-stage process (Figure 6) The key results reportedhere can be accounted for by a model that performsmandatory local integration affected by internal noiseat each location followed by flexible pooling over theoutputs from those regions (for the vector-averagingversion of the model these would seem equivalent tothe involuntary and voluntary averaging discussed byDakin Bex Cass amp Watt 2009) This account is inagreement with previous studies that have found lowerthresholds for stimuli with a greater signal area (Dakin2001 Jones et al 2003) The effect found for thearrangement of the elements in the display contradictsthe flexibility attributed to the pooling of local samplesby Dakin (2001) although it is possible that thedifferences in terms of the task set to the observer(signal in noise here vs fine discrimination in Dakin)would mean that they did not investigate the samesignal-combination process (as discussed in the Intro-duction to this paper) The close spacing of theelements in our stimuli and their extension into theperiphery would lead us to expect the individualelements to be crowded by each other Under theaccount provided by Dakin et al (2009) it is suggestedthat crowding only limits performance in tasks in whichthere is little orientation variability (eg a finediscrimination task) whereas for a signal-in-noise tasksuch as ours performance should be limited by thenumber of pooled samples If crowding were a limit onattentional resolution (Strasburger 2005) howeverthen we would expect it to affect the ability of ourobservers to segregate potential signal and noiseelements possibly explaining the elevated thresholds inthe frac142708 noise check condition In addition it is notpossible from our study to determine whether theobservers were pooling all of the available samples inthe stimulus or if they were making their decisionsbased on a subset of the local samples in the display (aswas found by Dakin 2001)

It is not entirely clear how the results presented herecan be reconciled with those of Husk et al (2012) whofound no summation with increasing signal area forsimilar stimuli The main difference between the stimuli

used in the two studies is that this study used theBattenberg summation paradigm whereas Husk et alincreased the signal area of their stimuli by increasingdiameter It is possible that a combination of decreas-ing sensitivity for the local orientation-discriminationtask and increasing the mandatory summation regionsize with eccentricity might flatten the threshold versusarea functions This question shall be addressed infuture work which will combine the Battenbergstimulus paradigm used here with a conventional area-summation design comparing stimuli of differentdiameters Performing this experiment at a variety ofspatial frequencies will also allow us to determinewhether the size of the mandatory integration region islinked to the scale of the elements that are beingpooled Previous studies that have investigated theprocessing of visual texture would lead us to predictthis to be the case (Kingdom Keeble amp Moulden1995 Kingdom amp Keeble 1999) however if themandatory pooling were related to crowding we wouldexpect its extent to be independent of the scale of thestimulus (Levi Hariharan amp Klein 2002)

It is noteworthy that the results presented in Figure2c and d bear a resemblance to those found in thecontrast Battenberg study by Meese (2010) with short-range linear summation followed by long-range square-law summation The explanation for the square-lawsummation differs however between the two studiesFor the orientation result presented here this nonlinearsummation is explained by a flexible pooling strategythat segregates out the input from the nonsignal regionsof the display whereas in the contrast study by Meesethe output from those regions was included with thatfrom the signal regions in the global combinationprocess and the nonlinear summation was accountedfor by a square-law transducer

The nature of the limiting internal noise

Figure 6 shows three possible locations for thelimiting noise in the modeling for this study (NearlyNmid or Nlate) The similarity of the results from thenoise check and blank check conditions indicate thatthe limiting noise is not driven by the response toindividual elements Instead they suggest that a level ofinternal noise is pooled that is proportional to thenumber of locations being monitored which includesblank locations that are being integrated at the firststage of the model Jones et al (2003) included lsquolsquolatenoisersquorsquo when modeling data from an orientationcoherence experiment that used filtered noise as stimuliIn the modeling for that study the noise was constantfor different signal areas (this would be Nlate in Figure6) A model based on dominant noise at Nlate wouldmake the same prediction as the SA model shown in

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 12

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

Figure 3a for both the noise check and blank checkconditions Such a model would not explain the resultspresented here as the improvement in performance seenfor the medium check size stimuli requires that theobserver is able to segregate out the limiting noise inirrelevant regions from the second combination stageIf dominant noise is contributed from each monitored

location then this could be performed by weighting thelocal outputs according to a template w as shown inFigure 6

In the internal noise model developed in this study(see Figure 4) the noise was implemented at eachpooled location after the initial filtering stage (Nearly)however due to the mandatory local combination at

Figure 6 Diagram of the TN model This diagram shows how the lsquolsquoverticalrsquorsquo response to an example stimulus (which includes blank

spaces in stimulus region n) is determined by filtering with a vertical filter element mandatory local summation and then global

summation of the local outputs (from stimulus regions 1 to n) weighted by the expected signal-to-noise ratio at each location (w) For

ease of presentation the local summation region shown is 1 middot 4 elements (our results suggest 3 middot 3) and only the responses to the

first and last stimulus regions are presented The lsquolsquohorizontalrsquorsquo response would be calculated in an identical manner except with a

horizontal filter element at the convolution stage Nearly Nmid and Nlate show three possible locations for the limiting internal noise

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 13

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

the first combination stage this is equivalent to addingnoise to the combined local outputs (Nmid) From theresults of this study it is not possible to determinewhether the limiting noise should be Nearly or Nmid inFigure 6 however Nmid seems more plausible as thelevel of early noise needed to exceed the external noiseintroduced by the randomly oriented elements in thenoise check stimuli would be very high In the best-fitting prediction to the noise check data (TA1) thestandard deviation of the early noise would be 85 ofthe mean response of the local detector to its idealstimulus in the best-fitting prediction to the blankcheck data (TA2) prediction it would be even higher(147) There is also the possibility that the limitinginternal noise could be multiplicative rather than theadditive noise implemented in the modeling here Weshall address this question through the use of anequivalent noise paradigm in future work

Keywords orientation summation integration tex-ture perception computational modeling

Acknowledgments

This work was supported by a grant from theNatural Sciences and Engineering Research Council(Canada) awarded to Robert Hess (46528-11) and agrant from the Engineering and Physical SciencesResearch Council (UK) awarded to Tim Meese andMark Georgeson (EPH0000381) The authors wouldlike to thank three anonymous reviewers for theirhelpful criticisms and suggestions

Commercial relationships noneCorresponding author Alex S BaldwinEmail alexsbaldwingooglemailcomAddress School of Life and Health Sciences AstonUniversity Birmingham UK

References

Baker D H amp Meese T S 2011 Contrastintegration over area is extensive A three-stagemodel of spatial summation Journal of Vision11(14)14 1ndash16 httpwwwjournalofvisionorgcontent111414 doi101167111414 [PubMed][Article]

Brainard D H 1997 The Psychophysics ToolboxSpatial Vision 10(4) 433ndash436

Dakin S C 2001 Information limit on the spatialintegration of local orientation signals Journal ofthe Optical Society of America A 18(5) 1016ndash1026

Dakin S C Bex P J Cass J R amp Watt R J 2009Dissociable effects of attention and crowding onorientation averaging Journal of Vision 9(11)281ndash16 httpwwwjournalofvisionorgcontent91128 doi10116791128 [PubMed] [Article]

Dakin S C amp Watt R J 1997 The computation oforientation statistics from visual texture VisionResearch 37(22) 3181ndash3192

De Valois R L amp De Valois K K 1990 Striatecortex In Spatial Vision (pp 94ndash146) Oxford UKOxford University Press

Husk J S Huang P-C amp Hess R F 2012Orientation coherence sensitivity Journal of Vision126(18) 1ndash15 httpwwwjournalofvisionorgcontent12618 doi10116712618 [PubMed][Article]

Jones D G Anderson N D amp Murphy K M 2003Orientation discrimination in visual noise usingglobal and local stimuli Vision Research 43 1223ndash1233

Kingdom F A Keeble D amp Moulden B 1995Sensitivity to orientation modulation in micro-pattern-based textures Vision Research 35(1) 79ndash91

Kingdom F A amp Keeble D R 1999 On themechanism for scale invariance in orientation-defined textures Vision Research 39(8) 1477ndash1489

Kleiner M Brainard D H amp Pelli D G 2007Whatrsquos new in Psychtoolbox-3 Perception 36(ECVP Abstract Supplement)

Landy M S amp Graham N 2004 Visual perceptionof texture In L M Chalupa amp J S Werner (Eds)The visual neurosciences (pp 1106ndash1118) Cam-bridge MA MIT Press

Levi D M Hariharan S amp Klein S A 2002Suppressive and facilitatory spatial interactions inperipheral vision Peripheral crowding is neithersize invariant nor simple contrast masking Journalof Vision 2 167ndash177 httpwwwjournalofvisionorgcontent223 doi101167223 [PubMed][Article]

Marr D 1982 Vision San Francisco W H Freemanand Company

Meese T S 2010 Spatially extensive summation ofcontrast energy is revealed by contrast detection ofmicro-pattern textures Journal of Vision 10(8)141ndash21 httpwwwjournalofvisionorgcontent10814 doi10116710814 [PubMed] [Article]

Meese T S amp Summers R J 2007 Area summationin human vision at and above detection thresholdProceedings of the Royal Society B 274(1627)2891ndash2900

Nichols M J amp Newsome W T 2002 Middle

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 14

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

meaning that at least part of the deficit for detecting

that stimulus was due to inefficient pooling of noise in

the noise check condition rather than a relative

insensitivity for performing the task in the periphery

(eg it may be more difficult for the observer to keep

track of the signal and nonsignal regions in the frac142708

noise check condition)

Modeling

Monte Carlo simulations

A set of models was developed to investigate thepooling strategy used by the observers The initial aim

Figure 4 Coherence thresholds plotted with predictions from the TN model fitted to the blank check data (andashb) and from the TA

model with the best-fitting parameters for the noise check data (TA1 cndashd) and the best-fitting parameters for the blank check data

(TA2 endashf) Each row shows the same averaged data replotted from Figure 2c and d The fitted parameters were the size of the pooling

region (k) the standard deviation of the internal noise (rint) and the size of the integration aperture (a)

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 8

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

of the modeling was to explain the surprising equiva-lence between the smallest check sizes in the noise checkand blank check conditions (see Figure 2c and d)which would not be expected if the source of thelimiting noise was in the responses to the individualmicropattern elements (as the additional randomlyoriented elements in the noise check condition wouldcause thresholds for those stimuli to be higher) Modelpredictions were obtained using stochastic MonteCarlo methods A set of model observers was developedwith different pooling strategies in MATLAB and runthrough 2000 simulated trials per stimulus level of alsquolsquomethod of constant stimulirsquorsquo version of the experi-ment The coherence thresholds for each modelobserver were calculated using the simulated data andexpressed as multiples of the full stimulus thresholdallowing them to be compared to the human resultswithout the need for fitting Three fitted parameters(the size of the local pooling regions k the standarddeviation of the internal noise rint and the size of theglobal pooling region a) were then added in order todevelop a model that provided a close account of thedata These parameters were each fitted by hand (seeAppendix B)

The filter-maxing combination processes

Several combination processes for the calculation ofglobal orientation from individual local samples havebeen suggested previously Here we implement thestrongest candidate for this task in which the observer

selects the orientation of the most strongly activatedoriented filter (after Jones et al 2003 Husk et al2012) This choice is not crucial to our findingshowever and a set of models developed using a vector-averaging combination process made similar predic-tions (Appendix C) In our filter-maxing model thestimulus is first filtered at the two potential targetorientations The filter elements are a pair of log-Gaborelements with the same tuning properties as those usedin the generation of the stimuli (spatial frequencybandwidth of 16 octaves and orientation bandwidth of6258) These bandwidths are typical of those usedpreviously in the literature to model simple cellresponses and compatible with those found in neuro-physiological investigations (De Valois amp De Valois1990 Meese 2010) As our filter elements wereidentical to the target elements in our stimuli they alsobehaved as the ideal detector for those elements Thefilter outputs are rectified and passed to the poolingstage In the pooling stage the outputs can be weightedaccording to their expected signal-to-noise ratio (de-pendent on the pooling strategy in operation seebelow) The weighted filter responses are then summedover the image for each orientation and these valuesare compared to each other The model observer picksthe orientation with the greater filter response

Pooling strategies

The simplest pooling strategy considered is the lsquolsquosumallrsquorsquo (SA) model in which the observer combines

Figure 5 Threshold elevation introduced by the noise check elements calculated from the data in Figure 2c and d as the thresholds

from the noise check condition divided by the thresholds from the blank check condition The predicted threshold elevations from

each of the models shown in Figures 3 and 4 are also plotted here Only the SA and TS models predict any systematic elevation effect

and neither of those match the pattern seen in the data

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 9

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

information from every element in the stimulus with anequal weighting regardless of whether it is a noise orpotential signal element The model with this strategypredicts that within each condition (noise check andblank check) there should be no effect of check size ormodulator phase The predicted coherence thresholdfor the noise check stimuli is approximately twice thatfor the full stimulus (Figure 3a) which is equivalent torequiring the same threshold proportion of coherentsignal elements across the whole stimulus For theblank check stimuli (Figure 3b) the predicted thresholdelevation is a factor of

ffiffiffi2p

as there are no interstitialnoise elements to limit performance in that conditionThese predictions capture the performance for thesmall check sizes (1 to 3) in the noise check conditionthe medium check sizes (5 to 9) in the blank checkcondition and the largest frac14 2708 size (15) in both thenoise check and blank check conditions (see Figure 3aand b) The predictions fail however to describe thegeneral form of either set of results which both featurea transition from greater to lesser summation as thecheck size increases

The lsquolsquosum ideallyrsquorsquo (SI) strategy involves the combi-nation of orientation information only from potentialsignal elements This is equivalent to weighting theelements according to their expected signal-to-noiseratio as the responses to the elements in the noiseregions provide zero signal Models with this strategypredict threshold elevation of a factor of approximatelyffiffiffi

2p

for both of the checked conditions (Figure 3c andd) which is identical to the prediction for the blankcheck stimuli with the SA model For this reason theresults in the blank check condition which were welldescribed by SA models are fit just as well by the SImodels In addition the SI models predict the

ffiffiffi2p

summation for the medium check sizes in the noisecheck condition However this model also fails tocapture the general form of the human data for eitherstimulus type

Two-stage hybrid models

The fact that the simpler candidate models featuringthe two different pooling strategies (SA and SI) eachpredicted performance for different subsets of theresults suggested that a more complete account couldbe provided by a model that combined their twobehaviors In the lsquolsquotwo-stage (TS) hybrid modelsrsquorsquomandatory local combination over a k middot k region(behaving like the SA model) is followed by flexiblepooling of the outputs from those regions weighted bytheir expected signal-to-noise ratios which wereapplied as a template (as in the SI model) The size ofthe local pooling region (k) was the only fittedparameter in this model The best fit to the noise check

data was provided by a k of 3 although the fit providedby a k of 2 was only marginally worse (see Appendix BFigure A2) In the noise check condition this modelpredicts an initial doubling of the coherence thresholdfor the small check sizes followed by an improvementin performance to approach a factor of

ffiffiffi2p

for themedium and large check sizes (see Figure 3e) Thiscaptures the performance for all but the largest (15)check sizes In the blank check condition (Figure 3f)the predictions are once again the same as for modelswith the SA and SI strategies

Internal noise

The SA SI and TS models all make predictions thatare identical to each other for the blank checkcondition This is because there are no interstitial noiseelements in that condition that can be inappropriatelypooled to elevate the coherence threshold The resultshowever show a doubling of threshold for the smallcheck sizes that is the same in the noise check and blankcheck conditions This is a larger performance deficitthan can be accounted for by any of the three models(which all predict a

ffiffiffi2p

threshold increase for the blankcheck condition) A two-stage hybrid model featuringadditive Gaussian internal noise at each location (TN)does however predict similar performance for theblank check conditions as for the noise check condi-tions (see Figure 4a and b) The internal noise in themodel is added after the rectification stage (meaningthat the noisy local outputs can be negative) Becauseof the mandatory combination rule at the first poolingstage this early noise model is equivalent to a model inwhich performance is limited by noise affecting theoutput of the first pooling stage

The standard deviation of the internal noise (rint) isnow an additional parameter in the model It isexpressed as a percentage of the summed element-wisefilter response to a matched log-Gabor element (ierelative to the maximum local filter output) As rint isincreased from zero it initially has the effect ofincreasing thresholds for the 1 middot 1 blank checkcondition to bring them in line with those from thenoise check condition Once these have becomeequivalent higher values of rint serve mainly to increaseor decrease the sensitivity for all conditions Calculat-ing the relative coherence thresholds from the output ofthe model therefore largely factors out the effects ofthis model parameter once rint is sufficiently high (seeAppendix B) Figure 4a and b shows the TN modelprediction with the best-fitting k and rint values for theblank check data (note that the best-fitting k for theblank check data is 2 whereas kfrac14 3 provided amarginally better fit to the noise check data as shown inFigure A2) For both the noise check and the blank

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 10

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

check conditions the TN model predicts an initialtwofold summation for the 1 middot 1 check size which thendecreases to approximately

ffiffiffi2p

as the check sizeincreases The predictions from this model account forthe average human thresholds in the noise checkcondition for all but the largest check size (Figure 4a)In the blank check condition (Figure 4b) thresholds arelower than those predicted by the model for themedium (9) and largest (15) frac14 908 condition but forall other conditions the human behavior is captured bythe model

Maximum integration aperture

In the TN model there are several reasons whycoherence thresholds could be elevated when signal ispresented only in the periphery (compare frac14 908 and2708 thresholds for the largest check size stimulus inFigure 4) The simplest would be if the observer wereonly able to pool information from elements in thecenter of the display This was tested using a lsquolsquotwo-stagehybrid model with internal noise and maximumintegration aperturersquorsquo (TA) in which the modelobserver only had access to information from elementsthat were within a central a middot a element squareaperture (equivalent to a degrees or 4a carrier cycles)This would also be equivalent to a model that featureda step-edge decline in sensitivity at this eccentricityMore complex accounts of the effect of visual fieldeccentricity are not explored here as our stimuli weredesigned to factor out these effects when possiblemeaning that any explanation we might provide wouldnot be well constrained by our data For examplefurther summation may be achieved by probabilitysummation between multiple integration apertures atdifferent locations across the visual field (resulting in amodel similar to that proposed for the area summationof contrast by Baker amp Meese 2011)

In order to show which features of our results the TAmodel could and could not account for two separatefits were performed to determine the optimal set ofparameters for the noise check (TA1) and the blankcheck (TA2) data The TA1 prediction is shown inFigure 4c and d The best-fitting integration aperturesize was 18 middot 18 degrees (compared to other squareapertures with integer dimensions see Appendix B)The TA1 model prediction is similar to that made bythe TN model (for the same kernel size see AppendixB) for all conditions except for the largest check size(15) For the 15 middot 15 check stimuli the predictedthresholds for the frac14 908 stimulus are reduced (whenexpressed relative to the full condition threshold) andthe thresholds for the frac14 2708 stimulus are elevatedThis model prediction provides a good fit to the noisecheck data (RMSefrac14 054 dB) and for the smaller check

sizes (1ndash3) in the blank check condition For the largercheck sizes in the blank check condition however theTA1 model prediction systematically underestimatesthe sensitivity of the human observers (resulting in arelatively large RMSe of 178 dB)

Fitting the aperture model to the blank checkcondition produces the TA2 prediction shown in Figure4e and f As in the TN model the best-fitting k for theblank check condition was 2 (as opposed to the valuefound from the TA1 fit to the noise check data whichfavored a k of 3) The best-fitting aperture size (a) was19 middot 19 degrees (18 wider than the size found by fittingto the noise check data) As would be expected thisversion of the model provides an inferior fit to the noisecheck data compared to the TA1 prediction (116 dB vs054 dB) underestimating both the threshold for the 3middot 3 check condition and the amount of separationbetween the two modulator phases at the largest checksize In the blank check condition although the qualityof the fit is improved from the TA1 prediction (112 vs178 dB) sensitivity is still underestimated for the larger(9ndash15) check sizes This indicates that there is nocombination of parameters for this model architecturethat can capture the performance for those conditions

Residual effects of check condition andmodulator phase

Figure 5 replots the data from Figure 2c and dshowing directly the threshold elevation effect that theinterstitial noise elements in the noise check conditionhave Also shown in Figure 5 are the thresholdelevation predictions from each of the six modelvariants presented earlier All models except SA and TSpredict no threshold elevation from the noise checks ineither modulator phase For the frac14 908 modulatorphase the data agree with this prediction with the ratiobetween the noise check and blank check thresholdsremaining close to one for all check sizes For the frac142708 modulator phase there is no threshold elevationintroduced by the noise checks at the smallest (1) checksize but as the check size increases the noise checkshave the effect of raising thresholds relative to theblank check condition up to a maximum factor of

ffiffiffi2p

for the largest (15) check size Only the SA and TSmodels have their performance limited by the intersti-tial randomly oriented elements in the noise checkstimuli and therefore predict threshold elevation in thenoise check condition The pattern of elevationpredicted however does not match that seen in the frac142708 data The key difference is that the data show athreshold elevation effect that both increases withcheck size and is dependent on the modulator phasewhereas none of the models investigated here make thisprediction One possible explanation for this effect

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 11

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

would be an increase in the size of the mandatory localintegration region with eccentricity (linking this to theexplanation of crowding provided by Parkes LundAngelucci Solomon amp Morgan 2001)

Discussion

Orientation integration is a noisy two-stageprocess

The results of this study suggest that the combina-tion of orientation information over space is a noisytwo-stage process (Figure 6) The key results reportedhere can be accounted for by a model that performsmandatory local integration affected by internal noiseat each location followed by flexible pooling over theoutputs from those regions (for the vector-averagingversion of the model these would seem equivalent tothe involuntary and voluntary averaging discussed byDakin Bex Cass amp Watt 2009) This account is inagreement with previous studies that have found lowerthresholds for stimuli with a greater signal area (Dakin2001 Jones et al 2003) The effect found for thearrangement of the elements in the display contradictsthe flexibility attributed to the pooling of local samplesby Dakin (2001) although it is possible that thedifferences in terms of the task set to the observer(signal in noise here vs fine discrimination in Dakin)would mean that they did not investigate the samesignal-combination process (as discussed in the Intro-duction to this paper) The close spacing of theelements in our stimuli and their extension into theperiphery would lead us to expect the individualelements to be crowded by each other Under theaccount provided by Dakin et al (2009) it is suggestedthat crowding only limits performance in tasks in whichthere is little orientation variability (eg a finediscrimination task) whereas for a signal-in-noise tasksuch as ours performance should be limited by thenumber of pooled samples If crowding were a limit onattentional resolution (Strasburger 2005) howeverthen we would expect it to affect the ability of ourobservers to segregate potential signal and noiseelements possibly explaining the elevated thresholds inthe frac142708 noise check condition In addition it is notpossible from our study to determine whether theobservers were pooling all of the available samples inthe stimulus or if they were making their decisionsbased on a subset of the local samples in the display (aswas found by Dakin 2001)

It is not entirely clear how the results presented herecan be reconciled with those of Husk et al (2012) whofound no summation with increasing signal area forsimilar stimuli The main difference between the stimuli

used in the two studies is that this study used theBattenberg summation paradigm whereas Husk et alincreased the signal area of their stimuli by increasingdiameter It is possible that a combination of decreas-ing sensitivity for the local orientation-discriminationtask and increasing the mandatory summation regionsize with eccentricity might flatten the threshold versusarea functions This question shall be addressed infuture work which will combine the Battenbergstimulus paradigm used here with a conventional area-summation design comparing stimuli of differentdiameters Performing this experiment at a variety ofspatial frequencies will also allow us to determinewhether the size of the mandatory integration region islinked to the scale of the elements that are beingpooled Previous studies that have investigated theprocessing of visual texture would lead us to predictthis to be the case (Kingdom Keeble amp Moulden1995 Kingdom amp Keeble 1999) however if themandatory pooling were related to crowding we wouldexpect its extent to be independent of the scale of thestimulus (Levi Hariharan amp Klein 2002)

It is noteworthy that the results presented in Figure2c and d bear a resemblance to those found in thecontrast Battenberg study by Meese (2010) with short-range linear summation followed by long-range square-law summation The explanation for the square-lawsummation differs however between the two studiesFor the orientation result presented here this nonlinearsummation is explained by a flexible pooling strategythat segregates out the input from the nonsignal regionsof the display whereas in the contrast study by Meesethe output from those regions was included with thatfrom the signal regions in the global combinationprocess and the nonlinear summation was accountedfor by a square-law transducer

The nature of the limiting internal noise

Figure 6 shows three possible locations for thelimiting noise in the modeling for this study (NearlyNmid or Nlate) The similarity of the results from thenoise check and blank check conditions indicate thatthe limiting noise is not driven by the response toindividual elements Instead they suggest that a level ofinternal noise is pooled that is proportional to thenumber of locations being monitored which includesblank locations that are being integrated at the firststage of the model Jones et al (2003) included lsquolsquolatenoisersquorsquo when modeling data from an orientationcoherence experiment that used filtered noise as stimuliIn the modeling for that study the noise was constantfor different signal areas (this would be Nlate in Figure6) A model based on dominant noise at Nlate wouldmake the same prediction as the SA model shown in

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 12

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

Figure 3a for both the noise check and blank checkconditions Such a model would not explain the resultspresented here as the improvement in performance seenfor the medium check size stimuli requires that theobserver is able to segregate out the limiting noise inirrelevant regions from the second combination stageIf dominant noise is contributed from each monitored

location then this could be performed by weighting thelocal outputs according to a template w as shown inFigure 6

In the internal noise model developed in this study(see Figure 4) the noise was implemented at eachpooled location after the initial filtering stage (Nearly)however due to the mandatory local combination at

Figure 6 Diagram of the TN model This diagram shows how the lsquolsquoverticalrsquorsquo response to an example stimulus (which includes blank

spaces in stimulus region n) is determined by filtering with a vertical filter element mandatory local summation and then global

summation of the local outputs (from stimulus regions 1 to n) weighted by the expected signal-to-noise ratio at each location (w) For

ease of presentation the local summation region shown is 1 middot 4 elements (our results suggest 3 middot 3) and only the responses to the

first and last stimulus regions are presented The lsquolsquohorizontalrsquorsquo response would be calculated in an identical manner except with a

horizontal filter element at the convolution stage Nearly Nmid and Nlate show three possible locations for the limiting internal noise

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 13

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

the first combination stage this is equivalent to addingnoise to the combined local outputs (Nmid) From theresults of this study it is not possible to determinewhether the limiting noise should be Nearly or Nmid inFigure 6 however Nmid seems more plausible as thelevel of early noise needed to exceed the external noiseintroduced by the randomly oriented elements in thenoise check stimuli would be very high In the best-fitting prediction to the noise check data (TA1) thestandard deviation of the early noise would be 85 ofthe mean response of the local detector to its idealstimulus in the best-fitting prediction to the blankcheck data (TA2) prediction it would be even higher(147) There is also the possibility that the limitinginternal noise could be multiplicative rather than theadditive noise implemented in the modeling here Weshall address this question through the use of anequivalent noise paradigm in future work

Keywords orientation summation integration tex-ture perception computational modeling

Acknowledgments

This work was supported by a grant from theNatural Sciences and Engineering Research Council(Canada) awarded to Robert Hess (46528-11) and agrant from the Engineering and Physical SciencesResearch Council (UK) awarded to Tim Meese andMark Georgeson (EPH0000381) The authors wouldlike to thank three anonymous reviewers for theirhelpful criticisms and suggestions

Commercial relationships noneCorresponding author Alex S BaldwinEmail alexsbaldwingooglemailcomAddress School of Life and Health Sciences AstonUniversity Birmingham UK

References

Baker D H amp Meese T S 2011 Contrastintegration over area is extensive A three-stagemodel of spatial summation Journal of Vision11(14)14 1ndash16 httpwwwjournalofvisionorgcontent111414 doi101167111414 [PubMed][Article]

Brainard D H 1997 The Psychophysics ToolboxSpatial Vision 10(4) 433ndash436

Dakin S C 2001 Information limit on the spatialintegration of local orientation signals Journal ofthe Optical Society of America A 18(5) 1016ndash1026

Dakin S C Bex P J Cass J R amp Watt R J 2009Dissociable effects of attention and crowding onorientation averaging Journal of Vision 9(11)281ndash16 httpwwwjournalofvisionorgcontent91128 doi10116791128 [PubMed] [Article]

Dakin S C amp Watt R J 1997 The computation oforientation statistics from visual texture VisionResearch 37(22) 3181ndash3192

De Valois R L amp De Valois K K 1990 Striatecortex In Spatial Vision (pp 94ndash146) Oxford UKOxford University Press

Husk J S Huang P-C amp Hess R F 2012Orientation coherence sensitivity Journal of Vision126(18) 1ndash15 httpwwwjournalofvisionorgcontent12618 doi10116712618 [PubMed][Article]

Jones D G Anderson N D amp Murphy K M 2003Orientation discrimination in visual noise usingglobal and local stimuli Vision Research 43 1223ndash1233

Kingdom F A Keeble D amp Moulden B 1995Sensitivity to orientation modulation in micro-pattern-based textures Vision Research 35(1) 79ndash91

Kingdom F A amp Keeble D R 1999 On themechanism for scale invariance in orientation-defined textures Vision Research 39(8) 1477ndash1489

Kleiner M Brainard D H amp Pelli D G 2007Whatrsquos new in Psychtoolbox-3 Perception 36(ECVP Abstract Supplement)

Landy M S amp Graham N 2004 Visual perceptionof texture In L M Chalupa amp J S Werner (Eds)The visual neurosciences (pp 1106ndash1118) Cam-bridge MA MIT Press

Levi D M Hariharan S amp Klein S A 2002Suppressive and facilitatory spatial interactions inperipheral vision Peripheral crowding is neithersize invariant nor simple contrast masking Journalof Vision 2 167ndash177 httpwwwjournalofvisionorgcontent223 doi101167223 [PubMed][Article]

Marr D 1982 Vision San Francisco W H Freemanand Company

Meese T S 2010 Spatially extensive summation ofcontrast energy is revealed by contrast detection ofmicro-pattern textures Journal of Vision 10(8)141ndash21 httpwwwjournalofvisionorgcontent10814 doi10116710814 [PubMed] [Article]

Meese T S amp Summers R J 2007 Area summationin human vision at and above detection thresholdProceedings of the Royal Society B 274(1627)2891ndash2900

Nichols M J amp Newsome W T 2002 Middle

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 14

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

of the modeling was to explain the surprising equiva-lence between the smallest check sizes in the noise checkand blank check conditions (see Figure 2c and d)which would not be expected if the source of thelimiting noise was in the responses to the individualmicropattern elements (as the additional randomlyoriented elements in the noise check condition wouldcause thresholds for those stimuli to be higher) Modelpredictions were obtained using stochastic MonteCarlo methods A set of model observers was developedwith different pooling strategies in MATLAB and runthrough 2000 simulated trials per stimulus level of alsquolsquomethod of constant stimulirsquorsquo version of the experi-ment The coherence thresholds for each modelobserver were calculated using the simulated data andexpressed as multiples of the full stimulus thresholdallowing them to be compared to the human resultswithout the need for fitting Three fitted parameters(the size of the local pooling regions k the standarddeviation of the internal noise rint and the size of theglobal pooling region a) were then added in order todevelop a model that provided a close account of thedata These parameters were each fitted by hand (seeAppendix B)

The filter-maxing combination processes

Several combination processes for the calculation ofglobal orientation from individual local samples havebeen suggested previously Here we implement thestrongest candidate for this task in which the observer

selects the orientation of the most strongly activatedoriented filter (after Jones et al 2003 Husk et al2012) This choice is not crucial to our findingshowever and a set of models developed using a vector-averaging combination process made similar predic-tions (Appendix C) In our filter-maxing model thestimulus is first filtered at the two potential targetorientations The filter elements are a pair of log-Gaborelements with the same tuning properties as those usedin the generation of the stimuli (spatial frequencybandwidth of 16 octaves and orientation bandwidth of6258) These bandwidths are typical of those usedpreviously in the literature to model simple cellresponses and compatible with those found in neuro-physiological investigations (De Valois amp De Valois1990 Meese 2010) As our filter elements wereidentical to the target elements in our stimuli they alsobehaved as the ideal detector for those elements Thefilter outputs are rectified and passed to the poolingstage In the pooling stage the outputs can be weightedaccording to their expected signal-to-noise ratio (de-pendent on the pooling strategy in operation seebelow) The weighted filter responses are then summedover the image for each orientation and these valuesare compared to each other The model observer picksthe orientation with the greater filter response

Pooling strategies

The simplest pooling strategy considered is the lsquolsquosumallrsquorsquo (SA) model in which the observer combines

Figure 5 Threshold elevation introduced by the noise check elements calculated from the data in Figure 2c and d as the thresholds

from the noise check condition divided by the thresholds from the blank check condition The predicted threshold elevations from

each of the models shown in Figures 3 and 4 are also plotted here Only the SA and TS models predict any systematic elevation effect

and neither of those match the pattern seen in the data

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 9

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

information from every element in the stimulus with anequal weighting regardless of whether it is a noise orpotential signal element The model with this strategypredicts that within each condition (noise check andblank check) there should be no effect of check size ormodulator phase The predicted coherence thresholdfor the noise check stimuli is approximately twice thatfor the full stimulus (Figure 3a) which is equivalent torequiring the same threshold proportion of coherentsignal elements across the whole stimulus For theblank check stimuli (Figure 3b) the predicted thresholdelevation is a factor of

ffiffiffi2p

as there are no interstitialnoise elements to limit performance in that conditionThese predictions capture the performance for thesmall check sizes (1 to 3) in the noise check conditionthe medium check sizes (5 to 9) in the blank checkcondition and the largest frac14 2708 size (15) in both thenoise check and blank check conditions (see Figure 3aand b) The predictions fail however to describe thegeneral form of either set of results which both featurea transition from greater to lesser summation as thecheck size increases

The lsquolsquosum ideallyrsquorsquo (SI) strategy involves the combi-nation of orientation information only from potentialsignal elements This is equivalent to weighting theelements according to their expected signal-to-noiseratio as the responses to the elements in the noiseregions provide zero signal Models with this strategypredict threshold elevation of a factor of approximatelyffiffiffi

2p

for both of the checked conditions (Figure 3c andd) which is identical to the prediction for the blankcheck stimuli with the SA model For this reason theresults in the blank check condition which were welldescribed by SA models are fit just as well by the SImodels In addition the SI models predict the

ffiffiffi2p

summation for the medium check sizes in the noisecheck condition However this model also fails tocapture the general form of the human data for eitherstimulus type

Two-stage hybrid models

The fact that the simpler candidate models featuringthe two different pooling strategies (SA and SI) eachpredicted performance for different subsets of theresults suggested that a more complete account couldbe provided by a model that combined their twobehaviors In the lsquolsquotwo-stage (TS) hybrid modelsrsquorsquomandatory local combination over a k middot k region(behaving like the SA model) is followed by flexiblepooling of the outputs from those regions weighted bytheir expected signal-to-noise ratios which wereapplied as a template (as in the SI model) The size ofthe local pooling region (k) was the only fittedparameter in this model The best fit to the noise check

data was provided by a k of 3 although the fit providedby a k of 2 was only marginally worse (see Appendix BFigure A2) In the noise check condition this modelpredicts an initial doubling of the coherence thresholdfor the small check sizes followed by an improvementin performance to approach a factor of

ffiffiffi2p

for themedium and large check sizes (see Figure 3e) Thiscaptures the performance for all but the largest (15)check sizes In the blank check condition (Figure 3f)the predictions are once again the same as for modelswith the SA and SI strategies

Internal noise

The SA SI and TS models all make predictions thatare identical to each other for the blank checkcondition This is because there are no interstitial noiseelements in that condition that can be inappropriatelypooled to elevate the coherence threshold The resultshowever show a doubling of threshold for the smallcheck sizes that is the same in the noise check and blankcheck conditions This is a larger performance deficitthan can be accounted for by any of the three models(which all predict a

ffiffiffi2p

threshold increase for the blankcheck condition) A two-stage hybrid model featuringadditive Gaussian internal noise at each location (TN)does however predict similar performance for theblank check conditions as for the noise check condi-tions (see Figure 4a and b) The internal noise in themodel is added after the rectification stage (meaningthat the noisy local outputs can be negative) Becauseof the mandatory combination rule at the first poolingstage this early noise model is equivalent to a model inwhich performance is limited by noise affecting theoutput of the first pooling stage

The standard deviation of the internal noise (rint) isnow an additional parameter in the model It isexpressed as a percentage of the summed element-wisefilter response to a matched log-Gabor element (ierelative to the maximum local filter output) As rint isincreased from zero it initially has the effect ofincreasing thresholds for the 1 middot 1 blank checkcondition to bring them in line with those from thenoise check condition Once these have becomeequivalent higher values of rint serve mainly to increaseor decrease the sensitivity for all conditions Calculat-ing the relative coherence thresholds from the output ofthe model therefore largely factors out the effects ofthis model parameter once rint is sufficiently high (seeAppendix B) Figure 4a and b shows the TN modelprediction with the best-fitting k and rint values for theblank check data (note that the best-fitting k for theblank check data is 2 whereas kfrac14 3 provided amarginally better fit to the noise check data as shown inFigure A2) For both the noise check and the blank

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 10

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

check conditions the TN model predicts an initialtwofold summation for the 1 middot 1 check size which thendecreases to approximately

ffiffiffi2p

as the check sizeincreases The predictions from this model account forthe average human thresholds in the noise checkcondition for all but the largest check size (Figure 4a)In the blank check condition (Figure 4b) thresholds arelower than those predicted by the model for themedium (9) and largest (15) frac14 908 condition but forall other conditions the human behavior is captured bythe model

Maximum integration aperture

In the TN model there are several reasons whycoherence thresholds could be elevated when signal ispresented only in the periphery (compare frac14 908 and2708 thresholds for the largest check size stimulus inFigure 4) The simplest would be if the observer wereonly able to pool information from elements in thecenter of the display This was tested using a lsquolsquotwo-stagehybrid model with internal noise and maximumintegration aperturersquorsquo (TA) in which the modelobserver only had access to information from elementsthat were within a central a middot a element squareaperture (equivalent to a degrees or 4a carrier cycles)This would also be equivalent to a model that featureda step-edge decline in sensitivity at this eccentricityMore complex accounts of the effect of visual fieldeccentricity are not explored here as our stimuli weredesigned to factor out these effects when possiblemeaning that any explanation we might provide wouldnot be well constrained by our data For examplefurther summation may be achieved by probabilitysummation between multiple integration apertures atdifferent locations across the visual field (resulting in amodel similar to that proposed for the area summationof contrast by Baker amp Meese 2011)

In order to show which features of our results the TAmodel could and could not account for two separatefits were performed to determine the optimal set ofparameters for the noise check (TA1) and the blankcheck (TA2) data The TA1 prediction is shown inFigure 4c and d The best-fitting integration aperturesize was 18 middot 18 degrees (compared to other squareapertures with integer dimensions see Appendix B)The TA1 model prediction is similar to that made bythe TN model (for the same kernel size see AppendixB) for all conditions except for the largest check size(15) For the 15 middot 15 check stimuli the predictedthresholds for the frac14 908 stimulus are reduced (whenexpressed relative to the full condition threshold) andthe thresholds for the frac14 2708 stimulus are elevatedThis model prediction provides a good fit to the noisecheck data (RMSefrac14 054 dB) and for the smaller check

sizes (1ndash3) in the blank check condition For the largercheck sizes in the blank check condition however theTA1 model prediction systematically underestimatesthe sensitivity of the human observers (resulting in arelatively large RMSe of 178 dB)

Fitting the aperture model to the blank checkcondition produces the TA2 prediction shown in Figure4e and f As in the TN model the best-fitting k for theblank check condition was 2 (as opposed to the valuefound from the TA1 fit to the noise check data whichfavored a k of 3) The best-fitting aperture size (a) was19 middot 19 degrees (18 wider than the size found by fittingto the noise check data) As would be expected thisversion of the model provides an inferior fit to the noisecheck data compared to the TA1 prediction (116 dB vs054 dB) underestimating both the threshold for the 3middot 3 check condition and the amount of separationbetween the two modulator phases at the largest checksize In the blank check condition although the qualityof the fit is improved from the TA1 prediction (112 vs178 dB) sensitivity is still underestimated for the larger(9ndash15) check sizes This indicates that there is nocombination of parameters for this model architecturethat can capture the performance for those conditions

Residual effects of check condition andmodulator phase

Figure 5 replots the data from Figure 2c and dshowing directly the threshold elevation effect that theinterstitial noise elements in the noise check conditionhave Also shown in Figure 5 are the thresholdelevation predictions from each of the six modelvariants presented earlier All models except SA and TSpredict no threshold elevation from the noise checks ineither modulator phase For the frac14 908 modulatorphase the data agree with this prediction with the ratiobetween the noise check and blank check thresholdsremaining close to one for all check sizes For the frac142708 modulator phase there is no threshold elevationintroduced by the noise checks at the smallest (1) checksize but as the check size increases the noise checkshave the effect of raising thresholds relative to theblank check condition up to a maximum factor of

ffiffiffi2p

for the largest (15) check size Only the SA and TSmodels have their performance limited by the intersti-tial randomly oriented elements in the noise checkstimuli and therefore predict threshold elevation in thenoise check condition The pattern of elevationpredicted however does not match that seen in the frac142708 data The key difference is that the data show athreshold elevation effect that both increases withcheck size and is dependent on the modulator phasewhereas none of the models investigated here make thisprediction One possible explanation for this effect

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 11

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

would be an increase in the size of the mandatory localintegration region with eccentricity (linking this to theexplanation of crowding provided by Parkes LundAngelucci Solomon amp Morgan 2001)

Discussion

Orientation integration is a noisy two-stageprocess

The results of this study suggest that the combina-tion of orientation information over space is a noisytwo-stage process (Figure 6) The key results reportedhere can be accounted for by a model that performsmandatory local integration affected by internal noiseat each location followed by flexible pooling over theoutputs from those regions (for the vector-averagingversion of the model these would seem equivalent tothe involuntary and voluntary averaging discussed byDakin Bex Cass amp Watt 2009) This account is inagreement with previous studies that have found lowerthresholds for stimuli with a greater signal area (Dakin2001 Jones et al 2003) The effect found for thearrangement of the elements in the display contradictsthe flexibility attributed to the pooling of local samplesby Dakin (2001) although it is possible that thedifferences in terms of the task set to the observer(signal in noise here vs fine discrimination in Dakin)would mean that they did not investigate the samesignal-combination process (as discussed in the Intro-duction to this paper) The close spacing of theelements in our stimuli and their extension into theperiphery would lead us to expect the individualelements to be crowded by each other Under theaccount provided by Dakin et al (2009) it is suggestedthat crowding only limits performance in tasks in whichthere is little orientation variability (eg a finediscrimination task) whereas for a signal-in-noise tasksuch as ours performance should be limited by thenumber of pooled samples If crowding were a limit onattentional resolution (Strasburger 2005) howeverthen we would expect it to affect the ability of ourobservers to segregate potential signal and noiseelements possibly explaining the elevated thresholds inthe frac142708 noise check condition In addition it is notpossible from our study to determine whether theobservers were pooling all of the available samples inthe stimulus or if they were making their decisionsbased on a subset of the local samples in the display (aswas found by Dakin 2001)

It is not entirely clear how the results presented herecan be reconciled with those of Husk et al (2012) whofound no summation with increasing signal area forsimilar stimuli The main difference between the stimuli

used in the two studies is that this study used theBattenberg summation paradigm whereas Husk et alincreased the signal area of their stimuli by increasingdiameter It is possible that a combination of decreas-ing sensitivity for the local orientation-discriminationtask and increasing the mandatory summation regionsize with eccentricity might flatten the threshold versusarea functions This question shall be addressed infuture work which will combine the Battenbergstimulus paradigm used here with a conventional area-summation design comparing stimuli of differentdiameters Performing this experiment at a variety ofspatial frequencies will also allow us to determinewhether the size of the mandatory integration region islinked to the scale of the elements that are beingpooled Previous studies that have investigated theprocessing of visual texture would lead us to predictthis to be the case (Kingdom Keeble amp Moulden1995 Kingdom amp Keeble 1999) however if themandatory pooling were related to crowding we wouldexpect its extent to be independent of the scale of thestimulus (Levi Hariharan amp Klein 2002)

It is noteworthy that the results presented in Figure2c and d bear a resemblance to those found in thecontrast Battenberg study by Meese (2010) with short-range linear summation followed by long-range square-law summation The explanation for the square-lawsummation differs however between the two studiesFor the orientation result presented here this nonlinearsummation is explained by a flexible pooling strategythat segregates out the input from the nonsignal regionsof the display whereas in the contrast study by Meesethe output from those regions was included with thatfrom the signal regions in the global combinationprocess and the nonlinear summation was accountedfor by a square-law transducer

The nature of the limiting internal noise

Figure 6 shows three possible locations for thelimiting noise in the modeling for this study (NearlyNmid or Nlate) The similarity of the results from thenoise check and blank check conditions indicate thatthe limiting noise is not driven by the response toindividual elements Instead they suggest that a level ofinternal noise is pooled that is proportional to thenumber of locations being monitored which includesblank locations that are being integrated at the firststage of the model Jones et al (2003) included lsquolsquolatenoisersquorsquo when modeling data from an orientationcoherence experiment that used filtered noise as stimuliIn the modeling for that study the noise was constantfor different signal areas (this would be Nlate in Figure6) A model based on dominant noise at Nlate wouldmake the same prediction as the SA model shown in

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 12

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

Figure 3a for both the noise check and blank checkconditions Such a model would not explain the resultspresented here as the improvement in performance seenfor the medium check size stimuli requires that theobserver is able to segregate out the limiting noise inirrelevant regions from the second combination stageIf dominant noise is contributed from each monitored

location then this could be performed by weighting thelocal outputs according to a template w as shown inFigure 6

In the internal noise model developed in this study(see Figure 4) the noise was implemented at eachpooled location after the initial filtering stage (Nearly)however due to the mandatory local combination at

Figure 6 Diagram of the TN model This diagram shows how the lsquolsquoverticalrsquorsquo response to an example stimulus (which includes blank

spaces in stimulus region n) is determined by filtering with a vertical filter element mandatory local summation and then global

summation of the local outputs (from stimulus regions 1 to n) weighted by the expected signal-to-noise ratio at each location (w) For

ease of presentation the local summation region shown is 1 middot 4 elements (our results suggest 3 middot 3) and only the responses to the

first and last stimulus regions are presented The lsquolsquohorizontalrsquorsquo response would be calculated in an identical manner except with a

horizontal filter element at the convolution stage Nearly Nmid and Nlate show three possible locations for the limiting internal noise

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 13

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

the first combination stage this is equivalent to addingnoise to the combined local outputs (Nmid) From theresults of this study it is not possible to determinewhether the limiting noise should be Nearly or Nmid inFigure 6 however Nmid seems more plausible as thelevel of early noise needed to exceed the external noiseintroduced by the randomly oriented elements in thenoise check stimuli would be very high In the best-fitting prediction to the noise check data (TA1) thestandard deviation of the early noise would be 85 ofthe mean response of the local detector to its idealstimulus in the best-fitting prediction to the blankcheck data (TA2) prediction it would be even higher(147) There is also the possibility that the limitinginternal noise could be multiplicative rather than theadditive noise implemented in the modeling here Weshall address this question through the use of anequivalent noise paradigm in future work

Keywords orientation summation integration tex-ture perception computational modeling

Acknowledgments

This work was supported by a grant from theNatural Sciences and Engineering Research Council(Canada) awarded to Robert Hess (46528-11) and agrant from the Engineering and Physical SciencesResearch Council (UK) awarded to Tim Meese andMark Georgeson (EPH0000381) The authors wouldlike to thank three anonymous reviewers for theirhelpful criticisms and suggestions

Commercial relationships noneCorresponding author Alex S BaldwinEmail alexsbaldwingooglemailcomAddress School of Life and Health Sciences AstonUniversity Birmingham UK

References

Baker D H amp Meese T S 2011 Contrastintegration over area is extensive A three-stagemodel of spatial summation Journal of Vision11(14)14 1ndash16 httpwwwjournalofvisionorgcontent111414 doi101167111414 [PubMed][Article]

Brainard D H 1997 The Psychophysics ToolboxSpatial Vision 10(4) 433ndash436

Dakin S C 2001 Information limit on the spatialintegration of local orientation signals Journal ofthe Optical Society of America A 18(5) 1016ndash1026

Dakin S C Bex P J Cass J R amp Watt R J 2009Dissociable effects of attention and crowding onorientation averaging Journal of Vision 9(11)281ndash16 httpwwwjournalofvisionorgcontent91128 doi10116791128 [PubMed] [Article]

Dakin S C amp Watt R J 1997 The computation oforientation statistics from visual texture VisionResearch 37(22) 3181ndash3192

De Valois R L amp De Valois K K 1990 Striatecortex In Spatial Vision (pp 94ndash146) Oxford UKOxford University Press

Husk J S Huang P-C amp Hess R F 2012Orientation coherence sensitivity Journal of Vision126(18) 1ndash15 httpwwwjournalofvisionorgcontent12618 doi10116712618 [PubMed][Article]

Jones D G Anderson N D amp Murphy K M 2003Orientation discrimination in visual noise usingglobal and local stimuli Vision Research 43 1223ndash1233

Kingdom F A Keeble D amp Moulden B 1995Sensitivity to orientation modulation in micro-pattern-based textures Vision Research 35(1) 79ndash91

Kingdom F A amp Keeble D R 1999 On themechanism for scale invariance in orientation-defined textures Vision Research 39(8) 1477ndash1489

Kleiner M Brainard D H amp Pelli D G 2007Whatrsquos new in Psychtoolbox-3 Perception 36(ECVP Abstract Supplement)

Landy M S amp Graham N 2004 Visual perceptionof texture In L M Chalupa amp J S Werner (Eds)The visual neurosciences (pp 1106ndash1118) Cam-bridge MA MIT Press

Levi D M Hariharan S amp Klein S A 2002Suppressive and facilitatory spatial interactions inperipheral vision Peripheral crowding is neithersize invariant nor simple contrast masking Journalof Vision 2 167ndash177 httpwwwjournalofvisionorgcontent223 doi101167223 [PubMed][Article]

Marr D 1982 Vision San Francisco W H Freemanand Company

Meese T S 2010 Spatially extensive summation ofcontrast energy is revealed by contrast detection ofmicro-pattern textures Journal of Vision 10(8)141ndash21 httpwwwjournalofvisionorgcontent10814 doi10116710814 [PubMed] [Article]

Meese T S amp Summers R J 2007 Area summationin human vision at and above detection thresholdProceedings of the Royal Society B 274(1627)2891ndash2900

Nichols M J amp Newsome W T 2002 Middle

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 14

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

information from every element in the stimulus with anequal weighting regardless of whether it is a noise orpotential signal element The model with this strategypredicts that within each condition (noise check andblank check) there should be no effect of check size ormodulator phase The predicted coherence thresholdfor the noise check stimuli is approximately twice thatfor the full stimulus (Figure 3a) which is equivalent torequiring the same threshold proportion of coherentsignal elements across the whole stimulus For theblank check stimuli (Figure 3b) the predicted thresholdelevation is a factor of

ffiffiffi2p

as there are no interstitialnoise elements to limit performance in that conditionThese predictions capture the performance for thesmall check sizes (1 to 3) in the noise check conditionthe medium check sizes (5 to 9) in the blank checkcondition and the largest frac14 2708 size (15) in both thenoise check and blank check conditions (see Figure 3aand b) The predictions fail however to describe thegeneral form of either set of results which both featurea transition from greater to lesser summation as thecheck size increases

The lsquolsquosum ideallyrsquorsquo (SI) strategy involves the combi-nation of orientation information only from potentialsignal elements This is equivalent to weighting theelements according to their expected signal-to-noiseratio as the responses to the elements in the noiseregions provide zero signal Models with this strategypredict threshold elevation of a factor of approximatelyffiffiffi

2p

for both of the checked conditions (Figure 3c andd) which is identical to the prediction for the blankcheck stimuli with the SA model For this reason theresults in the blank check condition which were welldescribed by SA models are fit just as well by the SImodels In addition the SI models predict the

ffiffiffi2p

summation for the medium check sizes in the noisecheck condition However this model also fails tocapture the general form of the human data for eitherstimulus type

Two-stage hybrid models

The fact that the simpler candidate models featuringthe two different pooling strategies (SA and SI) eachpredicted performance for different subsets of theresults suggested that a more complete account couldbe provided by a model that combined their twobehaviors In the lsquolsquotwo-stage (TS) hybrid modelsrsquorsquomandatory local combination over a k middot k region(behaving like the SA model) is followed by flexiblepooling of the outputs from those regions weighted bytheir expected signal-to-noise ratios which wereapplied as a template (as in the SI model) The size ofthe local pooling region (k) was the only fittedparameter in this model The best fit to the noise check

data was provided by a k of 3 although the fit providedby a k of 2 was only marginally worse (see Appendix BFigure A2) In the noise check condition this modelpredicts an initial doubling of the coherence thresholdfor the small check sizes followed by an improvementin performance to approach a factor of

ffiffiffi2p

for themedium and large check sizes (see Figure 3e) Thiscaptures the performance for all but the largest (15)check sizes In the blank check condition (Figure 3f)the predictions are once again the same as for modelswith the SA and SI strategies

Internal noise

The SA SI and TS models all make predictions thatare identical to each other for the blank checkcondition This is because there are no interstitial noiseelements in that condition that can be inappropriatelypooled to elevate the coherence threshold The resultshowever show a doubling of threshold for the smallcheck sizes that is the same in the noise check and blankcheck conditions This is a larger performance deficitthan can be accounted for by any of the three models(which all predict a

ffiffiffi2p

threshold increase for the blankcheck condition) A two-stage hybrid model featuringadditive Gaussian internal noise at each location (TN)does however predict similar performance for theblank check conditions as for the noise check condi-tions (see Figure 4a and b) The internal noise in themodel is added after the rectification stage (meaningthat the noisy local outputs can be negative) Becauseof the mandatory combination rule at the first poolingstage this early noise model is equivalent to a model inwhich performance is limited by noise affecting theoutput of the first pooling stage

The standard deviation of the internal noise (rint) isnow an additional parameter in the model It isexpressed as a percentage of the summed element-wisefilter response to a matched log-Gabor element (ierelative to the maximum local filter output) As rint isincreased from zero it initially has the effect ofincreasing thresholds for the 1 middot 1 blank checkcondition to bring them in line with those from thenoise check condition Once these have becomeequivalent higher values of rint serve mainly to increaseor decrease the sensitivity for all conditions Calculat-ing the relative coherence thresholds from the output ofthe model therefore largely factors out the effects ofthis model parameter once rint is sufficiently high (seeAppendix B) Figure 4a and b shows the TN modelprediction with the best-fitting k and rint values for theblank check data (note that the best-fitting k for theblank check data is 2 whereas kfrac14 3 provided amarginally better fit to the noise check data as shown inFigure A2) For both the noise check and the blank

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 10

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

check conditions the TN model predicts an initialtwofold summation for the 1 middot 1 check size which thendecreases to approximately

ffiffiffi2p

as the check sizeincreases The predictions from this model account forthe average human thresholds in the noise checkcondition for all but the largest check size (Figure 4a)In the blank check condition (Figure 4b) thresholds arelower than those predicted by the model for themedium (9) and largest (15) frac14 908 condition but forall other conditions the human behavior is captured bythe model

Maximum integration aperture

In the TN model there are several reasons whycoherence thresholds could be elevated when signal ispresented only in the periphery (compare frac14 908 and2708 thresholds for the largest check size stimulus inFigure 4) The simplest would be if the observer wereonly able to pool information from elements in thecenter of the display This was tested using a lsquolsquotwo-stagehybrid model with internal noise and maximumintegration aperturersquorsquo (TA) in which the modelobserver only had access to information from elementsthat were within a central a middot a element squareaperture (equivalent to a degrees or 4a carrier cycles)This would also be equivalent to a model that featureda step-edge decline in sensitivity at this eccentricityMore complex accounts of the effect of visual fieldeccentricity are not explored here as our stimuli weredesigned to factor out these effects when possiblemeaning that any explanation we might provide wouldnot be well constrained by our data For examplefurther summation may be achieved by probabilitysummation between multiple integration apertures atdifferent locations across the visual field (resulting in amodel similar to that proposed for the area summationof contrast by Baker amp Meese 2011)

In order to show which features of our results the TAmodel could and could not account for two separatefits were performed to determine the optimal set ofparameters for the noise check (TA1) and the blankcheck (TA2) data The TA1 prediction is shown inFigure 4c and d The best-fitting integration aperturesize was 18 middot 18 degrees (compared to other squareapertures with integer dimensions see Appendix B)The TA1 model prediction is similar to that made bythe TN model (for the same kernel size see AppendixB) for all conditions except for the largest check size(15) For the 15 middot 15 check stimuli the predictedthresholds for the frac14 908 stimulus are reduced (whenexpressed relative to the full condition threshold) andthe thresholds for the frac14 2708 stimulus are elevatedThis model prediction provides a good fit to the noisecheck data (RMSefrac14 054 dB) and for the smaller check

sizes (1ndash3) in the blank check condition For the largercheck sizes in the blank check condition however theTA1 model prediction systematically underestimatesthe sensitivity of the human observers (resulting in arelatively large RMSe of 178 dB)

Fitting the aperture model to the blank checkcondition produces the TA2 prediction shown in Figure4e and f As in the TN model the best-fitting k for theblank check condition was 2 (as opposed to the valuefound from the TA1 fit to the noise check data whichfavored a k of 3) The best-fitting aperture size (a) was19 middot 19 degrees (18 wider than the size found by fittingto the noise check data) As would be expected thisversion of the model provides an inferior fit to the noisecheck data compared to the TA1 prediction (116 dB vs054 dB) underestimating both the threshold for the 3middot 3 check condition and the amount of separationbetween the two modulator phases at the largest checksize In the blank check condition although the qualityof the fit is improved from the TA1 prediction (112 vs178 dB) sensitivity is still underestimated for the larger(9ndash15) check sizes This indicates that there is nocombination of parameters for this model architecturethat can capture the performance for those conditions

Residual effects of check condition andmodulator phase

Figure 5 replots the data from Figure 2c and dshowing directly the threshold elevation effect that theinterstitial noise elements in the noise check conditionhave Also shown in Figure 5 are the thresholdelevation predictions from each of the six modelvariants presented earlier All models except SA and TSpredict no threshold elevation from the noise checks ineither modulator phase For the frac14 908 modulatorphase the data agree with this prediction with the ratiobetween the noise check and blank check thresholdsremaining close to one for all check sizes For the frac142708 modulator phase there is no threshold elevationintroduced by the noise checks at the smallest (1) checksize but as the check size increases the noise checkshave the effect of raising thresholds relative to theblank check condition up to a maximum factor of

ffiffiffi2p

for the largest (15) check size Only the SA and TSmodels have their performance limited by the intersti-tial randomly oriented elements in the noise checkstimuli and therefore predict threshold elevation in thenoise check condition The pattern of elevationpredicted however does not match that seen in the frac142708 data The key difference is that the data show athreshold elevation effect that both increases withcheck size and is dependent on the modulator phasewhereas none of the models investigated here make thisprediction One possible explanation for this effect

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 11

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

would be an increase in the size of the mandatory localintegration region with eccentricity (linking this to theexplanation of crowding provided by Parkes LundAngelucci Solomon amp Morgan 2001)

Discussion

Orientation integration is a noisy two-stageprocess

The results of this study suggest that the combina-tion of orientation information over space is a noisytwo-stage process (Figure 6) The key results reportedhere can be accounted for by a model that performsmandatory local integration affected by internal noiseat each location followed by flexible pooling over theoutputs from those regions (for the vector-averagingversion of the model these would seem equivalent tothe involuntary and voluntary averaging discussed byDakin Bex Cass amp Watt 2009) This account is inagreement with previous studies that have found lowerthresholds for stimuli with a greater signal area (Dakin2001 Jones et al 2003) The effect found for thearrangement of the elements in the display contradictsthe flexibility attributed to the pooling of local samplesby Dakin (2001) although it is possible that thedifferences in terms of the task set to the observer(signal in noise here vs fine discrimination in Dakin)would mean that they did not investigate the samesignal-combination process (as discussed in the Intro-duction to this paper) The close spacing of theelements in our stimuli and their extension into theperiphery would lead us to expect the individualelements to be crowded by each other Under theaccount provided by Dakin et al (2009) it is suggestedthat crowding only limits performance in tasks in whichthere is little orientation variability (eg a finediscrimination task) whereas for a signal-in-noise tasksuch as ours performance should be limited by thenumber of pooled samples If crowding were a limit onattentional resolution (Strasburger 2005) howeverthen we would expect it to affect the ability of ourobservers to segregate potential signal and noiseelements possibly explaining the elevated thresholds inthe frac142708 noise check condition In addition it is notpossible from our study to determine whether theobservers were pooling all of the available samples inthe stimulus or if they were making their decisionsbased on a subset of the local samples in the display (aswas found by Dakin 2001)

It is not entirely clear how the results presented herecan be reconciled with those of Husk et al (2012) whofound no summation with increasing signal area forsimilar stimuli The main difference between the stimuli

used in the two studies is that this study used theBattenberg summation paradigm whereas Husk et alincreased the signal area of their stimuli by increasingdiameter It is possible that a combination of decreas-ing sensitivity for the local orientation-discriminationtask and increasing the mandatory summation regionsize with eccentricity might flatten the threshold versusarea functions This question shall be addressed infuture work which will combine the Battenbergstimulus paradigm used here with a conventional area-summation design comparing stimuli of differentdiameters Performing this experiment at a variety ofspatial frequencies will also allow us to determinewhether the size of the mandatory integration region islinked to the scale of the elements that are beingpooled Previous studies that have investigated theprocessing of visual texture would lead us to predictthis to be the case (Kingdom Keeble amp Moulden1995 Kingdom amp Keeble 1999) however if themandatory pooling were related to crowding we wouldexpect its extent to be independent of the scale of thestimulus (Levi Hariharan amp Klein 2002)

It is noteworthy that the results presented in Figure2c and d bear a resemblance to those found in thecontrast Battenberg study by Meese (2010) with short-range linear summation followed by long-range square-law summation The explanation for the square-lawsummation differs however between the two studiesFor the orientation result presented here this nonlinearsummation is explained by a flexible pooling strategythat segregates out the input from the nonsignal regionsof the display whereas in the contrast study by Meesethe output from those regions was included with thatfrom the signal regions in the global combinationprocess and the nonlinear summation was accountedfor by a square-law transducer

The nature of the limiting internal noise

Figure 6 shows three possible locations for thelimiting noise in the modeling for this study (NearlyNmid or Nlate) The similarity of the results from thenoise check and blank check conditions indicate thatthe limiting noise is not driven by the response toindividual elements Instead they suggest that a level ofinternal noise is pooled that is proportional to thenumber of locations being monitored which includesblank locations that are being integrated at the firststage of the model Jones et al (2003) included lsquolsquolatenoisersquorsquo when modeling data from an orientationcoherence experiment that used filtered noise as stimuliIn the modeling for that study the noise was constantfor different signal areas (this would be Nlate in Figure6) A model based on dominant noise at Nlate wouldmake the same prediction as the SA model shown in

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 12

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

Figure 3a for both the noise check and blank checkconditions Such a model would not explain the resultspresented here as the improvement in performance seenfor the medium check size stimuli requires that theobserver is able to segregate out the limiting noise inirrelevant regions from the second combination stageIf dominant noise is contributed from each monitored

location then this could be performed by weighting thelocal outputs according to a template w as shown inFigure 6

In the internal noise model developed in this study(see Figure 4) the noise was implemented at eachpooled location after the initial filtering stage (Nearly)however due to the mandatory local combination at

Figure 6 Diagram of the TN model This diagram shows how the lsquolsquoverticalrsquorsquo response to an example stimulus (which includes blank

spaces in stimulus region n) is determined by filtering with a vertical filter element mandatory local summation and then global

summation of the local outputs (from stimulus regions 1 to n) weighted by the expected signal-to-noise ratio at each location (w) For

ease of presentation the local summation region shown is 1 middot 4 elements (our results suggest 3 middot 3) and only the responses to the

first and last stimulus regions are presented The lsquolsquohorizontalrsquorsquo response would be calculated in an identical manner except with a

horizontal filter element at the convolution stage Nearly Nmid and Nlate show three possible locations for the limiting internal noise

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 13

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

the first combination stage this is equivalent to addingnoise to the combined local outputs (Nmid) From theresults of this study it is not possible to determinewhether the limiting noise should be Nearly or Nmid inFigure 6 however Nmid seems more plausible as thelevel of early noise needed to exceed the external noiseintroduced by the randomly oriented elements in thenoise check stimuli would be very high In the best-fitting prediction to the noise check data (TA1) thestandard deviation of the early noise would be 85 ofthe mean response of the local detector to its idealstimulus in the best-fitting prediction to the blankcheck data (TA2) prediction it would be even higher(147) There is also the possibility that the limitinginternal noise could be multiplicative rather than theadditive noise implemented in the modeling here Weshall address this question through the use of anequivalent noise paradigm in future work

Keywords orientation summation integration tex-ture perception computational modeling

Acknowledgments

This work was supported by a grant from theNatural Sciences and Engineering Research Council(Canada) awarded to Robert Hess (46528-11) and agrant from the Engineering and Physical SciencesResearch Council (UK) awarded to Tim Meese andMark Georgeson (EPH0000381) The authors wouldlike to thank three anonymous reviewers for theirhelpful criticisms and suggestions

Commercial relationships noneCorresponding author Alex S BaldwinEmail alexsbaldwingooglemailcomAddress School of Life and Health Sciences AstonUniversity Birmingham UK

References

Baker D H amp Meese T S 2011 Contrastintegration over area is extensive A three-stagemodel of spatial summation Journal of Vision11(14)14 1ndash16 httpwwwjournalofvisionorgcontent111414 doi101167111414 [PubMed][Article]

Brainard D H 1997 The Psychophysics ToolboxSpatial Vision 10(4) 433ndash436

Dakin S C 2001 Information limit on the spatialintegration of local orientation signals Journal ofthe Optical Society of America A 18(5) 1016ndash1026

Dakin S C Bex P J Cass J R amp Watt R J 2009Dissociable effects of attention and crowding onorientation averaging Journal of Vision 9(11)281ndash16 httpwwwjournalofvisionorgcontent91128 doi10116791128 [PubMed] [Article]

Dakin S C amp Watt R J 1997 The computation oforientation statistics from visual texture VisionResearch 37(22) 3181ndash3192

De Valois R L amp De Valois K K 1990 Striatecortex In Spatial Vision (pp 94ndash146) Oxford UKOxford University Press

Husk J S Huang P-C amp Hess R F 2012Orientation coherence sensitivity Journal of Vision126(18) 1ndash15 httpwwwjournalofvisionorgcontent12618 doi10116712618 [PubMed][Article]

Jones D G Anderson N D amp Murphy K M 2003Orientation discrimination in visual noise usingglobal and local stimuli Vision Research 43 1223ndash1233

Kingdom F A Keeble D amp Moulden B 1995Sensitivity to orientation modulation in micro-pattern-based textures Vision Research 35(1) 79ndash91

Kingdom F A amp Keeble D R 1999 On themechanism for scale invariance in orientation-defined textures Vision Research 39(8) 1477ndash1489

Kleiner M Brainard D H amp Pelli D G 2007Whatrsquos new in Psychtoolbox-3 Perception 36(ECVP Abstract Supplement)

Landy M S amp Graham N 2004 Visual perceptionof texture In L M Chalupa amp J S Werner (Eds)The visual neurosciences (pp 1106ndash1118) Cam-bridge MA MIT Press

Levi D M Hariharan S amp Klein S A 2002Suppressive and facilitatory spatial interactions inperipheral vision Peripheral crowding is neithersize invariant nor simple contrast masking Journalof Vision 2 167ndash177 httpwwwjournalofvisionorgcontent223 doi101167223 [PubMed][Article]

Marr D 1982 Vision San Francisco W H Freemanand Company

Meese T S 2010 Spatially extensive summation ofcontrast energy is revealed by contrast detection ofmicro-pattern textures Journal of Vision 10(8)141ndash21 httpwwwjournalofvisionorgcontent10814 doi10116710814 [PubMed] [Article]

Meese T S amp Summers R J 2007 Area summationin human vision at and above detection thresholdProceedings of the Royal Society B 274(1627)2891ndash2900

Nichols M J amp Newsome W T 2002 Middle

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 14

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

check conditions the TN model predicts an initialtwofold summation for the 1 middot 1 check size which thendecreases to approximately

ffiffiffi2p

as the check sizeincreases The predictions from this model account forthe average human thresholds in the noise checkcondition for all but the largest check size (Figure 4a)In the blank check condition (Figure 4b) thresholds arelower than those predicted by the model for themedium (9) and largest (15) frac14 908 condition but forall other conditions the human behavior is captured bythe model

Maximum integration aperture

In the TN model there are several reasons whycoherence thresholds could be elevated when signal ispresented only in the periphery (compare frac14 908 and2708 thresholds for the largest check size stimulus inFigure 4) The simplest would be if the observer wereonly able to pool information from elements in thecenter of the display This was tested using a lsquolsquotwo-stagehybrid model with internal noise and maximumintegration aperturersquorsquo (TA) in which the modelobserver only had access to information from elementsthat were within a central a middot a element squareaperture (equivalent to a degrees or 4a carrier cycles)This would also be equivalent to a model that featureda step-edge decline in sensitivity at this eccentricityMore complex accounts of the effect of visual fieldeccentricity are not explored here as our stimuli weredesigned to factor out these effects when possiblemeaning that any explanation we might provide wouldnot be well constrained by our data For examplefurther summation may be achieved by probabilitysummation between multiple integration apertures atdifferent locations across the visual field (resulting in amodel similar to that proposed for the area summationof contrast by Baker amp Meese 2011)

In order to show which features of our results the TAmodel could and could not account for two separatefits were performed to determine the optimal set ofparameters for the noise check (TA1) and the blankcheck (TA2) data The TA1 prediction is shown inFigure 4c and d The best-fitting integration aperturesize was 18 middot 18 degrees (compared to other squareapertures with integer dimensions see Appendix B)The TA1 model prediction is similar to that made bythe TN model (for the same kernel size see AppendixB) for all conditions except for the largest check size(15) For the 15 middot 15 check stimuli the predictedthresholds for the frac14 908 stimulus are reduced (whenexpressed relative to the full condition threshold) andthe thresholds for the frac14 2708 stimulus are elevatedThis model prediction provides a good fit to the noisecheck data (RMSefrac14 054 dB) and for the smaller check

sizes (1ndash3) in the blank check condition For the largercheck sizes in the blank check condition however theTA1 model prediction systematically underestimatesthe sensitivity of the human observers (resulting in arelatively large RMSe of 178 dB)

Fitting the aperture model to the blank checkcondition produces the TA2 prediction shown in Figure4e and f As in the TN model the best-fitting k for theblank check condition was 2 (as opposed to the valuefound from the TA1 fit to the noise check data whichfavored a k of 3) The best-fitting aperture size (a) was19 middot 19 degrees (18 wider than the size found by fittingto the noise check data) As would be expected thisversion of the model provides an inferior fit to the noisecheck data compared to the TA1 prediction (116 dB vs054 dB) underestimating both the threshold for the 3middot 3 check condition and the amount of separationbetween the two modulator phases at the largest checksize In the blank check condition although the qualityof the fit is improved from the TA1 prediction (112 vs178 dB) sensitivity is still underestimated for the larger(9ndash15) check sizes This indicates that there is nocombination of parameters for this model architecturethat can capture the performance for those conditions

Residual effects of check condition andmodulator phase

Figure 5 replots the data from Figure 2c and dshowing directly the threshold elevation effect that theinterstitial noise elements in the noise check conditionhave Also shown in Figure 5 are the thresholdelevation predictions from each of the six modelvariants presented earlier All models except SA and TSpredict no threshold elevation from the noise checks ineither modulator phase For the frac14 908 modulatorphase the data agree with this prediction with the ratiobetween the noise check and blank check thresholdsremaining close to one for all check sizes For the frac142708 modulator phase there is no threshold elevationintroduced by the noise checks at the smallest (1) checksize but as the check size increases the noise checkshave the effect of raising thresholds relative to theblank check condition up to a maximum factor of

ffiffiffi2p

for the largest (15) check size Only the SA and TSmodels have their performance limited by the intersti-tial randomly oriented elements in the noise checkstimuli and therefore predict threshold elevation in thenoise check condition The pattern of elevationpredicted however does not match that seen in the frac142708 data The key difference is that the data show athreshold elevation effect that both increases withcheck size and is dependent on the modulator phasewhereas none of the models investigated here make thisprediction One possible explanation for this effect

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 11

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

would be an increase in the size of the mandatory localintegration region with eccentricity (linking this to theexplanation of crowding provided by Parkes LundAngelucci Solomon amp Morgan 2001)

Discussion

Orientation integration is a noisy two-stageprocess

The results of this study suggest that the combina-tion of orientation information over space is a noisytwo-stage process (Figure 6) The key results reportedhere can be accounted for by a model that performsmandatory local integration affected by internal noiseat each location followed by flexible pooling over theoutputs from those regions (for the vector-averagingversion of the model these would seem equivalent tothe involuntary and voluntary averaging discussed byDakin Bex Cass amp Watt 2009) This account is inagreement with previous studies that have found lowerthresholds for stimuli with a greater signal area (Dakin2001 Jones et al 2003) The effect found for thearrangement of the elements in the display contradictsthe flexibility attributed to the pooling of local samplesby Dakin (2001) although it is possible that thedifferences in terms of the task set to the observer(signal in noise here vs fine discrimination in Dakin)would mean that they did not investigate the samesignal-combination process (as discussed in the Intro-duction to this paper) The close spacing of theelements in our stimuli and their extension into theperiphery would lead us to expect the individualelements to be crowded by each other Under theaccount provided by Dakin et al (2009) it is suggestedthat crowding only limits performance in tasks in whichthere is little orientation variability (eg a finediscrimination task) whereas for a signal-in-noise tasksuch as ours performance should be limited by thenumber of pooled samples If crowding were a limit onattentional resolution (Strasburger 2005) howeverthen we would expect it to affect the ability of ourobservers to segregate potential signal and noiseelements possibly explaining the elevated thresholds inthe frac142708 noise check condition In addition it is notpossible from our study to determine whether theobservers were pooling all of the available samples inthe stimulus or if they were making their decisionsbased on a subset of the local samples in the display (aswas found by Dakin 2001)

It is not entirely clear how the results presented herecan be reconciled with those of Husk et al (2012) whofound no summation with increasing signal area forsimilar stimuli The main difference between the stimuli

used in the two studies is that this study used theBattenberg summation paradigm whereas Husk et alincreased the signal area of their stimuli by increasingdiameter It is possible that a combination of decreas-ing sensitivity for the local orientation-discriminationtask and increasing the mandatory summation regionsize with eccentricity might flatten the threshold versusarea functions This question shall be addressed infuture work which will combine the Battenbergstimulus paradigm used here with a conventional area-summation design comparing stimuli of differentdiameters Performing this experiment at a variety ofspatial frequencies will also allow us to determinewhether the size of the mandatory integration region islinked to the scale of the elements that are beingpooled Previous studies that have investigated theprocessing of visual texture would lead us to predictthis to be the case (Kingdom Keeble amp Moulden1995 Kingdom amp Keeble 1999) however if themandatory pooling were related to crowding we wouldexpect its extent to be independent of the scale of thestimulus (Levi Hariharan amp Klein 2002)

It is noteworthy that the results presented in Figure2c and d bear a resemblance to those found in thecontrast Battenberg study by Meese (2010) with short-range linear summation followed by long-range square-law summation The explanation for the square-lawsummation differs however between the two studiesFor the orientation result presented here this nonlinearsummation is explained by a flexible pooling strategythat segregates out the input from the nonsignal regionsof the display whereas in the contrast study by Meesethe output from those regions was included with thatfrom the signal regions in the global combinationprocess and the nonlinear summation was accountedfor by a square-law transducer

The nature of the limiting internal noise

Figure 6 shows three possible locations for thelimiting noise in the modeling for this study (NearlyNmid or Nlate) The similarity of the results from thenoise check and blank check conditions indicate thatthe limiting noise is not driven by the response toindividual elements Instead they suggest that a level ofinternal noise is pooled that is proportional to thenumber of locations being monitored which includesblank locations that are being integrated at the firststage of the model Jones et al (2003) included lsquolsquolatenoisersquorsquo when modeling data from an orientationcoherence experiment that used filtered noise as stimuliIn the modeling for that study the noise was constantfor different signal areas (this would be Nlate in Figure6) A model based on dominant noise at Nlate wouldmake the same prediction as the SA model shown in

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 12

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

Figure 3a for both the noise check and blank checkconditions Such a model would not explain the resultspresented here as the improvement in performance seenfor the medium check size stimuli requires that theobserver is able to segregate out the limiting noise inirrelevant regions from the second combination stageIf dominant noise is contributed from each monitored

location then this could be performed by weighting thelocal outputs according to a template w as shown inFigure 6

In the internal noise model developed in this study(see Figure 4) the noise was implemented at eachpooled location after the initial filtering stage (Nearly)however due to the mandatory local combination at

Figure 6 Diagram of the TN model This diagram shows how the lsquolsquoverticalrsquorsquo response to an example stimulus (which includes blank

spaces in stimulus region n) is determined by filtering with a vertical filter element mandatory local summation and then global

summation of the local outputs (from stimulus regions 1 to n) weighted by the expected signal-to-noise ratio at each location (w) For

ease of presentation the local summation region shown is 1 middot 4 elements (our results suggest 3 middot 3) and only the responses to the

first and last stimulus regions are presented The lsquolsquohorizontalrsquorsquo response would be calculated in an identical manner except with a

horizontal filter element at the convolution stage Nearly Nmid and Nlate show three possible locations for the limiting internal noise

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 13

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

the first combination stage this is equivalent to addingnoise to the combined local outputs (Nmid) From theresults of this study it is not possible to determinewhether the limiting noise should be Nearly or Nmid inFigure 6 however Nmid seems more plausible as thelevel of early noise needed to exceed the external noiseintroduced by the randomly oriented elements in thenoise check stimuli would be very high In the best-fitting prediction to the noise check data (TA1) thestandard deviation of the early noise would be 85 ofthe mean response of the local detector to its idealstimulus in the best-fitting prediction to the blankcheck data (TA2) prediction it would be even higher(147) There is also the possibility that the limitinginternal noise could be multiplicative rather than theadditive noise implemented in the modeling here Weshall address this question through the use of anequivalent noise paradigm in future work

Keywords orientation summation integration tex-ture perception computational modeling

Acknowledgments

This work was supported by a grant from theNatural Sciences and Engineering Research Council(Canada) awarded to Robert Hess (46528-11) and agrant from the Engineering and Physical SciencesResearch Council (UK) awarded to Tim Meese andMark Georgeson (EPH0000381) The authors wouldlike to thank three anonymous reviewers for theirhelpful criticisms and suggestions

Commercial relationships noneCorresponding author Alex S BaldwinEmail alexsbaldwingooglemailcomAddress School of Life and Health Sciences AstonUniversity Birmingham UK

References

Baker D H amp Meese T S 2011 Contrastintegration over area is extensive A three-stagemodel of spatial summation Journal of Vision11(14)14 1ndash16 httpwwwjournalofvisionorgcontent111414 doi101167111414 [PubMed][Article]

Brainard D H 1997 The Psychophysics ToolboxSpatial Vision 10(4) 433ndash436

Dakin S C 2001 Information limit on the spatialintegration of local orientation signals Journal ofthe Optical Society of America A 18(5) 1016ndash1026

Dakin S C Bex P J Cass J R amp Watt R J 2009Dissociable effects of attention and crowding onorientation averaging Journal of Vision 9(11)281ndash16 httpwwwjournalofvisionorgcontent91128 doi10116791128 [PubMed] [Article]

Dakin S C amp Watt R J 1997 The computation oforientation statistics from visual texture VisionResearch 37(22) 3181ndash3192

De Valois R L amp De Valois K K 1990 Striatecortex In Spatial Vision (pp 94ndash146) Oxford UKOxford University Press

Husk J S Huang P-C amp Hess R F 2012Orientation coherence sensitivity Journal of Vision126(18) 1ndash15 httpwwwjournalofvisionorgcontent12618 doi10116712618 [PubMed][Article]

Jones D G Anderson N D amp Murphy K M 2003Orientation discrimination in visual noise usingglobal and local stimuli Vision Research 43 1223ndash1233

Kingdom F A Keeble D amp Moulden B 1995Sensitivity to orientation modulation in micro-pattern-based textures Vision Research 35(1) 79ndash91

Kingdom F A amp Keeble D R 1999 On themechanism for scale invariance in orientation-defined textures Vision Research 39(8) 1477ndash1489

Kleiner M Brainard D H amp Pelli D G 2007Whatrsquos new in Psychtoolbox-3 Perception 36(ECVP Abstract Supplement)

Landy M S amp Graham N 2004 Visual perceptionof texture In L M Chalupa amp J S Werner (Eds)The visual neurosciences (pp 1106ndash1118) Cam-bridge MA MIT Press

Levi D M Hariharan S amp Klein S A 2002Suppressive and facilitatory spatial interactions inperipheral vision Peripheral crowding is neithersize invariant nor simple contrast masking Journalof Vision 2 167ndash177 httpwwwjournalofvisionorgcontent223 doi101167223 [PubMed][Article]

Marr D 1982 Vision San Francisco W H Freemanand Company

Meese T S 2010 Spatially extensive summation ofcontrast energy is revealed by contrast detection ofmicro-pattern textures Journal of Vision 10(8)141ndash21 httpwwwjournalofvisionorgcontent10814 doi10116710814 [PubMed] [Article]

Meese T S amp Summers R J 2007 Area summationin human vision at and above detection thresholdProceedings of the Royal Society B 274(1627)2891ndash2900

Nichols M J amp Newsome W T 2002 Middle

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 14

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

would be an increase in the size of the mandatory localintegration region with eccentricity (linking this to theexplanation of crowding provided by Parkes LundAngelucci Solomon amp Morgan 2001)

Discussion

Orientation integration is a noisy two-stageprocess

The results of this study suggest that the combina-tion of orientation information over space is a noisytwo-stage process (Figure 6) The key results reportedhere can be accounted for by a model that performsmandatory local integration affected by internal noiseat each location followed by flexible pooling over theoutputs from those regions (for the vector-averagingversion of the model these would seem equivalent tothe involuntary and voluntary averaging discussed byDakin Bex Cass amp Watt 2009) This account is inagreement with previous studies that have found lowerthresholds for stimuli with a greater signal area (Dakin2001 Jones et al 2003) The effect found for thearrangement of the elements in the display contradictsthe flexibility attributed to the pooling of local samplesby Dakin (2001) although it is possible that thedifferences in terms of the task set to the observer(signal in noise here vs fine discrimination in Dakin)would mean that they did not investigate the samesignal-combination process (as discussed in the Intro-duction to this paper) The close spacing of theelements in our stimuli and their extension into theperiphery would lead us to expect the individualelements to be crowded by each other Under theaccount provided by Dakin et al (2009) it is suggestedthat crowding only limits performance in tasks in whichthere is little orientation variability (eg a finediscrimination task) whereas for a signal-in-noise tasksuch as ours performance should be limited by thenumber of pooled samples If crowding were a limit onattentional resolution (Strasburger 2005) howeverthen we would expect it to affect the ability of ourobservers to segregate potential signal and noiseelements possibly explaining the elevated thresholds inthe frac142708 noise check condition In addition it is notpossible from our study to determine whether theobservers were pooling all of the available samples inthe stimulus or if they were making their decisionsbased on a subset of the local samples in the display (aswas found by Dakin 2001)

It is not entirely clear how the results presented herecan be reconciled with those of Husk et al (2012) whofound no summation with increasing signal area forsimilar stimuli The main difference between the stimuli

used in the two studies is that this study used theBattenberg summation paradigm whereas Husk et alincreased the signal area of their stimuli by increasingdiameter It is possible that a combination of decreas-ing sensitivity for the local orientation-discriminationtask and increasing the mandatory summation regionsize with eccentricity might flatten the threshold versusarea functions This question shall be addressed infuture work which will combine the Battenbergstimulus paradigm used here with a conventional area-summation design comparing stimuli of differentdiameters Performing this experiment at a variety ofspatial frequencies will also allow us to determinewhether the size of the mandatory integration region islinked to the scale of the elements that are beingpooled Previous studies that have investigated theprocessing of visual texture would lead us to predictthis to be the case (Kingdom Keeble amp Moulden1995 Kingdom amp Keeble 1999) however if themandatory pooling were related to crowding we wouldexpect its extent to be independent of the scale of thestimulus (Levi Hariharan amp Klein 2002)

It is noteworthy that the results presented in Figure2c and d bear a resemblance to those found in thecontrast Battenberg study by Meese (2010) with short-range linear summation followed by long-range square-law summation The explanation for the square-lawsummation differs however between the two studiesFor the orientation result presented here this nonlinearsummation is explained by a flexible pooling strategythat segregates out the input from the nonsignal regionsof the display whereas in the contrast study by Meesethe output from those regions was included with thatfrom the signal regions in the global combinationprocess and the nonlinear summation was accountedfor by a square-law transducer

The nature of the limiting internal noise

Figure 6 shows three possible locations for thelimiting noise in the modeling for this study (NearlyNmid or Nlate) The similarity of the results from thenoise check and blank check conditions indicate thatthe limiting noise is not driven by the response toindividual elements Instead they suggest that a level ofinternal noise is pooled that is proportional to thenumber of locations being monitored which includesblank locations that are being integrated at the firststage of the model Jones et al (2003) included lsquolsquolatenoisersquorsquo when modeling data from an orientationcoherence experiment that used filtered noise as stimuliIn the modeling for that study the noise was constantfor different signal areas (this would be Nlate in Figure6) A model based on dominant noise at Nlate wouldmake the same prediction as the SA model shown in

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 12

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

Figure 3a for both the noise check and blank checkconditions Such a model would not explain the resultspresented here as the improvement in performance seenfor the medium check size stimuli requires that theobserver is able to segregate out the limiting noise inirrelevant regions from the second combination stageIf dominant noise is contributed from each monitored

location then this could be performed by weighting thelocal outputs according to a template w as shown inFigure 6

In the internal noise model developed in this study(see Figure 4) the noise was implemented at eachpooled location after the initial filtering stage (Nearly)however due to the mandatory local combination at

Figure 6 Diagram of the TN model This diagram shows how the lsquolsquoverticalrsquorsquo response to an example stimulus (which includes blank

spaces in stimulus region n) is determined by filtering with a vertical filter element mandatory local summation and then global

summation of the local outputs (from stimulus regions 1 to n) weighted by the expected signal-to-noise ratio at each location (w) For

ease of presentation the local summation region shown is 1 middot 4 elements (our results suggest 3 middot 3) and only the responses to the

first and last stimulus regions are presented The lsquolsquohorizontalrsquorsquo response would be calculated in an identical manner except with a

horizontal filter element at the convolution stage Nearly Nmid and Nlate show three possible locations for the limiting internal noise

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 13

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

the first combination stage this is equivalent to addingnoise to the combined local outputs (Nmid) From theresults of this study it is not possible to determinewhether the limiting noise should be Nearly or Nmid inFigure 6 however Nmid seems more plausible as thelevel of early noise needed to exceed the external noiseintroduced by the randomly oriented elements in thenoise check stimuli would be very high In the best-fitting prediction to the noise check data (TA1) thestandard deviation of the early noise would be 85 ofthe mean response of the local detector to its idealstimulus in the best-fitting prediction to the blankcheck data (TA2) prediction it would be even higher(147) There is also the possibility that the limitinginternal noise could be multiplicative rather than theadditive noise implemented in the modeling here Weshall address this question through the use of anequivalent noise paradigm in future work

Keywords orientation summation integration tex-ture perception computational modeling

Acknowledgments

This work was supported by a grant from theNatural Sciences and Engineering Research Council(Canada) awarded to Robert Hess (46528-11) and agrant from the Engineering and Physical SciencesResearch Council (UK) awarded to Tim Meese andMark Georgeson (EPH0000381) The authors wouldlike to thank three anonymous reviewers for theirhelpful criticisms and suggestions

Commercial relationships noneCorresponding author Alex S BaldwinEmail alexsbaldwingooglemailcomAddress School of Life and Health Sciences AstonUniversity Birmingham UK

References

Baker D H amp Meese T S 2011 Contrastintegration over area is extensive A three-stagemodel of spatial summation Journal of Vision11(14)14 1ndash16 httpwwwjournalofvisionorgcontent111414 doi101167111414 [PubMed][Article]

Brainard D H 1997 The Psychophysics ToolboxSpatial Vision 10(4) 433ndash436

Dakin S C 2001 Information limit on the spatialintegration of local orientation signals Journal ofthe Optical Society of America A 18(5) 1016ndash1026

Dakin S C Bex P J Cass J R amp Watt R J 2009Dissociable effects of attention and crowding onorientation averaging Journal of Vision 9(11)281ndash16 httpwwwjournalofvisionorgcontent91128 doi10116791128 [PubMed] [Article]

Dakin S C amp Watt R J 1997 The computation oforientation statistics from visual texture VisionResearch 37(22) 3181ndash3192

De Valois R L amp De Valois K K 1990 Striatecortex In Spatial Vision (pp 94ndash146) Oxford UKOxford University Press

Husk J S Huang P-C amp Hess R F 2012Orientation coherence sensitivity Journal of Vision126(18) 1ndash15 httpwwwjournalofvisionorgcontent12618 doi10116712618 [PubMed][Article]

Jones D G Anderson N D amp Murphy K M 2003Orientation discrimination in visual noise usingglobal and local stimuli Vision Research 43 1223ndash1233

Kingdom F A Keeble D amp Moulden B 1995Sensitivity to orientation modulation in micro-pattern-based textures Vision Research 35(1) 79ndash91

Kingdom F A amp Keeble D R 1999 On themechanism for scale invariance in orientation-defined textures Vision Research 39(8) 1477ndash1489

Kleiner M Brainard D H amp Pelli D G 2007Whatrsquos new in Psychtoolbox-3 Perception 36(ECVP Abstract Supplement)

Landy M S amp Graham N 2004 Visual perceptionof texture In L M Chalupa amp J S Werner (Eds)The visual neurosciences (pp 1106ndash1118) Cam-bridge MA MIT Press

Levi D M Hariharan S amp Klein S A 2002Suppressive and facilitatory spatial interactions inperipheral vision Peripheral crowding is neithersize invariant nor simple contrast masking Journalof Vision 2 167ndash177 httpwwwjournalofvisionorgcontent223 doi101167223 [PubMed][Article]

Marr D 1982 Vision San Francisco W H Freemanand Company

Meese T S 2010 Spatially extensive summation ofcontrast energy is revealed by contrast detection ofmicro-pattern textures Journal of Vision 10(8)141ndash21 httpwwwjournalofvisionorgcontent10814 doi10116710814 [PubMed] [Article]

Meese T S amp Summers R J 2007 Area summationin human vision at and above detection thresholdProceedings of the Royal Society B 274(1627)2891ndash2900

Nichols M J amp Newsome W T 2002 Middle

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 14

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

Figure 3a for both the noise check and blank checkconditions Such a model would not explain the resultspresented here as the improvement in performance seenfor the medium check size stimuli requires that theobserver is able to segregate out the limiting noise inirrelevant regions from the second combination stageIf dominant noise is contributed from each monitored

location then this could be performed by weighting thelocal outputs according to a template w as shown inFigure 6

In the internal noise model developed in this study(see Figure 4) the noise was implemented at eachpooled location after the initial filtering stage (Nearly)however due to the mandatory local combination at

Figure 6 Diagram of the TN model This diagram shows how the lsquolsquoverticalrsquorsquo response to an example stimulus (which includes blank

spaces in stimulus region n) is determined by filtering with a vertical filter element mandatory local summation and then global

summation of the local outputs (from stimulus regions 1 to n) weighted by the expected signal-to-noise ratio at each location (w) For

ease of presentation the local summation region shown is 1 middot 4 elements (our results suggest 3 middot 3) and only the responses to the

first and last stimulus regions are presented The lsquolsquohorizontalrsquorsquo response would be calculated in an identical manner except with a

horizontal filter element at the convolution stage Nearly Nmid and Nlate show three possible locations for the limiting internal noise

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 13

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

the first combination stage this is equivalent to addingnoise to the combined local outputs (Nmid) From theresults of this study it is not possible to determinewhether the limiting noise should be Nearly or Nmid inFigure 6 however Nmid seems more plausible as thelevel of early noise needed to exceed the external noiseintroduced by the randomly oriented elements in thenoise check stimuli would be very high In the best-fitting prediction to the noise check data (TA1) thestandard deviation of the early noise would be 85 ofthe mean response of the local detector to its idealstimulus in the best-fitting prediction to the blankcheck data (TA2) prediction it would be even higher(147) There is also the possibility that the limitinginternal noise could be multiplicative rather than theadditive noise implemented in the modeling here Weshall address this question through the use of anequivalent noise paradigm in future work

Keywords orientation summation integration tex-ture perception computational modeling

Acknowledgments

This work was supported by a grant from theNatural Sciences and Engineering Research Council(Canada) awarded to Robert Hess (46528-11) and agrant from the Engineering and Physical SciencesResearch Council (UK) awarded to Tim Meese andMark Georgeson (EPH0000381) The authors wouldlike to thank three anonymous reviewers for theirhelpful criticisms and suggestions

Commercial relationships noneCorresponding author Alex S BaldwinEmail alexsbaldwingooglemailcomAddress School of Life and Health Sciences AstonUniversity Birmingham UK

References

Baker D H amp Meese T S 2011 Contrastintegration over area is extensive A three-stagemodel of spatial summation Journal of Vision11(14)14 1ndash16 httpwwwjournalofvisionorgcontent111414 doi101167111414 [PubMed][Article]

Brainard D H 1997 The Psychophysics ToolboxSpatial Vision 10(4) 433ndash436

Dakin S C 2001 Information limit on the spatialintegration of local orientation signals Journal ofthe Optical Society of America A 18(5) 1016ndash1026

Dakin S C Bex P J Cass J R amp Watt R J 2009Dissociable effects of attention and crowding onorientation averaging Journal of Vision 9(11)281ndash16 httpwwwjournalofvisionorgcontent91128 doi10116791128 [PubMed] [Article]

Dakin S C amp Watt R J 1997 The computation oforientation statistics from visual texture VisionResearch 37(22) 3181ndash3192

De Valois R L amp De Valois K K 1990 Striatecortex In Spatial Vision (pp 94ndash146) Oxford UKOxford University Press

Husk J S Huang P-C amp Hess R F 2012Orientation coherence sensitivity Journal of Vision126(18) 1ndash15 httpwwwjournalofvisionorgcontent12618 doi10116712618 [PubMed][Article]

Jones D G Anderson N D amp Murphy K M 2003Orientation discrimination in visual noise usingglobal and local stimuli Vision Research 43 1223ndash1233

Kingdom F A Keeble D amp Moulden B 1995Sensitivity to orientation modulation in micro-pattern-based textures Vision Research 35(1) 79ndash91

Kingdom F A amp Keeble D R 1999 On themechanism for scale invariance in orientation-defined textures Vision Research 39(8) 1477ndash1489

Kleiner M Brainard D H amp Pelli D G 2007Whatrsquos new in Psychtoolbox-3 Perception 36(ECVP Abstract Supplement)

Landy M S amp Graham N 2004 Visual perceptionof texture In L M Chalupa amp J S Werner (Eds)The visual neurosciences (pp 1106ndash1118) Cam-bridge MA MIT Press

Levi D M Hariharan S amp Klein S A 2002Suppressive and facilitatory spatial interactions inperipheral vision Peripheral crowding is neithersize invariant nor simple contrast masking Journalof Vision 2 167ndash177 httpwwwjournalofvisionorgcontent223 doi101167223 [PubMed][Article]

Marr D 1982 Vision San Francisco W H Freemanand Company

Meese T S 2010 Spatially extensive summation ofcontrast energy is revealed by contrast detection ofmicro-pattern textures Journal of Vision 10(8)141ndash21 httpwwwjournalofvisionorgcontent10814 doi10116710814 [PubMed] [Article]

Meese T S amp Summers R J 2007 Area summationin human vision at and above detection thresholdProceedings of the Royal Society B 274(1627)2891ndash2900

Nichols M J amp Newsome W T 2002 Middle

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 14

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

the first combination stage this is equivalent to addingnoise to the combined local outputs (Nmid) From theresults of this study it is not possible to determinewhether the limiting noise should be Nearly or Nmid inFigure 6 however Nmid seems more plausible as thelevel of early noise needed to exceed the external noiseintroduced by the randomly oriented elements in thenoise check stimuli would be very high In the best-fitting prediction to the noise check data (TA1) thestandard deviation of the early noise would be 85 ofthe mean response of the local detector to its idealstimulus in the best-fitting prediction to the blankcheck data (TA2) prediction it would be even higher(147) There is also the possibility that the limitinginternal noise could be multiplicative rather than theadditive noise implemented in the modeling here Weshall address this question through the use of anequivalent noise paradigm in future work

Keywords orientation summation integration tex-ture perception computational modeling

Acknowledgments

This work was supported by a grant from theNatural Sciences and Engineering Research Council(Canada) awarded to Robert Hess (46528-11) and agrant from the Engineering and Physical SciencesResearch Council (UK) awarded to Tim Meese andMark Georgeson (EPH0000381) The authors wouldlike to thank three anonymous reviewers for theirhelpful criticisms and suggestions

Commercial relationships noneCorresponding author Alex S BaldwinEmail alexsbaldwingooglemailcomAddress School of Life and Health Sciences AstonUniversity Birmingham UK

References

Baker D H amp Meese T S 2011 Contrastintegration over area is extensive A three-stagemodel of spatial summation Journal of Vision11(14)14 1ndash16 httpwwwjournalofvisionorgcontent111414 doi101167111414 [PubMed][Article]

Brainard D H 1997 The Psychophysics ToolboxSpatial Vision 10(4) 433ndash436

Dakin S C 2001 Information limit on the spatialintegration of local orientation signals Journal ofthe Optical Society of America A 18(5) 1016ndash1026

Dakin S C Bex P J Cass J R amp Watt R J 2009Dissociable effects of attention and crowding onorientation averaging Journal of Vision 9(11)281ndash16 httpwwwjournalofvisionorgcontent91128 doi10116791128 [PubMed] [Article]

Dakin S C amp Watt R J 1997 The computation oforientation statistics from visual texture VisionResearch 37(22) 3181ndash3192

De Valois R L amp De Valois K K 1990 Striatecortex In Spatial Vision (pp 94ndash146) Oxford UKOxford University Press

Husk J S Huang P-C amp Hess R F 2012Orientation coherence sensitivity Journal of Vision126(18) 1ndash15 httpwwwjournalofvisionorgcontent12618 doi10116712618 [PubMed][Article]

Jones D G Anderson N D amp Murphy K M 2003Orientation discrimination in visual noise usingglobal and local stimuli Vision Research 43 1223ndash1233

Kingdom F A Keeble D amp Moulden B 1995Sensitivity to orientation modulation in micro-pattern-based textures Vision Research 35(1) 79ndash91

Kingdom F A amp Keeble D R 1999 On themechanism for scale invariance in orientation-defined textures Vision Research 39(8) 1477ndash1489

Kleiner M Brainard D H amp Pelli D G 2007Whatrsquos new in Psychtoolbox-3 Perception 36(ECVP Abstract Supplement)

Landy M S amp Graham N 2004 Visual perceptionof texture In L M Chalupa amp J S Werner (Eds)The visual neurosciences (pp 1106ndash1118) Cam-bridge MA MIT Press

Levi D M Hariharan S amp Klein S A 2002Suppressive and facilitatory spatial interactions inperipheral vision Peripheral crowding is neithersize invariant nor simple contrast masking Journalof Vision 2 167ndash177 httpwwwjournalofvisionorgcontent223 doi101167223 [PubMed][Article]

Marr D 1982 Vision San Francisco W H Freemanand Company

Meese T S 2010 Spatially extensive summation ofcontrast energy is revealed by contrast detection ofmicro-pattern textures Journal of Vision 10(8)141ndash21 httpwwwjournalofvisionorgcontent10814 doi10116710814 [PubMed] [Article]

Meese T S amp Summers R J 2007 Area summationin human vision at and above detection thresholdProceedings of the Royal Society B 274(1627)2891ndash2900

Nichols M J amp Newsome W T 2002 Middle

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 14

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

temporal visual area microstimulation influencesveridical judgements of motion direction TheJournal of Neuroscience 22(21) 9530ndash9540

Parkes L Lund J Angelucci A Solomon J A ampMorgan M 2001 Compulsory averaging ofcrowded orientation signals in human visionNature Neuroscience 4(7) 739ndash744

Prins N amp Kingdom F A A 2009 PalamedesMatlab routines for analyzing psychophysical datawwwpalamedestoolboxorg

Sagi D 1990 Detection of an orientation singularity inGabor textures Effect of signal density and spatialfrequency Vision Research 30(9) 1377ndash1388

Salzman C D amp Newsome W T 1994 Neuralmechanisms for forming a perceptual decisionScience 264 231ndash237

Strasburger H 2005 Unfocused spatial attentionunderlies the crowding effect in indirect formvision Journal of Vision 5(11)8 1024ndash1037 httpwwwjournalofvisionorgcontent5118 doi1011675118 [PubMed] [Article]

Vorhees H amp Poggio T 1988 Computing textureboundaries from images Nature 333(26) 364ndash367

Webb B S Ledgeway T amp McGraw P V 2007Cortical pooling algorithms for judging globalmotion direction Proceedings of the NationalAcademy of Sciences USA 104(9) 3532ndash3537

Webb B S Ledgeway T amp McGraw P V 2010Relating spatial and temporal orientation poolingto population decoding solutions in human visionVision Research 50 2274ndash2283

Appendix A Identification task biasdistributions

Figure A1 shows the distribution of the responsebiases found in the analysis of the data from theidentification task Across all observers and conditionsthe mean bias was466 with a standard deviation of749 This means that on average the observers wereas likely to respond lsquolsquohorizontalrsquorsquo as they were torespond lsquolsquoverticalrsquorsquo to a stimulus with 466 verticalcoherence Looking at the data from the two mainconditions (noise check and blank check) for eachobserver individually most observers showed a signif-icant response bias as decided by a one-sample t test(Table A1) For all observers except one the directionof any significant bias was consistent across the twoconditions For ASB the bias changed directionbetween the noise check and blank check conditionpossibly due to this observer being tested on theseconditions in two different labs several weeks apartAside from ASB the observers tested in each labshowed the same mixture of biases from horizontal(LFE and RJS) through mostly unbiased (SAW andVRP) to vertical (DHB and JSH)

Appendix B Determination ofparameter values

Predictions were generated from the TS model with arange of different kernel sizes (k) for the local

Figure A1 Distributions of response biases from each observer calculated as the level at which the fitted psychometric functions were

at 50 Bias values were pooled across all repetitions in all of the subconditions tested in each of the two main conditions noise

check versus blank check (shown in red and blue respectively) Statistical properties of the distributions are reported in Table A1

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 15

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

mandatory integration stage (1 middot 1 to 7 middot 7 in integersteps) The model curves produced for the noise checkcondition and the RMS errors between those curvesand the data are shown in Figure A2a and b For theblank check condition the model curves were identicalfor any kernel size producing the same predictions asthe SA and SI models The best-fitting value for the kparameter is 3 Smaller k values predict too littlesummation for the small check sizes whereas larger kvalues predict too much summation for the larger checksizes

The predicted blank check condition thresholdsgenerated by the TN model for a kernel size of 2 and arange of different internal noise levels (rint) are shownin Figure A3 Predictions with a kernel size of 3 areshown in Figure A4 As the internal noise levelincreases the prediction for the blank check conditionbecomes similar to that for the noise check conditionIn conditions of high internal noise the blank checkdata are best fit by a kernel size (k) of 2

Contour plots of the RMS error between the noisecheck data and predictions from the TA model are

Observer Condition Mean SD df t p Sig

ASB Noise check 552 363 47 1054 0001

Blank check 1277 760 47 1164 0001

DHB Noise check 154 484 47 220 0033

Blank check 329 459 47 496 0001

JSH Noise check 632 415 47 1054 0001

Blank check 589 655 47 623 0001

LFE Noise check 871 472 47 1277 0001

Blank check 1277 760 47 253 0015

RJS Noise check 390 856 47 315 0003

Blank check 542 624 47 601 0001

SAW Noise check 196 608 47 224 0030

Blank check 002 549 47 003 0976

VRP Noise check 097 788 47 085 0398

Blank check 016 619 47 018 0859

Table A1 Statistical properties of the bias distributions for each observer and major condition (noise check vs blank check) Notes Aone-sample t test was performed for each in order to determine whether any bias present was significant

Figure A2 Panel (a) shows model predictions for the two-stage hybrid model with a range of different kernel sizes (k) against the data

from the noise check condition (replotted from Figure 2c) Panel (b) shows the RMS error between the model predictions and the

data for each value of k

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 16

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

Figure A3 Panel (a) shows model predictions for the noisy two-stage hybrid model with a kernel size (k) of 2 and a range of different

internal noise levels (rint) against the data from the blank check condition (replotted from Figure 2d) Panel (b) shows the RMS error

between the model predictions and the data for each value of rint

Figure A4 Panel (a) shows model predictions for the noisy two-stage hybrid model with a kernel size (k) of 3 and a range of different

internal noise levels (rint) against the data from the blank check condition (replotted from Figure 2d) Panel (b) shows the RMS error

between the model predictions and the data for each value of rint

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 17

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

Figure A5 RMS error between the predictions from the TA model and the data for different combinations of the kernel size (k)

internal noise (rint) and aperture size (a) parameters The minimum on each plot is indicated by a lsquolsquothornrsquorsquo The minimum across the two

graphs (RMSe frac14 054 dB when k frac14 3 rint frac14 352 and a frac14 18) is the TA1 model prediction shown in Figure 4c

Figure A6 RMS error between the predictions from the TA model and the data for different combinations of the kernel size (k)

internal noise (rint) and aperture size (a) parameters The minimum on each plot is indicated by a lsquolsquothornrsquorsquo The minimum across the two

graphs (RMSe frac14 112 dB when k frac14 2 rint frac14 608 and a frac14 19) is the TA2 model prediction shown in Figure 4f

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 18

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

shown in Figure A5 The parameters that produce theglobal minimum across the two surfaces are used togenerate the TA1 prediction shown in Figure 4c and dThe RMS errors between the blank check data andpredictions from the TA model are shown in FigureA6 The parameters that produce the global minimumfrom these fits are used to generate the TA2 predictionshown in Figure 4e and f

Appendix C Vector-averagingmodel

In the vector-averaging model (Dakin amp Watt1997) each pooled element is represented as a vectorwith magnitude mi and orientation hi It is assumed thatthe observer is able to extract the orientation of eachelement (in the model this is implemented by taking theorientations directly from the stimulus-generationprocedure) These are then combined using vector-averaging to get the average orientation

havg frac141

2tan1

Xxy

mxysin 2hxy

Xxy

mxycos 2hxy

0BB

1CCA eth2THORN

Note that the local orientations are doubled beforeaveraging and that the output of the vector-averagingoperation is halved This wraps the orientations at 1808(rather than at 3608 which is the usual limit) becauseeach element in the display is symmetrical across itsmajor and minor axes (a 908 element is identical to a2708 element)

When every element is weighted equally all elementsare represented by unit vectors (mxy frac14 1) In cases inwhich the elements have different expected signal-to-noise ratios (eg the second stage of the two-stagehybrid model below) the magnitudes of the localvectors are each weighted by a template to control thecontribution each local vector makes to the calculatedaverage (see the section on pooling strategies) Themodel then picks the potential target orientation closestto the calculated average orientation The predictionsfrom vector-averaging models using the differentpooling strategies given are shown in Figure A7athrough d These predictions have the same form asthose made by the filter-maxing combination process inFigure 3

For the hybrid model the convolution kernel tosimulate mandatory local combination is applied to thevector components

K frac141

keth

1 k1 1

1

1 1 1THORN eth3THORN

Asin x yfrac12 frac14 mxysin hxy eth4THORN

Acos x yfrac12 frac14 mxycos hxy eth5THORN

Tsin frac14 AsinK eth6THORN

Tcos frac14 AcosK eth7THORN

These lsquolsquoblurredrsquorsquo components are then combinedthrough vector-averaging

havg frac141

2tan1

Xxy

Tsin

Xxy

Tcos

0BB

1CCA eth8THORN

Predictions from this vector-averaging hybrid modelare shown in Figure A7e and f The kernel size (k) waschosen to be the same as that used in the filter-maxingmodel A comparison between the predictions made bythe vector-averaging two-stage hybrid model and theequivalent filter-maxing combination model (Figure 3eand f) shows them to be very similar

The internal noise model adds a sample of zero-mean(l frac14 0) Gaussian noise to each element drawn from adistribution with the requested standard deviation (rint)

N x yfrac12 frac14 Nethl rintTHORN eth9THORN

havg frac141

2tan1

Xxy

Tsin thorn N1

Xxy

Tcos thorn N2

0BB

1CCA eth10THORN

Predictions from this model and the version of themodel with a maximum integration aperture are shownin Figure A8 In each case the parameters were set tobe the same as those used in the filter-maxing models(Figure 4) except for the internal noise parameter (rint)which was set to a value that produced a prediction thatwas close to the one from the equivalent filter-maxingmodel (determined by fitting the vector-averagingmodel to the filter-maxing model and choosing thevalue of rint that gave the lowest RMS error) In allcases the vector-averaging model was able to producesimilar behavior to that seen in the filter-maxing model

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 19

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

Figure A7 Coherence thresholds plotted with predictions from the vector-averaging versions of the SA (andashb) SI (cndashd) and HM (endashf)

models Each row shows the same averaged data replotted from Figure 2c and d with the threshold for each check condition

expressed as a multiple of that for the full stimulus

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 20

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

Figure A8 Coherence thresholds plotted with predictions from the vector-averaging version of the TN model and from the vector-

averaging version of the TA model with the best-fitting aperture and kernel size parameters from the filter-maxing model fits to the

noise check (cndashd) and blank check (endashf) data

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 21

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

• Introduction
• Methods
• e01
• t01
• f01
• Results
• f02
• f03
• Modeling
• f04
• f05
• Discussion
• f06
• Baker1
• Brainard1
• Dakin1
• Dakin2
• Dakin3
• DeValois1
• Husk1
• Jones1
• Kingdom1
• Kingdom2
• Kleiner1
• Landy1
• Levi1
• Marr1
• Meese1
• Meese2
• Nichols1
• Parkes1
• Prins1
• Sagi1
• Salzman1
• Strasburger1
• Vorhees1
• Webb1
• Webb2
• Appendix A Identification task bias
• Appendix B Determination of parameter
• f07
• t02
• f08
• f09
• f10
• f11
• f12
• Appendix C Vector-averaging model
• e02
• e03
• e04
• e05
• e06
• e07
• e08
• e09
• e10
• f13
• f14

Figure A3 Panel (a) shows model predictions for the noisy two-stage hybrid model with a kernel size (k) of 2 and a range of different

internal noise levels (rint) against the data from the blank check condition (replotted from Figure 2d) Panel (b) shows the RMS error

between the model predictions and the data for each value of rint

Figure A4 Panel (a) shows model predictions for the noisy two-stage hybrid model with a kernel size (k) of 3 and a range of different

internal noise levels (rint) against the data from the blank check condition (replotted from Figure 2d) Panel (b) shows the RMS error

between the model predictions and the data for each value of rint

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 17

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

Figure A5 RMS error between the predictions from the TA model and the data for different combinations of the kernel size (k)

internal noise (rint) and aperture size (a) parameters The minimum on each plot is indicated by a lsquolsquothornrsquorsquo The minimum across the two

graphs (RMSe frac14 054 dB when k frac14 3 rint frac14 352 and a frac14 18) is the TA1 model prediction shown in Figure 4c

Figure A6 RMS error between the predictions from the TA model and the data for different combinations of the kernel size (k)

internal noise (rint) and aperture size (a) parameters The minimum on each plot is indicated by a lsquolsquothornrsquorsquo The minimum across the two

graphs (RMSe frac14 112 dB when k frac14 2 rint frac14 608 and a frac14 19) is the TA2 model prediction shown in Figure 4f

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 18

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

shown in Figure A5 The parameters that produce theglobal minimum across the two surfaces are used togenerate the TA1 prediction shown in Figure 4c and dThe RMS errors between the blank check data andpredictions from the TA model are shown in FigureA6 The parameters that produce the global minimumfrom these fits are used to generate the TA2 predictionshown in Figure 4e and f

Appendix C Vector-averagingmodel

In the vector-averaging model (Dakin amp Watt1997) each pooled element is represented as a vectorwith magnitude mi and orientation hi It is assumed thatthe observer is able to extract the orientation of eachelement (in the model this is implemented by taking theorientations directly from the stimulus-generationprocedure) These are then combined using vector-averaging to get the average orientation

havg frac141

2tan1

Xxy

mxysin 2hxy

Xxy

mxycos 2hxy

0BB

1CCA eth2THORN

Note that the local orientations are doubled beforeaveraging and that the output of the vector-averagingoperation is halved This wraps the orientations at 1808(rather than at 3608 which is the usual limit) becauseeach element in the display is symmetrical across itsmajor and minor axes (a 908 element is identical to a2708 element)

When every element is weighted equally all elementsare represented by unit vectors (mxy frac14 1) In cases inwhich the elements have different expected signal-to-noise ratios (eg the second stage of the two-stagehybrid model below) the magnitudes of the localvectors are each weighted by a template to control thecontribution each local vector makes to the calculatedaverage (see the section on pooling strategies) Themodel then picks the potential target orientation closestto the calculated average orientation The predictionsfrom vector-averaging models using the differentpooling strategies given are shown in Figure A7athrough d These predictions have the same form asthose made by the filter-maxing combination process inFigure 3

For the hybrid model the convolution kernel tosimulate mandatory local combination is applied to thevector components

K frac141

keth

1 k1 1

1

1 1 1THORN eth3THORN

Asin x yfrac12 frac14 mxysin hxy eth4THORN

Acos x yfrac12 frac14 mxycos hxy eth5THORN

Tsin frac14 AsinK eth6THORN

Tcos frac14 AcosK eth7THORN

These lsquolsquoblurredrsquorsquo components are then combinedthrough vector-averaging

havg frac141

2tan1

Xxy

Tsin

Xxy

Tcos

0BB

1CCA eth8THORN

Predictions from this vector-averaging hybrid modelare shown in Figure A7e and f The kernel size (k) waschosen to be the same as that used in the filter-maxingmodel A comparison between the predictions made bythe vector-averaging two-stage hybrid model and theequivalent filter-maxing combination model (Figure 3eand f) shows them to be very similar

The internal noise model adds a sample of zero-mean(l frac14 0) Gaussian noise to each element drawn from adistribution with the requested standard deviation (rint)

N x yfrac12 frac14 Nethl rintTHORN eth9THORN

havg frac141

2tan1

Xxy

Tsin thorn N1

Xxy

Tcos thorn N2

0BB

1CCA eth10THORN

Predictions from this model and the version of themodel with a maximum integration aperture are shownin Figure A8 In each case the parameters were set tobe the same as those used in the filter-maxing models(Figure 4) except for the internal noise parameter (rint)which was set to a value that produced a prediction thatwas close to the one from the equivalent filter-maxingmodel (determined by fitting the vector-averagingmodel to the filter-maxing model and choosing thevalue of rint that gave the lowest RMS error) In allcases the vector-averaging model was able to producesimilar behavior to that seen in the filter-maxing model

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 19

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

Figure A7 Coherence thresholds plotted with predictions from the vector-averaging versions of the SA (andashb) SI (cndashd) and HM (endashf)

models Each row shows the same averaged data replotted from Figure 2c and d with the threshold for each check condition

expressed as a multiple of that for the full stimulus

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 20

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

Figure A8 Coherence thresholds plotted with predictions from the vector-averaging version of the TN model and from the vector-

averaging version of the TA model with the best-fitting aperture and kernel size parameters from the filter-maxing model fits to the

noise check (cndashd) and blank check (endashf) data

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 21

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

• Introduction
• Methods
• e01
• t01
• f01
• Results
• f02
• f03
• Modeling
• f04
• f05
• Discussion
• f06
• Baker1
• Brainard1
• Dakin1
• Dakin2
• Dakin3
• DeValois1
• Husk1
• Jones1
• Kingdom1
• Kingdom2
• Kleiner1
• Landy1
• Levi1
• Marr1
• Meese1
• Meese2
• Nichols1
• Parkes1
• Prins1
• Sagi1
• Salzman1
• Strasburger1
• Vorhees1
• Webb1
• Webb2
• Appendix A Identification task bias
• Appendix B Determination of parameter
• f07
• t02
• f08
• f09
• f10
• f11
• f12
• Appendix C Vector-averaging model
• e02
• e03
• e04
• e05
• e06
• e07
• e08
• e09
• e10
• f13
• f14

Figure A5 RMS error between the predictions from the TA model and the data for different combinations of the kernel size (k)

internal noise (rint) and aperture size (a) parameters The minimum on each plot is indicated by a lsquolsquothornrsquorsquo The minimum across the two

graphs (RMSe frac14 054 dB when k frac14 3 rint frac14 352 and a frac14 18) is the TA1 model prediction shown in Figure 4c

Figure A6 RMS error between the predictions from the TA model and the data for different combinations of the kernel size (k)

internal noise (rint) and aperture size (a) parameters The minimum on each plot is indicated by a lsquolsquothornrsquorsquo The minimum across the two

graphs (RMSe frac14 112 dB when k frac14 2 rint frac14 608 and a frac14 19) is the TA2 model prediction shown in Figure 4f

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 18

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

shown in Figure A5 The parameters that produce theglobal minimum across the two surfaces are used togenerate the TA1 prediction shown in Figure 4c and dThe RMS errors between the blank check data andpredictions from the TA model are shown in FigureA6 The parameters that produce the global minimumfrom these fits are used to generate the TA2 predictionshown in Figure 4e and f

Appendix C Vector-averagingmodel

In the vector-averaging model (Dakin amp Watt1997) each pooled element is represented as a vectorwith magnitude mi and orientation hi It is assumed thatthe observer is able to extract the orientation of eachelement (in the model this is implemented by taking theorientations directly from the stimulus-generationprocedure) These are then combined using vector-averaging to get the average orientation

havg frac141

2tan1

Xxy

mxysin 2hxy

Xxy

mxycos 2hxy

0BB

1CCA eth2THORN

Note that the local orientations are doubled beforeaveraging and that the output of the vector-averagingoperation is halved This wraps the orientations at 1808(rather than at 3608 which is the usual limit) becauseeach element in the display is symmetrical across itsmajor and minor axes (a 908 element is identical to a2708 element)

When every element is weighted equally all elementsare represented by unit vectors (mxy frac14 1) In cases inwhich the elements have different expected signal-to-noise ratios (eg the second stage of the two-stagehybrid model below) the magnitudes of the localvectors are each weighted by a template to control thecontribution each local vector makes to the calculatedaverage (see the section on pooling strategies) Themodel then picks the potential target orientation closestto the calculated average orientation The predictionsfrom vector-averaging models using the differentpooling strategies given are shown in Figure A7athrough d These predictions have the same form asthose made by the filter-maxing combination process inFigure 3

For the hybrid model the convolution kernel tosimulate mandatory local combination is applied to thevector components

K frac141

keth

1 k1 1

1

1 1 1THORN eth3THORN

Asin x yfrac12 frac14 mxysin hxy eth4THORN

Acos x yfrac12 frac14 mxycos hxy eth5THORN

Tsin frac14 AsinK eth6THORN

Tcos frac14 AcosK eth7THORN

These lsquolsquoblurredrsquorsquo components are then combinedthrough vector-averaging

havg frac141

2tan1

Xxy

Tsin

Xxy

Tcos

0BB

1CCA eth8THORN

Predictions from this vector-averaging hybrid modelare shown in Figure A7e and f The kernel size (k) waschosen to be the same as that used in the filter-maxingmodel A comparison between the predictions made bythe vector-averaging two-stage hybrid model and theequivalent filter-maxing combination model (Figure 3eand f) shows them to be very similar

The internal noise model adds a sample of zero-mean(l frac14 0) Gaussian noise to each element drawn from adistribution with the requested standard deviation (rint)

N x yfrac12 frac14 Nethl rintTHORN eth9THORN

havg frac141

2tan1

Xxy

Tsin thorn N1

Xxy

Tcos thorn N2

0BB

1CCA eth10THORN

Predictions from this model and the version of themodel with a maximum integration aperture are shownin Figure A8 In each case the parameters were set tobe the same as those used in the filter-maxing models(Figure 4) except for the internal noise parameter (rint)which was set to a value that produced a prediction thatwas close to the one from the equivalent filter-maxingmodel (determined by fitting the vector-averagingmodel to the filter-maxing model and choosing thevalue of rint that gave the lowest RMS error) In allcases the vector-averaging model was able to producesimilar behavior to that seen in the filter-maxing model

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 19

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

Figure A7 Coherence thresholds plotted with predictions from the vector-averaging versions of the SA (andashb) SI (cndashd) and HM (endashf)

models Each row shows the same averaged data replotted from Figure 2c and d with the threshold for each check condition

expressed as a multiple of that for the full stimulus

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 20

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

Figure A8 Coherence thresholds plotted with predictions from the vector-averaging version of the TN model and from the vector-

averaging version of the TA model with the best-fitting aperture and kernel size parameters from the filter-maxing model fits to the

noise check (cndashd) and blank check (endashf) data

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 21

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

• Introduction
• Methods
• e01
• t01
• f01
• Results
• f02
• f03
• Modeling
• f04
• f05
• Discussion
• f06
• Baker1
• Brainard1
• Dakin1
• Dakin2
• Dakin3
• DeValois1
• Husk1
• Jones1
• Kingdom1
• Kingdom2
• Kleiner1
• Landy1
• Levi1
• Marr1
• Meese1
• Meese2
• Nichols1
• Parkes1
• Prins1
• Sagi1
• Salzman1
• Strasburger1
• Vorhees1
• Webb1
• Webb2
• Appendix A Identification task bias
• Appendix B Determination of parameter
• f07
• t02
• f08
• f09
• f10
• f11
• f12
• Appendix C Vector-averaging model
• e02
• e03
• e04
• e05
• e06
• e07
• e08
• e09
• e10
• f13
• f14

shown in Figure A5 The parameters that produce theglobal minimum across the two surfaces are used togenerate the TA1 prediction shown in Figure 4c and dThe RMS errors between the blank check data andpredictions from the TA model are shown in FigureA6 The parameters that produce the global minimumfrom these fits are used to generate the TA2 predictionshown in Figure 4e and f

Appendix C Vector-averagingmodel

In the vector-averaging model (Dakin amp Watt1997) each pooled element is represented as a vectorwith magnitude mi and orientation hi It is assumed thatthe observer is able to extract the orientation of eachelement (in the model this is implemented by taking theorientations directly from the stimulus-generationprocedure) These are then combined using vector-averaging to get the average orientation

havg frac141

2tan1

Xxy

mxysin 2hxy

Xxy

mxycos 2hxy

0BB

1CCA eth2THORN

Note that the local orientations are doubled beforeaveraging and that the output of the vector-averagingoperation is halved This wraps the orientations at 1808(rather than at 3608 which is the usual limit) becauseeach element in the display is symmetrical across itsmajor and minor axes (a 908 element is identical to a2708 element)

When every element is weighted equally all elementsare represented by unit vectors (mxy frac14 1) In cases inwhich the elements have different expected signal-to-noise ratios (eg the second stage of the two-stagehybrid model below) the magnitudes of the localvectors are each weighted by a template to control thecontribution each local vector makes to the calculatedaverage (see the section on pooling strategies) Themodel then picks the potential target orientation closestto the calculated average orientation The predictionsfrom vector-averaging models using the differentpooling strategies given are shown in Figure A7athrough d These predictions have the same form asthose made by the filter-maxing combination process inFigure 3

For the hybrid model the convolution kernel tosimulate mandatory local combination is applied to thevector components

K frac141

keth

1 k1 1

1

1 1 1THORN eth3THORN

Asin x yfrac12 frac14 mxysin hxy eth4THORN

Acos x yfrac12 frac14 mxycos hxy eth5THORN

Tsin frac14 AsinK eth6THORN

Tcos frac14 AcosK eth7THORN

These lsquolsquoblurredrsquorsquo components are then combinedthrough vector-averaging

havg frac141

2tan1

Xxy

Tsin

Xxy

Tcos

0BB

1CCA eth8THORN

Predictions from this vector-averaging hybrid modelare shown in Figure A7e and f The kernel size (k) waschosen to be the same as that used in the filter-maxingmodel A comparison between the predictions made bythe vector-averaging two-stage hybrid model and theequivalent filter-maxing combination model (Figure 3eand f) shows them to be very similar

The internal noise model adds a sample of zero-mean(l frac14 0) Gaussian noise to each element drawn from adistribution with the requested standard deviation (rint)

N x yfrac12 frac14 Nethl rintTHORN eth9THORN

havg frac141

2tan1

Xxy

Tsin thorn N1

Xxy

Tcos thorn N2

0BB

1CCA eth10THORN

Predictions from this model and the version of themodel with a maximum integration aperture are shownin Figure A8 In each case the parameters were set tobe the same as those used in the filter-maxing models(Figure 4) except for the internal noise parameter (rint)which was set to a value that produced a prediction thatwas close to the one from the equivalent filter-maxingmodel (determined by fitting the vector-averagingmodel to the filter-maxing model and choosing thevalue of rint that gave the lowest RMS error) In allcases the vector-averaging model was able to producesimilar behavior to that seen in the filter-maxing model

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 19

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

Figure A7 Coherence thresholds plotted with predictions from the vector-averaging versions of the SA (andashb) SI (cndashd) and HM (endashf)

models Each row shows the same averaged data replotted from Figure 2c and d with the threshold for each check condition

expressed as a multiple of that for the full stimulus

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 20

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

Figure A8 Coherence thresholds plotted with predictions from the vector-averaging version of the TN model and from the vector-

averaging version of the TA model with the best-fitting aperture and kernel size parameters from the filter-maxing model fits to the

noise check (cndashd) and blank check (endashf) data

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 21

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

• Introduction
• Methods
• e01
• t01
• f01
• Results
• f02
• f03
• Modeling
• f04
• f05
• Discussion
• f06
• Baker1
• Brainard1
• Dakin1
• Dakin2
• Dakin3
• DeValois1
• Husk1
• Jones1
• Kingdom1
• Kingdom2
• Kleiner1
• Landy1
• Levi1
• Marr1
• Meese1
• Meese2
• Nichols1
• Parkes1
• Prins1
• Sagi1
• Salzman1
• Strasburger1
• Vorhees1
• Webb1
• Webb2
• Appendix A Identification task bias
• Appendix B Determination of parameter
• f07
• t02
• f08
• f09
• f10
• f11
• f12
• Appendix C Vector-averaging model
• e02
• e03
• e04
• e05
• e06
• e07
• e08
• e09
• e10
• f13
• f14

Figure A7 Coherence thresholds plotted with predictions from the vector-averaging versions of the SA (andashb) SI (cndashd) and HM (endashf)

models Each row shows the same averaged data replotted from Figure 2c and d with the threshold for each check condition

expressed as a multiple of that for the full stimulus

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 20

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

Figure A8 Coherence thresholds plotted with predictions from the vector-averaging version of the TN model and from the vector-

averaging version of the TA model with the best-fitting aperture and kernel size parameters from the filter-maxing model fits to the

noise check (cndashd) and blank check (endashf) data

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 21

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

• Introduction
• Methods
• e01
• t01
• f01
• Results
• f02
• f03
• Modeling
• f04
• f05
• Discussion
• f06
• Baker1
• Brainard1
• Dakin1
• Dakin2
• Dakin3
• DeValois1
• Husk1
• Jones1
• Kingdom1
• Kingdom2
• Kleiner1
• Landy1
• Levi1
• Marr1
• Meese1
• Meese2
• Nichols1
• Parkes1
• Prins1
• Sagi1
• Salzman1
• Strasburger1
• Vorhees1
• Webb1
• Webb2
• Appendix A Identification task bias
• Appendix B Determination of parameter
• f07
• t02
• f08
• f09
• f10
• f11
• f12
• Appendix C Vector-averaging model
• e02
• e03
• e04
• e05
• e06
• e07
• e08
• e09
• e10
• f13
• f14

Figure A8 Coherence thresholds plotted with predictions from the vector-averaging version of the TN model and from the vector-

averaging version of the TA model with the best-fitting aperture and kernel size parameters from the filter-maxing model fits to the

noise check (cndashd) and blank check (endashf) data

Journal of Vision (2014) 14(1)30 1ndash21 Baldwin Husk Meese amp Hess 21

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=datajournalsjov933546 on 06272018

• Introduction
• Methods
• e01
• t01
• f01
• Results
• f02
• f03
• Modeling
• f04
• f05
• Discussion
• f06
• Baker1
• Brainard1
• Dakin1
• Dakin2
• Dakin3
• DeValois1
• Husk1
• Jones1
• Kingdom1
• Kingdom2
• Kleiner1
• Landy1
• Levi1
• Marr1
• Meese1
• Meese2
• Nichols1
• Parkes1
• Prins1
• Sagi1
• Salzman1
• Strasburger1
• Vorhees1
• Webb1
• Webb2
• Appendix A Identification task bias
• Appendix B Determination of parameter
• f07
• t02
• f08
• f09
• f10
• f11
• f12
• Appendix C Vector-averaging model
• e02
• e03
• e04
• e05
• e06
• e07
• e08
• e09
• e10
• f13
• f14