Upload
lesley-pitts
View
219
Download
1
Tags:
Embed Size (px)
Citation preview
Modeling Combined Modeling Combined Proximity-Similarity Proximity-Similarity
Effects in Visual SearchEffects in Visual Search
Tamar Avraham*Tamar Avraham*Yaffa Yeshurun**Yaffa Yeshurun**
Michael Lindenbaum*Michael Lindenbaum*
*Computer science dept., Technion, Israel*Computer science dept., Technion, Israel**Psychology dept., Haifa University, Israel**Psychology dept., Haifa University, Israel
OutlineOutline► The FLNN and COVER models for computer visionThe FLNN and COVER models for computer vision
Avraham & Lindenbaum, IEEEAvraham & Lindenbaum, IEEE--PAMI 2006 PAMI 2006
► Study 1:Study 1:
Adapting the models for human performanceAdapting the models for human performanceAvraham, Yeshurun & Lindenbaum , Journal of Vision 2008Avraham, Yeshurun & Lindenbaum , Journal of Vision 2008
► Study 2:Study 2:
Extending the models to account for spatial Extending the models to account for spatial organization effects.organization effects.
The FLNN modelThe FLNN model
feature extraction
ix
Farthest Labeled Nearest Neighbor
feature space
Avraham & Lindenbaum, IEEE-PAMI 2006
0° 30° 60° orientation
T = 0° D = 30°, 60°
T D D
The FLNN model – contThe FLNN model – cont..
1
2
3
4
Avraham & Lindenbaum, IEEE-PAMI 2006
Alternative parallel explanation: dynamic priority map
► Homogeneous distractors Homogeneous distractors
► Clustered distractors Clustered distractors –– maximum one from each maximum one from each clustercluster
Qualitative model behaviorQualitative model behavior
D
T
D
T
D
D
1
2
3
1
2
– pop-out behavior
Avraham & Lindenbaum, IEEE-PAMI 2006
Visual Search DifficultyVisual Search Difficulty►Search difficulty depends on two factors:Search difficulty depends on two factors:
T-D similaritiesT-D similarities D-D similaritiesD-D similarities
►Quantitative measures of search Quantitative measures of search difficultydifficulty the the saliencysaliency measure (measure (Rosenholtz 99Rosenholtz 99)) our our COVERCOVER measure measure
D
T
easy
T
difficult
Duncan & Humphreys 89 similarity theory
Avraham & Lindenbaum, IEEE-PAMI 2006
The COVER measureThe COVER measure
Avraham & Lindenbaum, IEEE-PAMI 2006
► A minimum-d-cover (Kolmogorov 61): the minimum number of spheres with diameter d covering all points
d
The COVER measureThe COVER measure
D
T
dT
D
T
D
DdT
COVER = 1 COVER = 3
T-D similarity effects D-D grouping
Avraham & Lindenbaum, IEEE-PAMI 2006
► If d = dT (the minimum T-D distance),
minimum-d-cover (COVER) the search difficulty
D
T
D
DdT
COVER = 2
COVER = an inherent limitation COVER = an inherent limitation
for all models/algorithmsfor all models/algorithms
FLNN Performance ≥ COVERFLNN Performance ≥ COVER
COVER and FLNNCOVER and FLNN
Avraham & Lindenbaum, IEEE-PAMI 2006
Study 1:Study 1:
Testing the ability of COVER and FLNN Testing the ability of COVER and FLNN to predict human performanceto predict human performance
Avraham, Yeshurun & Lindenbaum, Journal of Vision 2008
►COVER with internal noise: COVER with internal noise:
Low noiseLow noise
Higher noiseHigher noise
►Other visual search modelsOther visual search models Temporal-serial (Temporal-serial (Bergen&Juletz 83’Bergen&Juletz 83’) ) Signal-Detection-Theory (Signal-Detection-Theory (Palmer et. al. 93’, Eckstein 00’Palmer et. al. 93’, Eckstein 00’)) Target-Saliency model Target-Saliency model ((Rosenholtz Rosenholtz 9999’’)) Best-Normal Best-Normal ((Rosenholtz Rosenholtz 0101’’)) RCref RCref ((Rosenholtz Rosenholtz 0101’’))
dT
T
dT
T
Avraham, Yeshurun & Lindenbaum, Journal of Vision 2008
Study 1Study 1
Manipulated T-D and D-D similarityAccuracy. 2IFC.
Avraham, Yeshurun & Lindenbaum, Journal of Vision 2008
Participant Saliency (Rosenholtz
99)
COVER
Exp 1 A.P. 0.812 0.999*
Y.B. 0.538 0.998*
D.A. 0.572 0.997*
V.S. 0.570 0.999*
A.P.Z. 0.510 1*
Exp 2 A.D. - 0.926*
A.A. - 0.903*
M.D. - 0.962*
L.F. - 0.873
Exp 3 D.A. 0.900 0.862
S.M. 0.945 0.883
E.D. 0.996* 0.993*
G.S. 0.880 0.997*
Exp 4 R.A. - 0.967*
O.R. - 0.956*
R.I. - 0.979*
A.O. - 0.951*
COVER: Prediction COVER: Prediction ComparisonComparison
Avraham, Yeshurun & Lindenbaum, Journal of Vision 2008
Good correlation between
accuracy and COVER
correlation between accuracy and measure
FLNN - Prediction FLNN - Prediction ComparisonComparison
Avraham, Yeshurun, Lindenbaum VSS 2011
FLNN best in 2 test.Lowest 2/df values
Avraham, Yeshurun & Lindenbaum, Journal of Vision 2008
Study 1 SummaryStudy 1 Summary
► FLNN and COVER predict T-D and D-D similarity FLNN and COVER predict T-D and D-D similarity effects better than other prominent effects better than other prominent computational models.computational models.
► The models quantify grouping-by-similarity The models quantify grouping-by-similarity involved in visual search, by suggesting that the involved in visual search, by suggesting that the degree of within-group heterogeneity depends degree of within-group heterogeneity depends on the T-D similarity.on the T-D similarity.
Avraham, Yeshurun, Lindenbaum VSS 2011
Study 2:Study 2:
Spatial Organization Effects Spatial Organization Effects
test and model how the effects of grouping by proximity and grouping by similarity
are combined in the context of visual search
Manipulating Spatial Manipulating Spatial OrganizationOrganization
Experiment 1Experiment 1T=0° D = 23°, 47°, 70°T=0° D = 23°, 47°, 70°
The same 30 elements in all conditionsThe same 30 elements in all conditions
Avraham, Yeshurun, Lindenbaum VSS 2011
no clusters 6 clusters 3 clusters
condition 1 condition 2 condition 3
Does spatial organization matter?
Experiment 1: ResultsExperiment 1: Results
► Previous models do not account for this significancePrevious models do not account for this significance
Avraham, Yeshurun, Lindenbaum VSS 2011
Spatial Organization Effects - Spatial Organization Effects - ModelingModeling
► One possibility: a multi scale approachOne possibility: a multi scale approach((e.g., Itti et al. 1998, Rosenholtz et al. 2007e.g., Itti et al. 1998, Rosenholtz et al. 2007))
How to combine the measure over scales? How to combine the measure over scales?
max? weight and sum?max? weight and sum?Avraham, Yeshurun, Lindenbaum VSS 2011
COVER=3
COVER=2
COVER=3
no clusters no clusters 3 3 clustersclusters
Spatial Organization Effects – Spatial Organization Effects – ModelingModeling
► Our models need only some measure of Our models need only some measure of distance between each two elementsdistance between each two elements
► Combine Combine feature difference feature difference and and spatial spatial distance distance
Avraham, Yeshurun, Lindenbaum VSS 2011
,i jD
feature, spatial(1 )i jD d d
Indicates the relative effect of the forces
feature,i jD d
Spatial Organization Effects – Spatial Organization Effects – ModelingModeling
► AdvantagesAdvantages: : same treatment for similarity and proximity same treatment for similarity and proximity understand and quantify the relative effect of eachunderstand and quantify the relative effect of each
► Questions to answer in this study:Questions to answer in this study: Will this enable our models to account for the Will this enable our models to account for the
spatial organization effect?spatial organization effect? What is the value of ?What is the value of ? Is stable or stimuli dependent?Is stable or stimuli dependent?
Avraham, Yeshurun, Lindenbaum VSS 2011
, feature spatial(1 )i jD d d
FLNN predictionsFLNN predictions
► Conclusions:Conclusions: Combination of feature distance and spatial distance is Combination of feature distance and spatial distance is
essential for predictionessential for prediction Limited possibilities: implies that the model is informativeLimited possibilities: implies that the model is informative Relates to previous findings regarding the combined effects Relates to previous findings regarding the combined effects
of proximity and similarity on perceptual groupingof proximity and similarity on perceptual grouping
((e.g., Kobovy & van den Berg 2008e.g., Kobovy & van den Berg 2008))Avraham, Yeshurun, Lindenbaum VSS 2011
0.35 prediction prediction withwith
predictive ability vs. predictive ability vs.
Maximum value of 2 to pass the 2 test
Preliminary: Is stable Preliminary: Is stable or stimuli dependent?or stimuli dependent?
► Experiment 2: Experiment 2: Manipulate the number of distractor typesManipulate the number of distractor types
► Experiment 3:Experiment 3:Manipulate the distractors varianceManipulate the distractors variance
Avraham, Yeshurun, Lindenbaum VSS 2011
Exp 2 (preliminary): Exp 2 (preliminary): manipulating the number of manipulating the number of
distractor typesdistractor types
Avraham, Yeshurun, Lindenbaum VSS 2011
2 distractor types(15°, 30°)
4 distractor types(15°, 30°, 45°, 60°)
4 clustersno clusters
Avraham, Yeshurun, Lindenbaum VSS 2011
D = 15°, 30°
clustered
not clustered
Exp 3 (preliminary): Exp 3 (preliminary): manipulating distractors manipulating distractors
variancevarianceD = 15°, 45°D = 15°, 60°
Avraham, Yeshurun, Lindenbaum VSS 2011
Experiment 3 ResultsExperiment 3 ResultsExperiment 2 Experiment 2 ResultsResults
Experiment 2 and 3: Experiment 2 and 3: (preliminary) Results(preliminary) Results
Avraham, Yeshurun, Lindenbaum VSS 2011
0.35 Experiment 2 and 3: Experiment 2 and 3:
FLNN Predictions withFLNN Predictions withExperiment 3 PredictionsExperiment 3 PredictionsExperiment 2 Experiment 2
PredictionsPredictions
SummarySummary► A study of the effects of spatial organization on A study of the effects of spatial organization on
visual searchvisual search
► The FLNN model can predict effects of grouping The FLNN model can predict effects of grouping by similarity and grouping be proximityby similarity and grouping be proximity
► As it uses an explicit combination of feature As it uses an explicit combination of feature difference and spatial distance, it can help us difference and spatial distance, it can help us understand the relative effect of similarity and understand the relative effect of similarity and proximity on visual searchproximity on visual search
Avraham, Yeshurun, Lindenbaum VSS 2011
Thank you Thank you
Tamar AvrahamTamar AvrahamMichael LindenbaumMichael Lindenbaum
Yaffa YeshurunYaffa Yeshurun